Where Can Bottlenecks Hide? Future Improvements to Watch For

When a system is first built, performance usually isn't the main concern.

The primary goal is simply making the system work correctly: a request comes in, the backend processes it, data is read from the database, and a response is sent back.

But as the system grows, things change.

The number of users increases, more requests arrive at the same time, the database handles more queries, dependencies on third-party services appear, and parts of the system that were once completely unnoticeable start to determine how fast the whole thing runs.

This is exactly where the concept of a bottleneck comes in.

A bottleneck is the slowest, most heavily loaded part of a system. And here's the tricky part: it's rarely where you expect it to be.

Sometimes the problem isn't the CPU — it's the database connection pool. Sometimes it's not the database itself, but several services writing to the same table. And sometimes the whole system runs fast, except a third-party API takes 2 seconds to respond, so the user still feels the delay.

That's why, when thinking about future improvements, \"just add a bigger server\" simply isn't a good enough answer.

1. The database is one of the biggest suspects

One of the most common bottlenecks in backend systems is the database.

In the beginning, a single request might only need 2–3 SQL queries. But as the system grows, that same endpoint gets used more often, and the load on the database climbs fast.

100 requests/second × 5 SQL queries = 500 database queries/second

If even one of those queries involves a heavy JOIN, a large table scan, or is missing a proper index, the problem is no longer sitting on the application server.

The first improvement here isn't switching databases.

First, you need to:

• identify slow queries

• analyze the execution plan

• cut unnecessary queries

• optimize indexes

• properly configure the connection pool

• implement pagination

• prevent the same data from being fetched from the database over and over

This matters even more in microservice systems that share a single database — the load one service creates can spill over and affect the others. AWS itself warns about this, highlighting the risk of \"hot tables\" and runtime coupling between services in a shared-database setup.

2. Reading everything from the database with no cache

Another potential bottleneck is fetching heavily-read data from the database every single time.

For example, some piece of data might get read thousands of times per second, yet it barely changes from one second to the next.

In that case, it makes far more sense to use a cache instead of sending every request straight to the database.

Before: the database carries every single request.

After: the cache absorbs most of that database load.

AWS notes that placing a cache between the application server and the database reduces database read load and improves latency.

The cache-aside pattern fits perfectly here: check the cache first, and only hit the database on a cache miss.

3. Synchronous communication can create delays down the road

In a microservice architecture, another potential bottleneck is communication between services.

Client → Order Service → Customer Service → Payment Service → Notification Service

At first glance, nothing here looks problematic.

But if the Order Service waits for a response from every single service before returning its own response, the slowest service ends up dictating the speed of the entire request.

• Customer Service → 50 ms

• Payment Service → 100 ms

• Notification Service → 800 ms

In this case, the 800 ms taken by the notification step can force the user to wait for the whole request.

Yet notifications often aren't even required to return a response to the user.

4. The connection pool can be a bottleneck too

Sometimes the application server's CPU sits at 30%, memory looks fine, and the database looks fine too.

Yet requests are still waiting.

The cause could be the connection pool.

For instance, the application might only be able to hold 20 database connections, while 200 requests are trying to reach the database at the same time.

The server's CPU can be practically idle, but if requests are stuck waiting for a connection, the system is still slow.

This takes the conversation beyond \"slow database queries\" and into a much more realistic production perspective.

5. External services are outside our control

Another interesting bottleneck in production systems is external dependencies.

Our Backend → Payment API, SMS Provider, Email Provider, External API

Our own application might respond in 50 ms.

But if an external API takes 1.5 seconds to reply, the user will experience that delay as if it were our system being slow.

The slowest dependency can define the user experience.

6. When a single database is no longer enough

As a system grows, you may eventually need to split up the database load as well.

The goal isn't to make the database bigger. The goal is to separate the read workload from the write path.

7. The most important improvement: seeing the bottleneck coming

In reality, the most important part of future improvements isn't picking a specific technology.

The core challenge is being able to measure exactly where the system is slowing down.

That's where observability comes in.

If a request takes 900 ms, the real question isn't \"is it slow?\" The real question is \"why?\"

• 620 ms — database

• 180 ms — external API

• 90 ms — connection

• the rest — application logic?

Distributed tracing exists precisely to show how much time a request spends across different services and components. AWS's microservices guidance also lists monitoring, centralized logging, and distributed tracing as core pillars of observability.

Conclusion

A system's bottleneck might be the database today, but tomorrow the database might not be the problem at all.

As the number of users grows, the bottleneck can shift to the application server, the connection pool, the network, a queue, or an external dependency.

That's why it's better to think about future improvements this way from the start:

Measure → Identify → Optimize → Scale

First, measure.

Then find out exactly where the problem is.

Then optimize that specific part.

Only after that, if it's genuinely needed, move on to horizontal scaling, caching, queues, read replicas, sharding, or more complex architectures.

Good architecture isn't the one that uses the most technology.

Good architecture is the one where you can actually see where a problem originates, and solve it with just the right amount of complexity — no more, no less.

The main directions for future improvement in this kind of system flow naturally from this idea:

database optimization → caching → asynchronous processing → connection pool tuning → dependency isolation → observability

None of these need to be adopted just because \"we might be a big system someday.\"

As the system grows, the metrics themselves will show you where the next bottleneck is going to be.

And that's really the whole point of good engineering:

Not making the system needlessly complex ahead of time, but growing the right part, in the right way, exactly when it's actually needed.

You can find more articles here:
https://aladdinbiyabangerd.site/en/writing