The 4 Core Challenges of Web System Design

Before diving into the building blocks of distributed systems, it's important to understand the problems those building blocks are designed to solve.

When you're building a small application with a handful of users, most challenges are purely coding problems. A single server, a single database, and straightforward application logic are often enough.

Things change once your application starts serving thousands or millions of users.

At scale, new challenges emerge that can't be solved simply by writing better code. Instead, they require architectural decisions and distributed system techniques.

Fortunately, these challenges tend to repeat across almost very large-scale application. Whether you're building a social network, an e-commerce platform, a streaming service, or a search engine, you'll encounter many of the same problems.

In this article, we'll explore four of the most common challenges in web system design and the techniques commonly used to solve them.

Challenge 1: Too Many Concurrent Users

The most obvious challenge is handling a large number of users at the same time.

Every server and database has a limit on how many requests if can process per second. When that limit is reached, response times increase, requests begin to fail, and the user experience deteriorates.

If you've ever build a simple demo application, you've probably deployed it on a single server. That works perfectly for development and small-scale usage. However, once traffic grows significantly, a single machine becomes a bottleneck.

The Solution: Replication

Instead of relying on a single server, we create multiple copies of it and distribute incoming traffic across them.

This technique is known as load balancing.

Rather than serving every request from one machine, a load balancer spreads requests across many identical application servers.

The same idea can be applied to databases. By creating read replicas, we can distribute queries across multiple machines and reduce the load on the primary database.

The core ideas is simple:

If one machine isn't enough, use many machines.

Replication is one of the fundamental techniques used in distributed systems because it improves both scalability and reliability.

Challenge 2: Too Much Data

As user traffic grows, data volume grows with it.

At some point, the dataset becomes too large to fit comfortably on a single machine.

Consider services such as:

• Google storing and indexing billions of web pages
• Netflix storing metadata for enormous content libraries
• Social media platforms storing years of user-generated content

Even the largest servers have storage and memory limits.

The Solution: Sharding

Instead of storing all data on one machine, we divide it into smaller partitions and distribute those partitions across multiple machines.

This technique is called sharding.

For example, a social media platform might partition data by user ID. All data related to User A could be stored on one database server, while User B's data lives on another.

As more users join the platform, additional shards can be added to accommodate growth.

Sharding allows systems to scale beyond the physical limits of a single machine.

Challenge 3: Keeping the System Fast

Users expect modern applications to respond almost instantly.

Even when a system can handle large amounts of traffic and data, it still needs to feel responsive.

Research consistently shows that users become noticeably less satisfied as response times increase. Delays measured in seconds can make an application feel slow and unreliable.

The challenge becomes especially difficult when operations involve complex processing.

Examples include:

• Processing payments
• Generating reports
• Sending notifications
• Uploading videos
• Running recommendation algorithms

These tasks can take seconds or even minutes to complete.

The Solution: Asynchronous Processing

Rather than forcing users to wait for every operation to finish, modern systems often process long-running tasks asynchronously.

A common workflow looks like this:

1. The application receives a request.
2. The request is placed into a message queue.
3. A worker service processes the task in the background.
4. The user receives an immediate response.

Technologies such as Kafka, RabbitMQ, Amazon SQS, and other messaging systems are frequently used to implement this pattern.

The result is a much better user experience because users don't have to wait for expensive operations to complete before continuing.

Challenge 4: Data Consistency

The previous solutions introduce a new problem.

Once data is replicated across multiple servers and updates are processes asynchronously, different parts of the system may temporarily disagree about the current state of the data.

Imagine posting a comment on a social media platform.

The write operation succeeds, but a read request immediately afterward might be served by a replica that hasn't yet received the latest update.

The result isn't usually incorrect data - it's slightly outdated data.

The Solution: Eventual Consistency

Many large-scale systems accept temporary inconsistencies because they provide significant benefits in scalability and performance.

This approach is known as eventual consistency.

The idea is straightforward:

• Different parts of the system may temporarily have different versions of the data.
• Given enough time, all copies will converge to the same state.

For most applications, this trade-off is perfectly acceptable.

If a social media post takes a few seconds to appear everywhere, users generally don't notice or care.

However, some domains cannot tolerate inconsistency.

Example include:

• Banking systems
• Payment processing
• Stock trading platforms

In these cases, correctness is more important than speed, and systems often sacrifice performance to guarantee consistency.

This is one reason financial applications may feel slower than services such as search engines or social networks.

Summary

Most large-scale web systems face four fundamental challenges:

1. Too many concurrent users
2. Too much data
3. The need for low latency
4. data consistency issues

The solutions are surprisingly repetitive:

• Replication helps handle more users.
• Sharding helps manage more data.
• Asynchronous processing keeps applications repsonsive.
• Eventual consistency enables scalable distributed architectures.

Together, these techniques allow modern systems to serve millions of users, manage enormous datasets, and continue scaling by adding more machines rather then rebuilding the entire architecture.

Understanding these challenges is the first step toward understanding the building blocks of distributed system design.