The CAP theorem, actually explained
You cannot choose CA in distributed systems because network partitions are unavoidable. The real architectural tradeoff is between CP and AP during a network split.
The CAP Theorem (formulated by Eric Brewer) is one of the most cited—and most misunderstood—principles in distributed computing.
It states that a distributed data store can guarantee at most two out of three guarantees:
Consistency (C)
▲
/ \
/ \
/ \
/ P \
/ is a law\
/ of physics\
/ \
Availability (A) ────────── Partition Tolerance (P)
[CP] vs [AP]
- Consistency (C): Every read receives the most recent write or returns an error. (Strict linearizability).
- Availability (A): Every non-failing node returns a successful response for every request—without guarantee that it contains the latest write.
- Partition Tolerance (P): The system continues to operate despite network messages being dropped, delayed, or severed between nodes.
The Myth of “CA” Distributed Systems
Textbooks often introduce CAP as a menu where you “pick any two”: CA, CP, or AP.
In the real world, “CA” does not exist for distributed systems.
Networks are physical infrastructure: fiber cables get cut, routers crash, packets drop, and garbage collection pauses introduce timeouts. Network partitions (P) are unavoidable.
Therefore, when a network split inevitably occurs, you are forced to choose between CP and AP.
What Happens During a Network Partition?
Imagine two database nodes, Node 1 in New York and Node 2 in London, with a severed fiber connection between them. A user sends a write to Node 1 (balance = $100):
[Client Write: $100]
│
▼
┌─────────────┐ ┌─────────────┐
│ Node 1 │ X-- PARTITION --X │ Node 2 │
│ ($100 saved)│ │ (Stale data)│
└─────────────┘ └─────────────┘
▲
│
[Client Read: ???]
Choice 1: CP (Prioritize Consistency)
- Node 2 realizes it cannot communicate with Node 1 to verify the latest state.
- When a read arrives at Node 2, it fails fast and returns an error (HTTP 503 / Timeout) instead of serving stale information.
- Result: No client ever sees outdated or contradictory state. System is consistent, but unavailable on partitioned nodes.
- Used by: Financial systems, bank ledgers, inventory systems, PostgreSQL synchronous replication, Google Cloud Spanner, etcd.
Choice 2: AP (Prioritize Availability)
- Node 2 accepts the read request and immediately serves its local data, even though it may be stale.
- Node 2 may also accept new writes, storing conflicting versions that must be reconciled later (e.g., Last-Write-Wins, CRDTs, or vector clocks).
- Result: The system achieves 100% uptime, but clients may read stale data temporarily (Eventual Consistency).
- Used by: Social media feeds, YouTube view counts, shopping carts (Amazon DynamoDB), DNS, global CDNs, Apache Cassandra.
Architectural Comparison
| Dimension | CP System | AP System |
|---|---|---|
| Response on Split | Error / 503 / Timeout | Stale / Cached Data |
| Consistency Model | Linearizable (Strict) | Eventual Consistency |
| Write Availability | Blocked if quorum lost | Always accepts writes |
| Conflict Resolution | Prevented at write time | Reconciled asynchronously |
| Primary Domain | Banking, auth, inventory | Analytics, feeds, metrics |
Summary
When designing distributed architectures, remember: You do not choose whether to have Partition Tolerance; the network chooses it for you. Your design decision is strictly how your system behaves when that partition happens—fail to protect consistency (CP), or answer to preserve uptime (AP).
Test Your Understanding
Quick architectural scenario challenge based on this breakdown.
A 3rd-party payment gateway starts timing out with 504 errors during peak traffic.
Q1.How should a downstream dependency failure be handled to protect upstream service availability?
Comments –
Sign in to share your thoughts
Nothing yet. Be the first.