OwlTheEngineer
System Design in 60 SecondsEpisode 33 · System design

The CAP theorem, actually explained

You cannot choose CA in distributed systems because network partitions are unavoidable. The real architectural tradeoff is between CP and AP during a network split.

Owl · 2 min read · 0 views
– comments
Ep 33 · System design
60s VISUAL BREAKDOWN
Episode 33
Watch the fast visual breakdown or discuss on social:

The CAP Theorem (formulated by Eric Brewer) is one of the most cited—and most misunderstood—principles in distributed computing.

It states that a distributed data store can guarantee at most two out of three guarantees:

                            Consistency (C)
                                  ▲
                                 / \
                                /   \
                               /     \
                              /   P   \
                             / is a law\
                            / of physics\
                           /             \
            Availability (A) ────────── Partition Tolerance (P)
                          [CP] vs [AP]
  • Consistency (C): Every read receives the most recent write or returns an error. (Strict linearizability).
  • Availability (A): Every non-failing node returns a successful response for every request—without guarantee that it contains the latest write.
  • Partition Tolerance (P): The system continues to operate despite network messages being dropped, delayed, or severed between nodes.

The Myth of “CA” Distributed Systems

Textbooks often introduce CAP as a menu where you “pick any two”: CA, CP, or AP.

In the real world, “CA” does not exist for distributed systems.

Networks are physical infrastructure: fiber cables get cut, routers crash, packets drop, and garbage collection pauses introduce timeouts. Network partitions (P) are unavoidable.

Therefore, when a network split inevitably occurs, you are forced to choose between CP and AP.


What Happens During a Network Partition?

Imagine two database nodes, Node 1 in New York and Node 2 in London, with a severed fiber connection between them. A user sends a write to Node 1 (balance = $100):

       [Client Write: $100]
                 │
                 ▼
          ┌─────────────┐                      ┌─────────────┐
          │   Node 1    │   X-- PARTITION --X  │   Node 2    │
          │ ($100 saved)│                      │ (Stale data)│
          └─────────────┘                      └─────────────┘
                                                      ▲
                                                      │
                                             [Client Read: ???]

Choice 1: CP (Prioritize Consistency)

  • Node 2 realizes it cannot communicate with Node 1 to verify the latest state.
  • When a read arrives at Node 2, it fails fast and returns an error (HTTP 503 / Timeout) instead of serving stale information.
  • Result: No client ever sees outdated or contradictory state. System is consistent, but unavailable on partitioned nodes.
  • Used by: Financial systems, bank ledgers, inventory systems, PostgreSQL synchronous replication, Google Cloud Spanner, etcd.

Choice 2: AP (Prioritize Availability)

  • Node 2 accepts the read request and immediately serves its local data, even though it may be stale.
  • Node 2 may also accept new writes, storing conflicting versions that must be reconciled later (e.g., Last-Write-Wins, CRDTs, or vector clocks).
  • Result: The system achieves 100% uptime, but clients may read stale data temporarily (Eventual Consistency).
  • Used by: Social media feeds, YouTube view counts, shopping carts (Amazon DynamoDB), DNS, global CDNs, Apache Cassandra.

Architectural Comparison

DimensionCP SystemAP System
Response on SplitError / 503 / TimeoutStale / Cached Data
Consistency ModelLinearizable (Strict)Eventual Consistency
Write AvailabilityBlocked if quorum lostAlways accepts writes
Conflict ResolutionPrevented at write timeReconciled asynchronously
Primary DomainBanking, auth, inventoryAnalytics, feeds, metrics

Summary

When designing distributed architectures, remember: You do not choose whether to have Partition Tolerance; the network chooses it for you. Your design decision is strictly how your system behaves when that partition happens—fail to protect consistency (CP), or answer to preserve uptime (AP).

SYSTEM DESIGN CHECK

Test Your Understanding

Quick architectural scenario challenge based on this breakdown.

SCENARIO

A 3rd-party payment gateway starts timing out with 504 errors during peak traffic.

Q1.How should a downstream dependency failure be handled to protect upstream service availability?

Comments –