Distributed Transactions Coordination · Distributed Transactions 2PC

Key Question

What makes a transaction distributed, and why is it harder than a local transaction?

Deep Dive

A transaction is distributed when it touches data on more than one node. On a single machine, ACID is hard enough. Across machines, every guarantee frays.

On a single-node database, atomicity is solved by a write-ahead log (WAL): write the changes to a durable log before applying them. If the machine crashes, the log tells you what to replay. Isolation is solved by locks held in memory. Both rely on shared memory and a single point of failure — the machine.

In a distributed database, a single transaction may read X from Node A and write Y to Node B. No shared memory. No shared clock. If Node A commits but Node B crashes before committing, you have a partial commit — atomicity is violated.

Transaction T: READ(X) from Node 1, WRITE(Y=42) to Node 2

    Client
      |
      | BEGIN TX
      |----------------------+
      |                      |
      v                      v
   Node 1                   Node 2
   [READ X]                 [WRITE Y=42]
      |                        |
   "OK, ready"             *CRASH*  <-- partial commit!
      |
   Time passes...
      |
   Node 1 keeps its locks    Node 2 lost everything

This is the distributed atomicity problem. The coordinator needs a protocol to make sure all or nothing happens.

Check Your Understanding

What is a partial commit?
Why can’t a distributed database use a single write-ahead log for atomicity?
What shared resource exists on a single-node DB but not on a multi-node DB?

The “So What?”

Without distributed transactions, banks can’t transfer money between accounts on different shards, shopping carts can’t deduct inventory across warehouses, and any multi-step operation across services would risk data corruption. Every system that spans machines needs this.

✏️ Exercises

Exercises: Distributed Transactions & 2PC

2PC Log State: In the Two-Phase Commit protocol, what exact state does a participant write to its local durable log before responding “YES” to the coordinator’s prepare request? Why must this state be written to a log rather than kept in memory?
Blocking Explained: Consider a 2PC execution where the coordinator crashes after receiving all “YES” responses but before any participant has received the commit/abort decision. Participant 1 is running, Participant 2 is running, and Participant 3 is also running. Can any of these participants safely decide on their own? Explain what happens to each.
Distributed 2PL and Phantoms: A distributed database uses Two-Phase Locking (2PL) for concurrency control. Transaction T1 runs a query: SELECT * FROM accounts WHERE balance > 1000 across two shards. Transaction T2 inserts a new row with balance = 2000 on shard 2. How can a phantom read occur, and what mechanism (in addition to 2PL) is needed to prevent it?

👁️ View Solutions

Solutions: Distributed Transactions & 2PC

Exercise 1

The participant writes PREPARED (or READY) to its log, along with the transaction’s writes (enough information to commit later). This must be durable (fsynced to disk) before the YES response is sent.

Why a log instead of memory: If the participant crashes after sending YES but before receiving the coordinator’s decision, memory is lost. On recovery, the participant reads its log. If the log says PREPARED, it knows it made a promise and must wait for the coordinator. If the log says COMMIT or ABORT, it follows that. Without the log, the participant would have no memory of the promise after restarting.

Exercise 2

No participant can safely decide on their own.

Participant 1 received the PREPARE, said YES, and is now waiting. It has not received a COMMIT or ABORT. It holds locks and is blocked until the coordinator recovers.
Participant 2 is in the same state — blocked.
Participant 3 is also in the same state — blocked.

If any participant unilaterally commits, it might violate atomicity: the coordinator might have decided to abort (e.g., if it suspected a timeout from one participant). If any participant unilaterally aborts, it might violate atomicity: the coordinator might have logged COMMIT before crashing.

All three participants are blocked until the coordinator recovers and reveals the decision.

Exercise 3

Phantom read scenario:

T1’s query SELECT * FROM accounts WHERE balance > 1000 on shard 2 returns no rows.
T2 inserts a new row with balance = 2000 on shard 2 and commits.
If T1 re-runs the same query, it now sees the new row — a “phantom” that appeared from nowhere.

Standard 2PL (locking individual rows) does not prevent this because the new row didn’t exist when T1 first read.

Solution: A predicate lock or next-key lock is needed. In a distributed setting, this means locking the range balance > 1000 on each shard. The lock prevents T2 from inserting rows matching that predicate. In practice, many systems use serializable snapshot isolation (SSI) instead, which detects phantoms at commit time by checking for conflicting predicate reads.