Decentralized Systems · IPFS Decentralized Storage

Key Question

What are the layers of the IPFS protocol and how do they work together?

Deep Dive

IPFS isn’t one protocol — it’s a stack of four protocols, each solving a different problem. Think of it like the TCP/IP stack, but for content-addressed peer-to-peer networks.

IPFS Protocol Stack:

┌──────────────────────────────────────────┐
│              IPLD                         │  ← Data format layer
│  (InterPlanetary Linked Data)             │    (Merkle DAG, CIDs)
├──────────────────────────────────────────┤
│              BitSwap                      │  ← Block exchange layer
│  (Block trading protocol)                 │    (want list / have list)
├──────────────────────────────────────────┤
│              DHT                          │  ← Routing layer
│  (S/Kademlia Distributed Hash Table)      │    (find peers for a CID)
├──────────────────────────────────────────┤
│              libp2p                       │  ← Network layer
│  (Peer identity, transport, NAT, crypto)  │    (TCP, QUIC, WebSockets)
└──────────────────────────────────────────┘

Layer 1: libp2p — Everything below

libp2p handles peer-to-peer networking. Each peer has a PeerID = hash of their public key. This is a self-authenticating identity: if someone claims to be PeerID QmPeer…, they prove it by signing a message with their private key.

libp2p transports:
┌────────────┐  ┌────────────┐  ┌────────────┐
│    TCP     │  │    QUIC    │  │ WebSocket  │
│ (reliable) │  │ (fast)     │  │ (browser)  │
└────────────┘  └────────────┘  └────────────┘
        │              │              │
        └──────────────┴──────────────┘
                       │
                  ┌────┴────┐
                  │  Noise  │ ← Encrypted, authenticated
                  │  TLS    │    communication
                  └─────────┘

libp2p also handles NAT traversal (hole-punching), relay connections (for peers behind NAT), and multi-addresses (a format that describes how to reach a peer: /ip4/1.2.3.4/tcp/4001/p2p/QmPeerID).

Layer 2: DHT (S/Kademlia) — Find who has the data

IPFS uses a Distributed Hash Table based on S/Kademlia. Each peer is responsible for a portion of the keyspace (the hash space of CIDs). The DHT stores Provider Records: which peerIDs have announced they hold a given CID.

DHT lookup for CID QmX...:

Peer A wants QmX...
  │
  ├──► Ask closest peers in keyspace: "Who has QmX...?"
  │
  ├──► Those peers respond: "Peer Z has it" OR refer to closer peers
  │
  └──► Eventually find Peer Z (or discover it doesn't exist)

Lookup cost: O(log n) network hops for n peers in the network.

Layer 3: BitSwap — Trade blocks like a market

Once you know who has the data, BitSwap handles the exchange. BitSwap is modeled as a barter market:

BitSwap exchange:

Peer A wants: [QmX, QmY]
Peer A has:   [QmZ, QmW]

Peer A → Peer B: "Here's my want list [QmX, QmY]. Here's what I have [QmZ, QmW]."
Peer B → Peer A: "I have QmX. Send me QmZ and it's yours."

Peer A sends QmZ → Peer B sends QmX → both verify hashes.

Incentive: Peers track a "credit" score. If you never share, others deprioritize you.

This is fundamentally different from BitTorrent: BitSwap is a pairwise negotiation (not a swarm with a tracker), and peers can decide which blocks to trade based on their own wants.

Layer 4: IPLD — Make sense of the data

IPLD (InterPlanetary Linked Data) is a data model that turns any Merkle DAG into a universal format. A CID + a path lets you traverse the DAG:

IPLD path traversal:

ipfs cat QmRootTree/src/main.go

QmRootTree (tree node)
     │
     ├── "src" ───► QmSubTree (tree node)
     │                   │
     │                   └── "main.go" ───► QmBlob (actual file bytes)
     │
     └── result: contents of main.go

IPLD is designed to work across systems: blockchain state (Ethereum state tries), Git objects, and IPFS files all use the same underlying Merkle DAG model.

Walk through ipfs get QmX...:

libp2p: Connect to the IPFS network. Dial peers via TCP/QUIC.
DHT: Ask the DHT “who has QmX…?” Get back a list of peer IDs.
BitSwap: Connect to those peers. Send a want list: [QmX…]. Negotiate.
BitSwap: Receive the raw block data for QmX… (could be a blob, list, or tree node).
IPLD: If QmX… is a list node, recursively fetch children. If a tree node, parse the directory.
Verification: Hash each received block. Confirm it matches the requested CID.

Check Your Understanding

How does PeerID serve as a self-authenticating identity?
What’s the difference between BitSwap and BitTorrent’s tit-for-tat?
What happens if the DHT doesn’t have a provider record for a CID you request?

The “So What?”

IPFS’s layered design (libp2p → DHT → BitSwap → IPLD) is a blueprint for decentralized systems. Each layer solves one problem and can be reused independently. libp2p, for instance, is used by Ethereum 2.0, Polkadot, and Filecoin — proving that good protocol design outlives any single application.

✏️ Exercises

IPFS & Decentralized Storage: Exercises

Exercise 1: CID Mutability

Alice creates a file hello.txt with content “Hello, world!” and adds it to IPFS. She gets CID QmPZ9gcCe5rsLKbrJfFQW9dLLgKNLoJN7da8uDNmhCWZqJ. She then changes the file to “Hello, world?” (changing ! to ?) and adds it again.

Does the CID change? Why or why not?
Bob downloads both files. How can he verify which one is the original?
If Alice wants to share a link that always points to her latest version, what IPFS mechanism does she need?

Exercise 2: IPFS vs BitTorrent

Consider downloading a 2 GB open-source operating system ISO. Compare IPFS and BitTorrent:

Discovery: How does each system find peers who have the file?
Verification: How does each system verify that downloaded data is correct?
Incentives: How does each system encourage peers to upload after downloading?
Merkle structure: Both BitTorrent and IPFS use a Merkle tree / DAG. Is there a conceptual difference in how they structure and address data?

Exercise 3: PoRep Necessity

Filecoin miners earn money by storing clients’ data. A dishonest miner considers the following attack:

Client wants to store 100 copies of a 1 GB dataset D.
Miner stores 1 copy of D and claims to have 100.
When challenged with PoRep, miner quickly generates the sealed data from the single copy.

Explain why this attack fails due to the design of Proof-of-Replication. Be specific about the sealing process and what makes each sealed copy unique to a specific miner and a specific deal.

Bonus: Could this attack work if Filecoin used only Proof-of-Spacetime without Proof-of-Replication?

👁️ View Solutions

IPFS & Decentralized Storage: Solutions

Exercise 1 Solution

1. Does the CID change?

Yes. The CID is the cryptographic hash of the file’s content. Changing even one byte completely changes the hash output (avalanche effect). Assuming SHA-256 is the hash function:

Original: SHA-256(“Hello, world!”) → QmPZ9gcCe5rsLKbrJfFQW9dLLgKNLoJN7da8uDNmhCWZqJ
Modified: SHA-256(“Hello, world?”) → completely different CID

The two CIDs share no relationship. You cannot derive one from the other.

2. How to verify which is original?

Download each file and compute the hash. If the hash matches the CID claimed by Alice, you have the file she intended. Since the CIDs are different, you can tell they’re different files. Without Alice telling you which CID is the “original,” you can’t know the authorial intention — but you can be certain about the content.

3. Always pointing to the latest version?

Alice needs IPNS (InterPlanetary Name System). IPNS creates a pointer from Alice’s PeerID (public key hash) to a CID. Alice can update the pointer: ipns://QmAlicePeerID always resolves to her latest CID. Users who trust Alice’s public key will always get her latest file.

Exercise 2 Solution

1. Discovery:

BitTorrent	IPFS
Centralized tracker or DHT with infohash	Kademlia DHT keyed by CID
Tracker returns list of peers	DHT returns provider records
PEX (Peer Exchange) for gossip	BitSwap handles peer discovery during exchange

BitTorrent traditionally relied on trackers (centralized). Modern BitTorrent uses DHT (Mainline DHT), which inspired IPFS’s DHT. IPFS’s approach is fully decentralized from the start.

2. Verification:

BitTorrent	IPFS
Merkle tree: root hash (infohash), 256 KB piece hashes	Merkle DAG: every node is content-addressed
Verify each piece against its hash in the torrent metadata	Verify each block against its CID on download
Root hash in .torrent file or magnet link	CID is the root hash

Both use Merkle verification. The key difference: BitTorrent’s Merkle tree is flat (one level of pieces), while IPFS’s Merkle DAG can be nested (trees within trees).

3. Incentives:

BitTorrent	IPFS
Tit-for-tat: “I’ll only upload to you if you upload to me”	BitSwap barter: “I’ll trade blocks I have for blocks I want”
Strict: peer is choked if they don’t reciprocate	Soft: based on credit/debit ratios
Leeching is directly punished	Leeching is indirectly punished (credit score drops)

BitTorrent’s tit-for-tat is more aggressive about enforcing sharing. IPFS’s BitSwap is more flexible — a peer with low credit can still fetch data, just at lower priority.

4. Merkle structure difference:

BitTorrent’s Merkle tree is a static structure: the piece list is fixed when the torrent is created. You cannot add files or reorganize without creating a new torrent.

IPFS’s Merkle DAG is a dynamic structure: you can add files, create directories, and link objects arbitrarily. IPFS directories are DAG nodes; BitTorrent has no concept of directory hierarchy in its data structure.

Exercise 3 Solution

Why the attack fails:

Proof-of-Replication involves sealing — a sequential, resource-intensive encoding process that ties a specific copy of the data to a specific miner:

Sealing process (simplified):

Original data D
      │
      ├──► Miner ID (M) ──────┐
      ├──► Deal ID (Deal) ─────┤
      ├──► Random nonce ───────┤
      └────────────────────────┘
               │
               ▼
         Layer-by-layer encoding (AES + PoSW)
               │
               ▼
         Sealed sector S = Seal(D, M, Deal, nonce)
         Time: ~30 minutes per sector

Each sealed copy S is different because:

The miner ID is different (each miner has a unique public key)
The deal ID is different (each deal is a separate contract)
The nonce is different (random value per deal)

For the miner claiming 100 copies stored:

They would need 100 different sealed sectors: S₁, S₂, …, S₁₀₀
Each requires ~30 minutes of sequential computation
They cannot compute 100 proofs from 1 copy because the sealing input (miner ID, deal ID) differs per deal
PoRep challenges ask about the sealed data, which is unique per copy

If caught cheating (unable to produce the correct PoRep), the miner’s entire collateral for all 100 deals is slashed.

Bonus: Without PoRep (PoSt only):

The attack would likely succeed. PoSt proves that you currently hold some data, but it doesn’t prove that the data is a unique copy. With PoSt alone:

Store 1 copy of D
Generate PoSt for that one copy
Have the same PoSt serve as proof for all 100 deals (since PoSt just proves “I have this hash”)
Collect 100× payment for 1× storage

This is why PoRep is necessary: it binds each deal to a physically distinct, computation-bound encoding.