Build a Wallet Ledger System at Scale
databases distributed-systems scalability
System Design Deep Dive
Wallet Ledger System at Scale
Every cent must be accounted for. The brutal tension between financial accuracy and throughput - and how to win both.
Think of a general ledger as an accountant’s notebook that is never allowed to have anything crossed out. Every financial event - a payment received, a fee charged, a refund issued - gets written on the next blank line. Pages are never torn out. Nothing is erased. If you made an error in entry 10,000, you do not white it out; you write entry 10,001 that corrects it. This is the core principle of double-entry bookkeeping, and it is not a legacy accounting concept - it is the correct mental model for designing a financial ledger system that will be auditable by regulators, trusted by users, and correct under network failures.
The scale numbers make this design genuinely hard. A mid-size fintech with 10 million active users might see 3,000 transactions per second during peak hours - a Black Friday promotion, a cash-back campaign going viral, a payment deadline at midnight. Each transaction requires writing two ledger entries atomically (a debit and a corresponding credit) and updating two account balances without ever allowing a negative balance or a partial write. At 3,000 TPS, that is 6,000 ledger inserts and 6,000 balance updates per second, all needing ACID guarantees. A single slow lock on a popular account (a merchant receiving thousands of micro-payments simultaneously) can cascade into a queue of stalled transactions.
The analogy I use is a busy bank branch with one teller window. Single-threaded serial processing gives perfect correctness - the teller processes each customer completely before the next begins, so no two customers ever touch the same account simultaneously. But correctness at the cost of throughput is not a business outcome. The real design challenge is equivalent to opening 16 teller windows while guaranteeing they cannot accidentally give the same dollar to two customers. Partitioning by account identifier is the key, but it breaks down the moment a transaction spans two accounts on different partitions.
The three core architectural decisions that cascade from this: first, whether to compute balances on the fly by summing ledger entries or to maintain a materialized balance with its own locking strategy. Second, how to partition the ledger to eliminate cross-shard transactions while supporting multi-party payments. Third, how to build an idempotency layer robust enough that a retried network request never posts a duplicate debit.
Requirements and Constraints
Functional Requirements
- Record every credit and debit as an immutable ledger entry with a unique entry ID
- Enforce double-entry bookkeeping: every transaction produces a debit entry and a corresponding credit entry, and their amounts must be equal
- Provide real-time balance reads: GET /balance returns the current available balance within 50ms P99
- Prevent negative balances: a debit must be rejected if the available balance is insufficient
- Support idempotent transaction submission: retrying the same request with the same idempotency key must not create duplicate entries
- Expose a full transaction history: paginated list of ledger entries per account with timestamps and reference IDs
Non-Functional Requirements
- Write throughput: 5,000 transactions per second sustained, 15,000 TPS peak for 30-second bursts
- Read throughput: 50,000 balance reads per second (balance checks are called before every payment)
- Write latency: P99 under 150ms for a committed transaction (both ledger entries + balance update)
- Read latency: P99 under 50ms for balance reads; P99 under 500ms for full transaction history
- Durability: zero tolerance for data loss - RPO = 0, achieved via synchronous replication
- Availability: 99.99% uptime (52 minutes downtime per year maximum) on the write path
- Audit trail: every ledger entry immutable and cryptographically chained for regulatory compliance
- Accounts: support up to 100 million active accounts across 16 ledger shards
Constraints and Scope
- Currency handling is in integer minor units (paise, cents) to avoid floating-point rounding
- Cross-currency conversion is out of scope - each wallet holds one currency denomination
- Fraud detection is a downstream consumer of the event stream, not inline with transaction commits
- The system does not handle card authorization holds - only settled transactions are recorded
High-Level Architecture
The system comprises five layers working together. The API Gateway handles rate limiting, authentication, and routes requests to stateless Transaction Service instances. The Transaction Service is the orchestrator - it validates input, checks idempotency, and drives the Double-Entry Engine that writes the paired ledger entries. The Ledger Storage Layer holds the immutable append-only ledger shards and the materialized balance table. The Balance Cache (Redis Cluster) absorbs the 50K RPS read load so balance checks never hit the primary database. The Event Stream (Kafka) decouples downstream consumers - audit trail, notifications, analytics, and reconciliation - from the synchronous write path.
The critical design choice is that the synchronous write path only touches two things: the ledger table (INSERT) and the balances table (UPDATE with optimistic lock). Everything else - audit archiving, notification delivery, fraud scoring - is asynchronous and reads from the Kafka event stream. This keeps the P99 write latency under 150ms even as downstream consumers grow.
Premium Content
Unlock the full article along with everything else in the archive — all in one place.