Tuesday morning standup. The product team announces a new feature for Service A - adding a priority column to the tasks table. Simple change, right? Ship it Friday. But then DevOps speaks up: “Service B also uses that table. We need to coordinate the deployment.”

The room goes quiet. Your microservices architecture - the one that promised independent deployments and team autonomy - just revealed its dirty secret. Both services write to the same PostgreSQL schema. A database column change in Service A breaks Service B’s queries. Schema migrations must be synchronized across teams. Deployments require cross-team coordination calls.

It’s like having two separate houses that share the same electrical panel. You want to upgrade your kitchen outlets, but your neighbor has to shut down their home office for the rewiring. The houses look independent from the outside, but they’re coupled at the most fundamental level - the power source that keeps everything running.

This is the shared database anti-pattern in microservices. Independence is an illusion when services couple through storage.

Why This Happens

Most teams adopt microservices for organizational benefits - independent deployments, team autonomy, and technology diversity. But when services share a database schema, they retain all the coupling of a monolith while adding the complexity of distributed systems.

The failure chain is predictable:

Service A needs schema change
  -> Service B queries depend on old schema
    -> Migration breaks Service B
      -> Rollback breaks Service A
        -> Both teams blocked on coordination

The trap emerges gradually. Early in a project, sharing a database feels pragmatic. Both services need user data and order data. Why duplicate it? The shared schema starts simple and manageable. Teams can coordinate easily when there are only two services and three developers.

But as the system grows, the coordination tax compounds. Each new service that touches shared tables adds another dependency. Schema changes require impact analysis across all consuming services. Database migrations become multi-team release events with rollback coordination plans.

The Naive Solution (and where it breaks)

Most teams reach for one of two approaches when they hit this wall:

Option 1: Careful Coordination Schedule all schema changes through a central committee. Create elaborate deployment orchestration. Require cross-team approval for any database modification.

This is like solving traffic jams by adding more traffic lights. The coordination overhead grows quadratically with the number of services. What started as a simple column addition becomes a three-week process involving four teams, two meetings, and a detailed rollback plan.

Option 2: Backward-Compatible Changes Only Never drop columns, never change data types, never add NOT NULL constraints. Only append new columns and tables. Deprecate old fields but never remove them.

The database becomes a museum of historical decisions. After two years, the users table has email, email_address, user_email, and primary_email columns because teams were afraid to clean up deprecated fields. Query performance degrades as tables accumulate zombie columns. New developers can’t tell which fields are actually used.

At 10 services accessing shared tables, backward-compatibility constraints become impossibly complex. You can’t evolve your data model without archaeological research into every consuming service.

Database-Per-Service Pattern

Here’s what actually fixes this: give each service its own database. Not just separate schemas in the same PostgreSQL instance - separate database instances with separate connection strings, separate access controls, and separate migration pipelines.

The database-per-service pattern enforces schema ownership at the infrastructure level. Service A owns its PostgreSQL instance. Service B owns its MongoDB cluster. Neither can directly access the other’s storage.

// Service A: User Management Service
type UserService struct {
    db *sql.DB  // postgres://user-service-db:5432/users
}

func (s *UserService) CreateUser(ctx context.Context, u User) error {
    // Only this service can write to users table
    _, err := s.db.ExecContext(ctx, 
        "INSERT INTO users (id, email, created_at) VALUES ($1, $2, $3)",
        u.ID, u.Email, time.Now())
    return err
}

// Service B: Order Management Service  
type OrderService struct {
    db *sql.DB  // postgres://order-service-db:5432/orders
    userClient UserClient  // HTTP client to Service A
}

func (s *OrderService) CreateOrder(ctx context.Context, o Order) error {
    // Validate user exists via API call, not database join
    user, err := s.userClient.GetUser(ctx, o.UserID)
    if err != nil {
        return fmt.Errorf("invalid user: %w", err)
    }
    
    // Store denormalized user data needed for orders
    _, err = s.db.ExecContext(ctx,
        "INSERT INTO orders (id, user_id, user_email, amount) VALUES ($1, $2, $3, $4)",
        o.ID, o.UserID, user.Email, o.Amount)
    return err
}

This approach eliminates schema coupling at the source. Service A can drop columns, change data types, and migrate to different databases without affecting Service B. Each team controls their own migration schedule.

Data Duplication Tradeoffs

The elephant in the room: data duplication. When Service B needs user information, it can’t join across databases. The order service must store user_email and user_name locally, duplicating data from the user service.

This feels wrong to developers trained on database normalization. But at microservice scale, denormalization becomes a feature, not a bug:

-- Service A: Canonical user data
CREATE TABLE users (
    id UUID PRIMARY KEY,
    email VARCHAR(255) NOT NULL,
    full_name VARCHAR(255) NOT NULL,
    subscription_tier VARCHAR(50),
    created_at TIMESTAMP,
    updated_at TIMESTAMP
);

-- Service B: Denormalized user data for orders
CREATE TABLE orders (
    id UUID PRIMARY KEY,
    user_id UUID NOT NULL,
    user_email VARCHAR(255) NOT NULL,  -- Duplicated for query performance
    user_name VARCHAR(255) NOT NULL,   -- Duplicated for order history display
    amount DECIMAL(10,2) NOT NULL,
    status VARCHAR(50) NOT NULL,
    created_at TIMESTAMP
);

The duplication enables independent queries. Service B can generate order reports, display customer purchase history, and calculate user lifetime value without calling Service A’s API. No network calls, no cascading failures, no coupling.

Handling Data Consistency

But what happens when a user changes their email address in Service A? The old email remains in Service B’s orders table, creating inconsistency.

The solution is eventual consistency through events:

// Service A publishes user changes
func (s *UserService) UpdateEmail(ctx context.Context, userID string, newEmail string) error {
    // Update canonical data
    _, err := s.db.ExecContext(ctx,
        "UPDATE users SET email = $1, updated_at = NOW() WHERE id = $2",
        newEmail, userID)
    if err != nil {
        return err
    }
    
    // Publish event for other services
    event := UserEmailChanged{
        UserID:   userID,
        NewEmail: newEmail,
        Timestamp: time.Now(),
    }
    return s.eventBus.Publish(ctx, "user.email.changed", event)
}

// Service B subscribes to user events
func (s *OrderService) HandleUserEmailChanged(ctx context.Context, event UserEmailChanged) error {
    // Update denormalized data
    _, err := s.db.ExecContext(ctx,
        "UPDATE orders SET user_email = $1 WHERE user_id = $2",
        event.NewEmail, event.UserID)
    return err
}

This creates eventual consistency. For a brief period, Service A has the new email while Service B has the old email. But the system converges to consistency without blocking operations or requiring distributed transactions.

Strangler Fig Migration Strategy

Migrating from shared database to database-per-service can’t happen overnight. The strangler fig pattern provides a gradual migration path that doesn’t require stopping the world.

Start by identifying service boundaries within your shared schema:

-- Original shared schema
CREATE TABLE users (id, email, name, created_at);
CREATE TABLE orders (id, user_id, amount, status, created_at);
CREATE TABLE products (id, name, price, inventory_count);
CREATE TABLE order_items (order_id, product_id, quantity, price);

Extract one bounded context at a time. Begin with the service that has the clearest ownership and fewest dependencies:

Phase 1: Extract Product Service

1. Create separate product database
2. Migrate product data
3. Update product service to write to both databases
4. Update consuming services to read from product service API
5. Remove product tables from shared database

# Product service migration config
apiVersion: v1
kind: ConfigMap
metadata:
  name: product-service-db
data:
  DATABASE_URL: "postgres://product-service-db:5432/products"
  SHARED_DB_URL: "postgres://shared-db:5432/legacy"  # Temporary during migration

Phase 2: Extract User Service

1. Create user service database
2. Implement dual-write pattern
3. Backfill user data to new database
4. Switch reads to user service API
5. Remove users table from shared database

The strangler fig approach minimizes risk by allowing gradual validation at each step. If issues emerge, you can pause the migration without affecting unextracted services.

The Full Architecture

The final architecture eliminates database-level coupling through several layers:

Service Layer: Each microservice owns its database instance and storage technology. The user service uses PostgreSQL for transactional data. The analytics service uses ClickHouse for time-series queries. The search service uses Elasticsearch for full-text indexing.

Data Layer: Services duplicate necessary data locally and maintain consistency through events. The order service stores user email and name for query performance. The recommendation service caches product data for low-latency suggestions.

Communication Layer: Services communicate through well-defined APIs and asynchronous events. The event bus (Kafka, RabbitMQ, or AWS EventBridge) handles cross-service data synchronization. API gateways provide load balancing, authentication, and request routing.

Migration Layer: During transitions, services run dual-write patterns to maintain consistency across old and new schemas. Feature flags control read/write routing to enable gradual cutover.

Component Deep Dives

Event Bus Configuration

The event bus handles cross-service data synchronization without introducing tight coupling. Services publish events when their data changes and subscribe to events they need for local denormalization.

# Kafka topic configuration for user events
apiVersion: kafka.strimzi.io/v1beta2
kind: KafkaTopic
metadata:
  name: user-events
  labels:
    strimzi.io/cluster: event-cluster
spec:
  partitions: 12
  replicas: 3
  config:
    retention.ms: 604800000  # 7 days
    segment.ms: 86400000     # 1 day
    compression.type: lz4

// Event publishing with retry and dead letter queue
type EventPublisher struct {
    producer *kafka.Producer
    dlq      *kafka.Producer
}

func (p *EventPublisher) PublishUserEvent(ctx context.Context, event UserEvent) error {
    message := &kafka.Message{
        Topic:     "user-events",
        Key:       []byte(event.UserID),
        Value:     event.Serialize(),
        Timestamp: time.Now(),
    }
    
    err := p.producer.WriteMessages(ctx, message)
    if err != nil {
        // Send to dead letter queue for manual processing
        dlqMessage := &kafka.Message{
            Topic: "user-events-dlq", 
            Value: message.Value,
            Headers: []kafka.Header{
                {Key: "original-error", Value: []byte(err.Error())},
                {Key: "retry-count", Value: []byte("3")},
            },
        }
        return p.dlq.WriteMessages(ctx, dlqMessage)
    }
    return nil
}

Database Ownership Enforcement

Infrastructure-as-code enforces schema ownership by provisioning separate database instances with isolated network access.

# User service database
resource "aws_db_instance" "user_service" {
  identifier = "user-service-db"
  engine     = "postgres"
  engine_version = "14.9"
  instance_class = "db.t3.micro"
  allocated_storage = 20
  
  db_name  = "users"
  username = "user_service"
  password = var.user_service_db_password
  
  vpc_security_group_ids = [aws_security_group.user_service_db.id]
  db_subnet_group_name   = aws_db_subnet_group.private.name
  
  backup_retention_period = 7
  backup_window          = "03:00-04:00"
  maintenance_window     = "sun:04:00-sun:05:00"
  
  tags = {
    Service = "user-service"
    Owner   = "user-team"
  }
}

# Security group allows only user service access
resource "aws_security_group" "user_service_db" {
  name_prefix = "user-service-db"
  vpc_id      = var.vpc_id

  ingress {
    from_port       = 5432
    to_port         = 5432
    protocol        = "tcp"
    security_groups = [aws_security_group.user_service.id]
  }
  
  # No other services can access this database
  egress {
    from_port   = 0
    to_port     = 0
    protocol    = "-1"
    cidr_blocks = ["0.0.0.0/0"]
  }
}

Bounded Context Identification

Domain-driven design principles help identify natural service boundaries within shared schemas. Look for aggregates that change together and have clear ownership.

// User aggregate - clear ownership boundary
type User struct {
    ID          string
    Email       string
    Profile     UserProfile
    Preferences UserPreferences
    CreatedAt   time.Time
}

// Order aggregate - separate business context
type Order struct {
    ID          string
    UserID      string  // Reference, not embedded
    Items       []OrderItem
    Status      OrderStatus
    CreatedAt   time.Time
}

// Product aggregate - independent lifecycle
type Product struct {
    ID          string
    Name        string
    Price       Money
    Inventory   InventoryLevel
    Category    ProductCategory
}

Aggregates that frequently change together belong in the same service. Aggregates with independent lifecycles and different teams should be separate services with separate databases.

Migration Dual-Write Pattern

During schema extraction, services temporarily write to both old and new databases to ensure data consistency throughout the migration.

type UserService struct {
    newDB    *sql.DB  // New dedicated database
    legacyDB *sql.DB  // Shared legacy database
    migrationMode bool
}

func (s *UserService) CreateUser(ctx context.Context, user User) error {
    // Always write to new database
    err := s.writeToNewDB(ctx, user)
    if err != nil {
        return err
    }
    
    // During migration, also write to legacy database
    if s.migrationMode {
        legacyErr := s.writeToLegacyDB(ctx, user)
        if legacyErr != nil {
            // Log error but don't fail - new DB is source of truth
            log.Error("Failed to write to legacy database", 
                     "user_id", user.ID, "error", legacyErr)
        }
    }
    
    return nil
}

Comparison Table

Approach	Write Complexity	Read Complexity	Latency	Storage Cost	Failure Modes	Best Use Case
Shared Database	Low	Low	Low	Low	Schema coupling, coordination overhead	Small teams, early development
Database-Per-Service	Medium	Medium	Medium	High	Event lag, data inconsistency	Independent teams, scale requirements
Event Sourcing	High	High	Medium	High	Event replay complexity	Audit requirements, complex business logic
CQRS with Projections	High	Low	Low	Very High	Projection sync failures	Read-heavy workloads, reporting
Federated Schema	Medium	High	High	Medium	Gateway bottlenecks	Legacy integration, gradual migration

For most teams scaling beyond 3-4 microservices, database-per-service provides the best balance of independence and complexity. The operational benefits of autonomous deployments outweigh the costs of data duplication and eventual consistency.

Key Takeaways

• Database-per-service eliminates schema coupling by giving each service complete ownership of its storage layer and migration lifecycle.

• Data duplication becomes acceptable and beneficial when it enables service independence and eliminates network dependencies for common queries.

• Eventual consistency through events provides a middle ground between strong consistency and complete data divergence in distributed systems.

• Bounded contexts from domain-driven design provide natural guidelines for identifying which data belongs in which service database.

• Strangler fig migration allows gradual extraction from shared databases without requiring big-bang rewrites or system downtime.

• Infrastructure enforcement through separate database instances, security groups, and access controls makes schema ownership impossible to accidentally violate.

• Dual-write patterns enable safe migrations by temporarily maintaining consistency across old and new storage systems during transitions.

• Service autonomy requires accepting tradeoffs in data consistency, storage costs, and query complexity in exchange for deployment independence.

The most successful microservice architectures are those that embrace the tradeoffs rather than fighting them. Design for the independence you need, not the theoretical purity you want.

Frequently Asked Questions

Q: How do you handle transactions that span multiple services? A: Use the saga pattern for distributed transactions. Coordinate multi-service workflows through choreographed events or orchestrated commands with compensation logic for failures. Avoid distributed transactions across service boundaries.

Q: What about referential integrity without foreign keys across services? A: Implement eventual consistency through event handlers. When a user is deleted, publish a UserDeleted event. Consuming services handle orphaned references by either soft-deleting related records or maintaining tombstone records for audit trails.

Q: How do you prevent data divergence when event processing fails? A: Implement idempotent event handlers and dead letter queues. Store event processing state to enable retries. Use event sourcing for critical workflows where complete consistency is required.

Q: Should read-only services also have their own databases? A: Yes, if they have different performance requirements. Analytics services benefit from columnar databases like ClickHouse. Search services need Elasticsearch. Reporting services can use read replicas or materialized views optimized for their query patterns.

Q: How do you handle schema evolution when multiple services duplicate the same entity? A: Use versioned events and backward-compatible serialization. Services can lag behind schema changes. Implement field mappers to handle differences between canonical and denormalized representations.

Q: What’s the performance impact of eliminating cross-service joins? A: Initially higher due to denormalization overhead and API calls. Long-term performance improves due to service-specific optimization, better caching strategies, and elimination of cross-database query coordination.

Interview Questions

Q: Walk me through migrating from a shared database to database-per-service. What are the key steps and risks? Expected depth: Strangler fig pattern, dual-write implementation, event-driven consistency, rollback strategies, team coordination, data validation approaches.

Q: How would you design data synchronization between services that need eventual consistency? Expected depth: Event bus architecture, idempotent consumers, dead letter queues, event ordering, conflict resolution, schema evolution strategies.

Q: A service needs to display data from 5 different microservices. How do you optimize this without violating service boundaries? Expected depth: Backend for frontend pattern, service composition, caching strategies, denormalization tradeoffs, API gateway aggregation, materialized views.

Q: Two services have different performance requirements for the same logical entity. How do you handle this? Expected depth: Polyglot persistence, read replicas, CQRS patterns, data pipeline architecture, storage technology selection, consistency level tradeoffs.

Q: How do you maintain data quality when the same entity is duplicated across multiple service databases?
Expected depth: Event-driven validation, schema registries, data quality monitoring, reconciliation jobs, canonical data sources, data lineage tracking.