Design Instagram - System Design Tutorial for Beginners

Overview

In this tutorial, we will learn how to design a photo-sharing platform like Instagram. Instagram allows users to upload photos and videos, follow other users, and see a feed of posts from people they follow. Designing such a system requires a good understanding of scalability, storage, user interactions, and performance optimization.

Key Requirements

Functional Requirements

• Users can create accounts and log in.
• Users can upload photos and videos.
• Users can follow or unfollow other users.
• Users can see a feed of posts from the users they follow.
• Users can like and comment on posts.

Non-Functional Requirements

• High availability and reliability.
• Scalability to millions of users and billions of photos.
• Low latency feed generation.
• Efficient media storage and delivery.

Step 1: High-Level Architecture

At a high level, Instagram’s architecture can be broken down into multiple components:

• User Service
• Post Service
• Feed Service
• Follow Service
• Media Storage Service
• Authentication Service

Step 2: Database Schema

Users Table

User {
  user_id (PK)
  username
  email
  password_hash
  created_at
}

Posts Table

Post {
  post_id (PK)
  user_id (FK)
  image_url
  caption
  created_at
}

Followers Table

Follow {
  follower_id
  followee_id
  created_at
}

Question: Should we use a relational or NoSQL database?

Answer: It depends. For relational data like user info, relationships, and post metadata, a relational database (e.g., PostgreSQL or MySQL) is suitable. For scalable storage of media files, we can use a distributed object store like AWS S3. Feed generation may benefit from NoSQL databases due to high read/write throughput (e.g., Cassandra, DynamoDB).

Step 3: Image Upload and Storage

When a user uploads an image:

1. Client sends image to a backend API endpoint.
2. Server stores the image in a cloud storage service (e.g., Amazon S3).
3. The storage service returns a URL which is saved in the post metadata.

Example: Upload Image Workflow

Let’s say a user uploads a photo with a caption "Sunset at the beach". The flow would look like:

1. Frontend uploads the photo to a pre-signed URL from S3.
2. The image is saved as https://s3.amazonaws.com/instagram/photos/123.jpg.
3. The backend creates a record in the Post table with this image URL and caption.

Step 4: User Feed Generation

Feed generation is one of the most challenging parts. We have two strategies:

1. Pull Model

Every time a user opens the app, we dynamically fetch the latest posts from people they follow.

2. Push Model

Every time a user creates a post, we push it to the timelines of all their followers.

Question: Which model is better for Instagram?

Answer: Instagram uses a hybrid model. For normal users (who have few followers), push model works well. For celebrity users (millions of followers), pushing posts to every timeline is inefficient. In such cases, the pull model is used when a follower opens the app.

Step 5: Feed Service with Caching

To make the feed fast:

• We cache the timeline of active users using Redis.
• For inactive users, we compute the feed on the fly to save storage and cache space.

Step 6: Handling Likes and Comments

Likes and comments can be stored in separate tables:

Likes Table

Like {
  user_id
  post_id
  liked_at
}

Comments Table

Comment {
  comment_id
  post_id
  user_id
  text
  commented_at
}

Likes can be counted using counters stored in Redis for fast access and synced periodically to the database.

Step 7: Media Delivery using CDN

All media content is served using a CDN (Content Delivery Network). This reduces load on origin servers and ensures fast access globally.

Step 8: Scale with Microservices

Each component (user, post, feed, follow, media) can be independently deployed and scaled. Communication happens via REST or gRPC APIs.

Example: Scaling Feed Service

If the Feed Service becomes a bottleneck, we can replicate it across multiple regions and introduce a load balancer. We can also partition feed data by user_id to distribute the load across shards.

Step 9: Monitoring and Alerting

Use tools like Prometheus and Grafana to monitor:

• API latency
• Upload failures
• Cache hit/miss ratio

Set up alerts if any metrics cross defined thresholds.

Points to Remember

• Design for scalability from day one.
• Use CDNs and caching aggressively for media delivery and feeds.
• Choose storage systems according to access patterns (frequent read vs write-heavy).
• Separate read and write models where needed (CQRS).
• Use eventual consistency for non-critical operations like feed updates.

Question: How do we avoid single points of failure?

Answer: By replicating services, databases, and using distributed storage and load balancers. Also, circuit breakers and retry logic improve fault tolerance.