Bit.ly — URL Shortener

Requirements

Functional

• Create a short URL from a long URL.
• Optionally support a custom alias (user-chosen short code).
• Optionally support an expiration time for the short URL.
• Redirect the user to the original URL when they visit the short link.

Non-functional

• Low redirect latency — target ~200 ms (the hot path).
• Scale — 100M DAU, 1B URLs.
• Uniqueness — every short code is globally unique.
• High availability; eventual consistency is acceptable on the write path.
• Heavily read-skewed (~100:1) — redirects dominate.

Scale & back-of-the-envelope

Short-code length (base62 = 0-9 A-Z a-z)

• 62^5 ≈ 916M — too few for 1B URLs.
• 62^6 ≈ 56.8B — comfortably sufficient → target 6–7 chars.
• 62^7 ≈ 3.5T — ample headroom.

Random generation is subject to the birthday paradox (~expected collisions grow with n²/2M), which is why a uniqueness strategy matters (see deep dive).

Core entities

• Original URL — the long destination.
• Short URL — the generated code or custom alias that maps to the original.
• User — the creator/owner of a short URL.

API design

Shorten

POST /urls
{
  "originalUrl":    "https://example.com/very/long/path?x=1",
  "alias":          "my-link",                 // optional custom alias
  "expirationTime": "2026-12-31T00:00:00Z"     // optional
}

201 Created  → { "shortUrl": "https://bit.ly/abc123" }
409 Conflict → requested custom alias already taken

Redirect

GET /{shortCode}
302 Found
Location: <originalUrl>
404 / 410 → unknown or expired code

Write endpoints sit behind auth + rate limiting to prevent abuse and keyspace scraping.

High-level design

A client hits an API Gateway (TLS, auth, rate limiting, routing) that splits traffic into a Read Service (redirects — the hot path) and a Write Service (creating short URLs). The Read Service is fronted by a Redis read-through LRU cache keyed shortCode → longUrl; on a miss it falls back to the database. The Write Service generates a unique code via an atomic global counter, persists the mapping, and returns the short URL. Redirects are returned as 302.

flowchart LR
    Client["Client"]
    GW["API Gateway"]
    Read["Read Service"]
    Write["Write Service"]
    Cache["Redis Cache (LRU)"]
    Counter["Global Counter (INCR)"]
    DB[("Database")]

    Client -->|"shorten / visit"| GW
    GW --> Read
    GW --> Write
    Read -->|"read-through"| Cache
    Read --> DB
    Write --> DB
    Write -->|"INCR"| Counter
    GW -->|"302 redirect"| Client
      

Read/write split behind a gateway; Redis fronts the redirect hot path.

Deep dives

1 · Short-code generation

Codes must be fast, unique, and short. Four candidates:

Chosen: counter + base62, mitigating predictability with a bijective scramble (e.g. sqids/Hashids) so codes are non-sequential yet still unique & reversible, plus rate limiting and a "don't shorten private URLs" caveat. Scale the counter with Redis INCR and range/block leasing (each writer leases 1,000 IDs at a time) to remove per-request contention.

flowchart TD
    Start["Need short code"]
    Q1{"Custom alias?"}
    UseAlias["Use alias, verify unique"]
    Gen["Counter + base62"]
    Scramble["Bijective scramble (sqids)"]
    Done["Store mapping"]

    Start --> Q1
    Q1 -->|yes| UseAlias
    Q1 -->|no| Gen
    Gen --> Scramble
    Scramble --> Done
    UseAlias --> Done
      

2 · Read path & caching (scaling the hot path)

Redirects carry ~100× the traffic of writes, so the path is cache-first. URL popularity follows a power law, so a modest Redis cache yields a high hit ratio and keeps most redirects in single-digit ms. On a miss, read the DB and populate the cache with a TTL aligned to the URL's expiration; add read replicas and optionally a CDN/edge layer.

sequenceDiagram
    participant C as Client
    participant R as Read Service
    participant Cache as Redis
    participant DB as Database
    C->>R: GET /shortCode
    R->>Cache: GET shortCode
    alt cache hit
        Cache-->>R: longUrl
    else miss
        Cache-->>R: nil
        R->>DB: SELECT longUrl WHERE shortCode
        DB-->>R: longUrl
        R->>Cache: SET shortCode longUrl
    end
    R-->>C: 302 redirect to longUrl
      

3 · 301 vs 302 redirect

Use 302 because expiration and analytics require every click to reach the service. Choose 301 only when raw redirect performance outweighs tracking.

4 · Expiration & analytics

Lazy deletion on read (now > expirationTime → 410 + evict from cache) plus a background purge job. Analytics are emitted asynchronously (fire-and-forget to Kafka → stream processing) so they never block the redirect.

Data model

Indexes: clustered PK on short_url (the O(1) redirect lookup), secondary on user_id ("my links"), optional on expiration_time for cleanup.

DB choice: the access pattern is a key-value lookup by short code, so either relational (Postgres/MySQL with replicas, shard by hash(short_code)) or a NoSQL KV store (DynamoDB/Cassandra with native TTL) works. Shard by short_code for even load.

Bottlenecks & scaling