Design and Implementation of a High‑Concurrency Short URL Service
This article explains the benefits of short URLs, outlines their basic workflow, and details a backend design that uses an incremental issuer, base conversion, persistent storage, caching, batch ID allocation, and distributed issuers to achieve efficient, high‑concurrency short‑URL generation.
Short URLs are widely used in SMS and social platforms because they fit length limits, look cleaner, enable analytics, and hide parameters for security.
The article describes the basic process: a service maps a long URL to a short one, the short URL is embedded in messages, the user clicks it, the browser follows a 301/302 redirect to the original long URL, and the target content is displayed.
For the mapping, the simplest approach is an incremental issuer: each incoming long URL receives a sequential number (e.g., 0, 1, 2 …). This number is then converted to a higher base such as base‑32 or base‑64 to produce a compact short URL like www.x.cn/0 or www.x.cn/1 .
Storage of the long‑short relationship must be persistent; a relational database (MySQL) with an auto‑increment primary key works for low QPS, while a key‑value store (Redis) can be used for higher traffic. Achieving a strict one‑to‑one mapping would require additional space or caching mechanisms.
To handle high concurrency, the design recommends caching popular mappings in Redis, counting accesses to long URLs, and storing recent entries in an in‑memory cache. Batch issuing of IDs reduces database load by fetching a block of identifiers (e.g., 10,000) at once and serving them from memory, with asynchronous write‑back.
A distributed approach avoids a single point of failure by partitioning the ID space across multiple issuer services (e.g., each service handles numbers ending with a specific suffix or increments by a step size), allowing the system to scale without inter‑service synchronization.
The article provides a Java implementation that uses Redis for both storage and caching. It defines constants, a getShortURL method that checks the cache, increments the issuer, stores the mapping, and returns the short URL, as well as utilities for converting numbers to arbitrary bases and an enum for supported bases (32 and 64).
package util;
import redis.clients.jedis.Jedis;
/**
* Created by pfliu on 2019/06/23.
*/
public class ShortURLUtil {
private static final String SHORT_URL_KEY = "SHORT_URL_KEY";
private static final String LOCALHOST = "http://localhost:4444/";
private static final String SHORT_LONG_PREFIX = "short_long_prefix_";
private static final String CACHE_KEY_PREFIX = "cache_key_prefix_";
private static final int CACHE_SECONDS = 1 * 60 * 60;
private final String redisConfig;
private final Jedis jedis;
public ShortURLUtil(String redisConfig) {
this.redisConfig = redisConfig;
this.jedis = new Jedis(this.redisConfig);
}
public String getShortURL(String longURL, Decimal decimal) {
// Query cache
String cache = jedis.get(CACHE_KEY_PREFIX + longURL);
if (cache != null) {
return LOCALHOST + toOtherBaseString(Long.valueOf(cache), decimal.x);
}
// Auto‑increment
long num = jedis.incr(SHORT_URL_KEY);
// Save mapping (could be MySQL)
jedis.set(SHORT_LONG_PREFIX + num, longURL);
// Write cache
jedis.setex(CACHE_KEY_PREFIX + longURL, CACHE_SECONDS, String.valueOf(num));
return LOCALHOST + toOtherBaseString(num, decimal.x);
}
/** Characters for base conversion */
final static char[] digits = {
'0','1','2','3','4','5','6','7','8','9',
'A','B','C','D','E','F','G','H','I','J','K','L',
'M','N','O','P','Q','R','S','T','U','V','W','X','Y','Z',
'a','b','c','d','e','f','g','h','i','j','k','l','m','n','o','p','q','r','s','t','u','v','w','x','y','z'};
/** Convert a decimal number to another base */
private String toOtherBaseString(long n, int base) {
long num = (n < 0) ? ((long)2 * 0x7fffffff) + n + 2 : n;
char[] buf = new char[32];
int charPos = 32;
while ((num / base) > 0) {
buf[--charPos] = digits[(int)(num % base)];
num /= base;
}
buf[--charPos] = digits[(int)(num % base)];
return new String(buf, charPos, (32 - charPos));
}
enum Decimal {
D32(32), D64(64);
int x;
Decimal(int x) { this.x = x; }
}
public static void main(String[] args) {
for (int i = 0; i < 100; i++) {
System.out.println(new ShortURLUtil("localhost").getShortURL("www.baidudu.com", Decimal.D32));
System.out.println(new ShortURLUtil("localhost").getShortURL("www.baidu.com", Decimal.D64));
}
}
}Top Architect
Top Architect focuses on sharing practical architecture knowledge, covering enterprise, system, website, large‑scale distributed, and high‑availability architectures, plus architecture adjustments using internet technologies. We welcome idea‑driven, sharing‑oriented architects to exchange and learn together.
How this landed with the community
Was this worth your time?
0 Comments
Thoughtful readers leave field notes, pushback, and hard-won operational detail here.