Whistle: Implementation Details of a Node.js Based Cross‑Platform HTTP Proxy

Whistle is a cross‑platform web debugging (HTTP) proxy implemented with Node.js. It provides real‑time packet capture for HTTP, HTTPS, HTTP/2, WebSocket, TCP and other common web requests, rule‑based request/response modification, and an extensible plugin system.

The article introduces the implementation principles of Whistle step by step, covering the following main topics:

What is an HTTP proxy

How to implement a simple HTTP proxy in Node.js

The complete Whistle HTTP proxy architecture

The concrete implementation of each of the five core modules

Rule management and plugin mechanisms

Simple HTTP proxy example (Node.js)

const http = require('http');
const { connect } = require('net');

// Utility to parse host:port
const getHostPort = (host, defaultPort) => {
  let port = defaultPort || 80;
  const index = host.indexOf(':');
  if (index !== -1) {
    port = host.substring(index + 1);
    host = host.substring(0, index);
  }
  return { host, port };
};

const getOptions = (req, defaultPort) => {
  // Assume host always exists; full implementation in Whistle
  const { host, port } = getHostPort(req.headers.host, defaultPort);
  return {
    hostname: host, // DNS resolves server IP and sets Host header
    port,
    path: req.url || '/',
    method: req.method,
    headers: req.headers,
    rejectUnauthorized: false // for HTTPS requests, ignored for HTTP
  };
};

const handleClose = (req, res) => {
  const destroy = err => {
    req.destroy();
    res && res.destroy();
  };
  res && res.on('error', destroy);
  req.on('error', destroy);
  req.once('close', destroy);
};

// Create HTTP server
const server = http.createServer();
// Handle normal HTTP requests
server.on('request', (req, res) => {
  const client = http.request(getOptions(req), svrRes => {
    res.writeHead(svrRes.statusCode, svrRes.headers);
    svrRes.pipe(res);
  });
  req.pipe(client);
  handleClose(res, client);
});

// Tunnel proxy for HTTPS, HTTP/2, WebSocket, TCP, etc.
server.on('connect', (req, socket) => {
  const client = connect(getHostPort(req.url), () => {
    socket.write('HTTP/1.1 200 Connection Established\r\n\r\n');
    socket.pipe(client).pipe(socket);
  });
  handleClose(socket, client);
});

server.listen(8080);

This code demonstrates that an HTTP proxy is essentially a regular HTTP server that listens for request and connect events. The request event handles ordinary HTTP traffic, while the connect event implements a tunnel proxy for HTTPS, HTTP/2, WebSocket and TCP connections.

The full Whistle architecture is divided into five modules:

Request entry module – supports direct HTTP/HTTPS requests, HTTP proxy, HTTPS proxy, and SOCKS5 proxy.

Tunnel proxy module – parses tunnel requests, performs MITM for HTTPS, and forwards data.

HTTP request processing module – matches global rules, executes plugins, records state and forwards the request.

Rule management module – handles global and private rules, supports dynamic, static, and remote rule loading.

Plugin management module – provides authentication, UI, request handling, data statistics, rule setting, custom certificates, and more.

Design principles of Whistle’s plugin system focus on completeness (all features are extensible), stability (each plugin runs in an isolated process), and usability (simple npm package + scaffolding). Plugins communicate with the core via HTTP, and the framework uses the pfork npm package to spawn plugin processes.

References and resources are provided, including the Whistle GitHub repository, official plugin repository, and detailed documentation links.