Designing Low‑Latency Applications: Simplicity, Data Transformation, and Architecture

In development and architecture design, we are often required to build low‑latency applications; fast response to various requests is crucial for optimizing user experience. Designing low‑latency applications involves many factors that distinguish them from ordinary applications, including generating less waste, transforming data fewer times, and ensuring required data is always readily available.

Simplicity is key

If we want something to run faster, the best way is to make it do less work. That means generating less waste, transforming data fewer times, and keeping required data always accessible.

If you want stable performance, simplicity is especially important. If you aim for 99% or 99.9% accuracy, a system error means most things go wrong. The more complex a system, the worse it becomes. If throughput matters, a slow output can badly affect results; you need to worry about 99.99% or higher accuracy because even a rare latency can affect many requests, and hundreds or thousands of latencies are not uncommon.

Don’t think about how easy it is to add, think about how easy it is to remove

You don’t need to focus on how conveniently you can add code, but rather on how easily you can discard unnecessary work. Many frameworks are designed to be easy to adopt and add features, so if you need the framework to do less, it can.

The best way to achieve this is to use very lightweight libraries and frameworks (in call stack and data transformation), and if not needed you can do even less.

You want a clear, simple solution, but also want options to replace or delete any functionality you don’t need.

Transform data fewer times

I am often amazed by the intricacy of code. It is repeatedly executed like kneading dough or data. The same data is transformed again and again, then transformed back into another form, finally placed into a custom data structure to build another structure. All these complex steps slow down the application and obscure what it actually does.

You need to design your own application so you can see what happens to data from one end to the other, allowing you to remove steps you don’t actually need (and that may be sources of errors).

I have seen an example with a date: converting a 64‑bit integer to a date object, then to an MM/DD/YYYY string, then to a yyyy/MM/DD string, then back to a date type, and finally back to a 64‑bit integer.

I have also seen another example: converting a 64‑bit double to a double object, then to a string and back to double, applying incorrect or complex rounding, and finally ending up with the original 64‑bit double.

It sounds silly, but some frameworks do this. Each layer in each library is written by different people, and data is transformed between the models used by each library.

All problems in computer science can be solved from an abstract level.

Code readability

Much code is written in a natural‑language style that describes the application at a high level. Even if I don’t know programming, I can understand what the code is supposed to do. It feels like magic. But the problem is that while the code is written at a high level, its execution is not necessarily as imagined. Functional issues can be caught by good tests. When your design hides the low‑level details of what the computer actually executes, the code may run correctly but be hard to judge its performance. Some code works well most of the time and performs poorly rarely, making detection difficult. Even performance analysis tools can’t help much.

The advantage of low‑latency code is that it makes it easy to read what the computer needs to implement. Is the operation O(1), O(log N) or O(N)? Is it thread‑safe? If I don’t need thread safety, is it easier?

This low‑level code is useful, but you still need to understand at a high level what the code does. There is always a part of the code that tells you what a component is for. High‑level component calls go down so you can see how it works, and then you can see the low‑level details.

Separation of business and infrastructure code

If the application does not do what it should, you should change your business logic to fix it. If the application does not behave that way, you should change the infrastructure so the application still does what it needs to.

Some good examples

Aeron ( https://github.com/real-logic/Aeron ) Chronicle ( http://chronicle.software/hft-products/ )

How many differences are there?

Using low‑latency techniques can significantly improve typical performance and increase throughput. Under consistent conditions, latency can be reduced by 99%.

Is the design already as fast as possible?

For the projects we are developing, this is not the most critical benefit. It is not just about productivity. The main advantage is developing the simplest solution to address performance needs, making it faster with less effort, and only executing it when it is proven necessary.

Simplicity also improves maintainability

Our thumb rule is, a successful system’s maintenance cost is three times its construction cost . Simplicity and transparency help minimize long‑term cost because they enable you to run an application with lower risk and shorter time.

Since simplicity is so important, why doesn’t everyone do it?

Building a large, complex system is easy; you just keep adding features. Keeping a system as simple as possible is very difficult and requires experience. To avoid adding unnecessary things during development, you often need years of experience building similar solutions.