Understanding Protocol Buffers (ProtoBuf): Principles, Practices, and Performance

ProtoBuf (Protocol Buffers) is a cross‑platform, language‑neutral, extensible method for serializing structured data, widely used for network data exchange and storage. With the rise of heterogeneous systems and the growing popularity of gRPC, understanding ProtoBuf’s principles is essential for modern backend development.

Why ProtoBuf? It was originally created by Google to solve version‑compatibility problems in request/response protocols. ProtoBuf enables easy addition of new fields without breaking older servers, provides a compact binary format, and supports automatic code generation for many languages (C++, Java, Python, Go, Ruby, Objective‑C, C#).

Key characteristics include:

Ease of introducing new fields while maintaining backward compatibility.

Self‑describing data format that can be processed by various languages.

Automatic generation of serialization/deserialization code, reducing manual parsing effort.

Use as a persistent storage format beyond RPC.

Message definition example (customer.proto)

syntax = "proto3";

package domain;

option java_package = "com.protobuf.generated.domain";
option java_outer_classname = "CustomerProtos";

message Customers {
    repeated Customer customer = 1;
}

message Customer {
    int32 id = 1;
    string firstName = 2;
    string lastName = 3;

    enum EmailType {
        PRIVATE = 0;
        PROFESSIONAL = 1;
    }

    message EmailAddress {
        string email = 1;
        EmailType type = 2;
    }

    repeated EmailAddress email = 5;
}

To generate Java source files, install the protoc compiler and run:

protoc --java_out=./src/main/java ./src/main/idl/customer.proto

Sample Java usage:

CustomerProtos.Customer.EmailAddress email = CustomerProtos.Customer.EmailAddress.newBuilder()
        .setType(CustomerProtos.Customer.EmailType.PROFESSIONAL)
        .setEmail("[email protected]")
        .build();

CustomerProtos.Customer customer = CustomerProtos.Customer.newBuilder()
        .setId(1)
        .setFirstName("Lee")
        .setLastName("Richardson")
        .addEmail(email)
        .build();

// Serialization
byte[] binaryInfo = customer.toByteArray();
System.out.println(bytes_String16(binaryInfo));
System.out.println(customer.toByteArray().length);

// Deserialization
CustomerProtos.Customer anotherCustomer = CustomerProtos.Customer.parseFrom(binaryInfo);
System.out.println(anotherCustomer.toString());

Performance comparison

Serializing a simple JSON object yields 118 bytes, while the same data serialized with ProtoBuf occupies only 48 bytes:

// JSON size: 118 byte
// ProtoBuf size: 48 byte

ProtoBuf also offers 20–100× faster serialization/deserialization compared with XML/JSON.

Advantages

High efficiency – smaller payloads and faster processing.

Cross‑platform, multi‑language support (C++, Java, Python, Go, etc.).

Excellent extensibility and backward compatibility.

Simple usage thanks to automatic code generation.

Disadvantages

Binary format is not human‑readable; without the .proto definition, debugging is difficult.

Lacks self‑describing metadata compared with XML/JSON.

Tools such as Charles Proxy can decode ProtoBuf traffic when provided with the .proto file, and the binary nature can add a layer of protection against casual data inspection.