Build Military‑Grade Kafka Security in Four Easy Steps

As the core hub of distributed messaging systems, securing Kafka is essential for operations teams. This article guides you through four stages from basic to financial‑grade protection, building a robust message fortress.

Stage 1: Bare‑metal Mode – Basic Configuration (Use with Caution)

Reference: author's blog for installing Kafka. This is the original configuration where data is transmitted in plaintext, suitable only for isolated test environments; production must be upgraded.

Risk Warning: No encryption or authentication; attackers can easily eavesdrop or tamper with data.

Stage 2: SSL Armor – Transport Encryption

Creating a Digital Certificate Authority

<code># 1. Generate CA private key and self‑signed certificate
openssl req -new -x509 -nodes \
 -keyout ca.key -out ca.crt \
 -days 3650 -subj "/C=CN/ST=GuangDong/L=GuangZhou/CN=Kafka Root CA"

# 2. Import CA certificate into PKCS#12 truststore (shared by client and server)
keytool -keystore kafka.server.truststore.p12 -storetype PKCS12 \
 -alias CARoot -import -file ca.crt \
 -storepass truststore_password -noprompt

# Clean temporary files
rm -f ca.srl
</code>

Server Certificate

<code># 1. Generate server PKCS#12 keystore (private key + unsigned cert)
keytool -keystore kafka.server.keystore.p12 -storetype PKCS12 \
 -alias localhost -validity 3650 \
 -genkey -keyalg RSA -keysize 2048 \
 -storepass keystore_password \
 -dname "C=CN/ST=GuangDong/L=GuangZhou/CN=kafka-server"

# 2. Generate CSR
keytool -keystore kafka.server.keystore.p12 -storetype PKCS12 \
 -alias localhost -certreq -file server.csr \
 -storepass keystore_password

# 3. Sign server certificate with CA (add SAN)
openssl x509 -req -CA ca.crt -CAkey ca.key \
 -in server.csr -out server-signed.crt \
 -days 3650 -CAcreateserial \
 -extfile <(printf "subjectAltName=IP:172.139.20.17,IP:172.139.20.81,IP:172.139.20.177")

# 4. Import CA and signed certificate into server keystore
keytool -keystore kafka.server.keystore.p12 -storetype PKCS12 \
 -alias CARoot -import -file ca.crt \
 -storepass keystore_password -noprompt

keytool -keystore kafka.server.keystore.p12 -storetype PKCS12 \
 -alias localhost -import -file server-signed.crt \
 -storepass keystore_password -noprompt

# 5. Verify SAN extension
keytool -list -v -keystore kafka.server.keystore.p12 -alias localhost -storepass keystore_password | grep -A4 SubjectAlternativeName

# Clean temporary files
rm -f server.csr
</code>

Key Configuration

<code>listeners=PLAINTEXT://:9092,SSL://:9093
advertised.listeners=PLAINTEXT://172.139.20.17:9092,SSL://172.139.20.17:9093

# SSL settings
ssl.keystore.location=/app/kafka/pki/kafka.server.keystore.p12
ssl.keystore.password=keystore_password
ssl.keystore.type=PKCS12
ssl.truststore.location=/app/kafka/pki/kafka.server.truststore.p12
ssl.truststore.password=truststore_password
ssl.truststore.type=PKCS12
ssl.client.auth=none
</code>

Validate SSL

Client configuration (ssl-client.properties)

<code>security.protocol=SSL
ssl.truststore.location=/app/kafka/pki/kafka.server.truststore.p12
ssl.truststore.password=truststore_password
ssl.truststore.type=PKCS12
</code>

Consume messages (SSL)

<code>bin/kafka-console-consumer.sh --bootstrap-server 172.139.20.17:9093 --topic test --consumer.config ~/ssl-client.properties --from-beginning
</code>

Protection Effect: TLS encrypted transport defends against man‑in‑the‑middle attacks.

Stage 3: SCRAM Shield – Authentication

Configure SCRAM‑SHA‑512

<code>bin/kafka-configs.sh --bootstrap-server localhost:9092 --alter --add-config 'SCRAM-SHA-512=[password=admin-password]' --entity-type users --entity-name admin
</code>

JAAS Configuration

<code>KafkaServer {
  org.apache.kafka.common.security.scram.ScramLoginModule required
  username="admin"
  password="admin-secret";
};
</code>

Key Configuration

<code>listeners=PLAINTEXT://:9092,SSL://:9093,SASL_PLAINTEXT://:9094
advertised.listeners=PLAINTEXT://172.139.20.17:9092,SSL://172.139.20.17:9093,SASL_PLAINTEXT://172.139.20.17:9094

# SASL settings
sasl.enabled.mechanisms=SCRAM-SHA-512
</code>

Validate SASL_PLAINTEXT

Client configuration (sasl-plain-client.properties)

<code>security.protocol=SASL_PLAINTEXT
sasl.mechanism=SCRAM-SHA-512
sasl.jaas.config=org.apache.kafka.common.security.scram.ScramLoginModule required username="admin" password="admin-password";
</code>

Consume messages (SSL)

<code>bin/kafka-console-consumer.sh --bootstrap-server 172.139.20.17:9094 --topic test --consumer.config ~/sasl-plain-client.properties --from-beginning
</code>

Security Upgrade: Username/password authentication with dynamic salt encryption blocks illegal access.

Stage 4: Dual Sword – SSL + SASL

Key Configuration

<code>listeners=PLAINTEXT://:9092,SSL://:9093,SASL_PLAINTEXT://:9094,SASL_SSL://:9095
advertised.listeners=PLAINTEXT://172.139.20.17:9092,SSL://172.139.20.17:9093,SASL_PLAINTEXT://172.139.20.17:9094,SASL_SSL://172.139.20.17:9095
</code>

Validate SASL_SSL

Client configuration (sasl-ssl-client.properties)

<code>security.protocol=SASL_SSL
sasl.mechanism=SCRAM-SHA-512
sasl.jaas.config=org.apache.kafka.common.security.scram.ScramLoginModule required username="admin" password="admin-password";
ssl.truststore.location=/app/kafka/pki/kafka.server.truststore.p12
ssl.truststore.password=truststore_password
</code>

Consume messages (SASL_SSL)

<code>bin/kafka-console-consumer.sh --bootstrap-server 172.139.20.17:9095 --topic test --consumer.config ~/sasl-ssl-client.properties --from-beginning
</code>

Military‑grade Protection: Transport encryption + mutual authentication + dynamic credentials meet financial‑level security requirements.

Avoiding Pitfalls (Hard‑Earned Lessons)

Certificate trap: SAN must include all broker IPs.

Protocol isolation: keep management commands on PLAINTEXT channel.

Password management: use different passwords for keystore and truststore.

Version compatibility: JDK 11+ needs PKCS12 compatibility considerations.