An Overview of Tencent Cloud Message Queue (CMQ): Architecture, Features, and Practical Use Cases

Background – Message queues are widely used for decoupling services, peak‑shaving, traffic control, and broadcasting. Popular open‑source solutions include RabbitMQ, Kafka, and RocketMQ, each with distinct trade‑offs. Tencent Cloud Message Queue (CMQ) offers high reliability, strong consistency, and rich features such as push/pull models, message replay, delayed delivery, publish/subscribe, routing, and encryption.

Underlying Architecture – CMQ’s architecture consists of sets of three broker replicas that ensure reliable storage and high availability, using the Raft algorithm for data consistency. Each set follows the CP side of the CAP theorem; only when a majority of nodes are healthy can messages be produced and consumed.

Practical Cases

1. Broadcast Pull Model – CMQ supports both queue (pull) and topic (push) models. The queue model allows clients to pull messages at their own pace, while the topic model pushes messages to multiple subscribers. For environments where push is infeasible (e.g., internal VPCs), CMQ can combine queue and topic to achieve a broadcast‑pull pattern where a topic’s subscriber is a queue instance.

# python sdk demo code: create subscription of queue protocol
my_sub = my_account.get_subscription(topic_name, subscription_name)
subscription_meta = SubscriptionMeta()
subscription_meta.Endpoint = "queue1"
subscription_meta.Protocal = "queue"
my_sub.create(subscription_meta)

2. Long‑Polling Pull – To avoid inefficient frequent polling, CMQ provides a long‑polling feature. When a consumer issues a pull request, the server holds the request for up to the configured wait time (e.g., 10 seconds). If a new message arrives, it is returned immediately; otherwise the request times out.

# python sdk demo code: receive message through long polling
pollingWaitSeconds = 3
recv_msg = my_queue.receive_message(pollingWaitSeconds)

3. Delayed Messages – CMQ allows messages to become visible only after a specified delay, enabling simple implementation of scheduled tasks.

# python sdk demo code: send delayed message
msg_body = "I am delay message"
msg = Message(msg_body)
delaySeconds = 3
my_queue.send_message(msg, delaySeconds)

4. Message Rewind – Similar to Kafka, CMQ can rewind a queue to a previous timestamp, allowing re‑consumption of messages that were already processed, useful for financial reconciliation or retry scenarios.

# python sdk demo code: rewind the queue
# backtrack one hour
backTrackingTime = int(time.time()) - 3600
my_queue.rewindQueue(backTrackingTime)

5. Topic Routing Matching – CMQ’s topic model supports routing keys with wildcard patterns (* matches one word, # matches one or many). Subscribers can define binding keys to receive only messages that match specific patterns.

# python sdk demo code: set topic‑subscription route‑rule
my_sub = my_account.get_subscription(topic_name, subscription_name)
subscription_meta = SubscriptionMeta()
subscription_meta.Endpoint = "http://test.com"
subscription_meta.Protocal = "http"
subscription_meta.bindingKey = ['*.*.rabbit','lazy.#']
my_sub.create(subscription_meta)

message = Message()
message.msgBody = "route msg test"
my_topic.publish_message(message, 'quick.orange.rabbit')

6. Large Message Transmission – CMQ limits a single message to 1 MB. For messages larger than 64 KB, QPS is not guaranteed. Two solutions are provided: (a) message fragmentation with custom headers for reassembly, and (b) storing the payload in COS (Tencent Object Storage) and sending the COS URL via CMQ.

7. Message Encryption – CMQ integrates with Tencent KMS. Two encryption approaches exist: (a) client‑side encryption where the SDK encrypts data using a data key obtained from KMS, and (b) server‑side encryption where CMQ automatically encrypts/decrypts messages via HTTPS (still under development).

For further reading, the article includes links to additional resources on software architecture, the role of architects, and micro‑service design principles.