Your First Topic
Learn how to create topics, produce messages, and consume them.
What is a Topic?
A topic is a named, ordered log of events. Topics are divided into partitions for parallel processing. Each message in a topic has an offset that uniquely identifies it within its partition. Unlike traditional message queues, messages in a topic are persisted and can be replayed by consumers.
Creating a Topic
Topics can be created via the REST API, gRPC, CLI, or the web console. When creating a topic, you specify the number of partitions and optional configuration like retention period and compression.
# Create a topic with 6 partitions and 7-day retention
streamctl topic create \
--name user-events \
--partitions 6 \
--retention 7d \
--compression lz4Producing Messages
Producers send messages to a specific topic. Messages can optionally include a key, which determines which partition receives the message. Messages with the same key are always routed to the same partition, preserving ordering for related events.
# Produce with a key (ensures ordering per user)
streamctl produce --topic user-events \
--key "user-123" \
--message '{"event": "page_view", "page": "/pricing"}'
# Produce without a key (round-robin across partitions)
streamctl produce --topic user-events \
--message '{"event": "heartbeat"}'Consuming Messages
Consumers read messages from topics. They can start from the beginning, the end, or a specific offset. Consumer groups allow multiple consumers to share the work of processing a topic, with each partition assigned to exactly one consumer in the group.
# Consume from the beginning
streamctl consume --topic user-events --from-beginning
# Consume as part of a consumer group
streamctl consume --topic user-events \
--group analytics-pipeline \
--auto-commit