Getting started with Kafka — ITVERSITY Lab

Login to ITVERSITY

Click on the ‘Quick Links’

Click on ‘Grafana’ under Quick links. It may ask for username/password . Give the username/password from ITVERSITY

You can see Kafka brokers on the rightside (see above)

Now log into a terminal in ITVERSITY

Create a “Topic” in Kafka

Before creating a topic, a Kafka cluster must be up and running. The name of the script to create topic is “kafka-topics.sh”

Create topic(s) in Kafka

Kafka topic creation

kafka-topics.sh — create \
> — zookeeper m01.itversity.com:2181,m02.itversity.com:2181,w01.itversity.com:2181 \
> — replication-factor 1 \
> — partitions 1 \
> — topic Lipsa_Kafka_Topic1

List all topics in Kafka

kafka-topics.sh — list \ — zookeeper m01.itversity.com:2181,m02.itversity.com:2181,w01.itversity.com:2181

List one topic in Kafka

kafka-topics.sh — list \ — zookeeper m01.itversity.com:2181,m02.itversity.com:2181,w01.itversity.com:2181 \ — topic

List one topic in Kafka

Command line producer for producing data

Command to produce messages, start typing after running this kakfa-console-producer command. The messages will be stored in topic Lipsa_Kafka_Topic1 on the host where brokers are running

kafka-console-producer.sh \
> — broker-list w01.itversity.com:9092,w02.itversity.com:9092,w03.itversity.com:9092 \
> — topic Lipsa_Kafka_Topic1

Command line consumer to consume

Open another shell and then run kafka-console-consumer command to see streaming messages

kafka-console-consumer.sh \
— bootstrap-server w01.itversity.com:9092,w02.itversity.com:9092,w03.itversity.com:9092 \
— topic Lipsa_Kafka_Topic1 \
— from-beginning