Kafka work Day 1
I will recommend the course on Udemy!
Download Kafaka ((2.11 is recommended)
and click http://www.gtlib.gatech.edu/pub/apache/kafka/2.0.0/kafka_2.11-2.0.0.tgz
Launch Zookeeper
# open a shell — zookeeper is at localhost:2181
bin/zookeeper-server-start.sh config/zookeeper.properties
Launch Kafka
# open another shell — kafka is at localhost:9092
bin/kafka-server-start.sh config/server.properties
Create input topic
bin/kafka-topics.sh — create — zookeeper localhost:2181 — replication-factor 1 — partitions 1 — topic streams-plaintext-input
create output topic
bin/kafka-topics.sh — create — zookeeper localhost:2181 — replication-factor 1 — partitions 1 — topic streams-wordcount-output
start a kafka producer
bin/kafka-console-producer.sh — broker-list localhost:9092 — topic streams-plaintext-input
input some data you want
kafka streams udemy
kafka data processing
kafka streams course
check the data is there
bin/kafka-console-consumer.sh — bootstraprver localhost:9092 — topic streams-plaintext-input — from-beginning
start a consumer on the output topic (it will show the result)
open another tab and input
bin/kafka-console-consumer.sh — bootstrap-server localhost:9092 \
— topic streams-wordcount-output \
— from-beginning \
— formatter kafka.tools.DefaultMessageFormatter \
— property print.key=true \
— property print.value=true \
— property key.deserializer=org.apache.kafka.common.serialization.StringDeserializer \
— property value.deserializer=org.apache.kafka.common.serialization.LongDeserializer
start the streams application
bin/kafka-run-class.sh org.apache.kafka.streams.examples.wordcount.WordCountDemo
check the topics now we have
The flow is the following:
