Skip to content

Commit 15f5a69

Browse files
Add files via upload
1 parent 1d35b90 commit 15f5a69

File tree

5 files changed

+148
-0
lines changed

5 files changed

+148
-0
lines changed
Binary file not shown.
Binary file not shown.
Binary file not shown.
Lines changed: 148 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,148 @@
1+
**Setup**
2+
3+
Run [landoop/fast-data-dev](https://github.com/Landoop/fast-data-dev) docker image. For example:
4+
5+
```
6+
docker run --rm -it -p 2181:2181 -p 3030:3030 -p 8081:8081 -p 8082:8082 -p 8083:8083 -p 9092:9092 -p 9581:9581 -p 9582:9582 -p 9583:9583 -p 9584:9584 -e ADV_HOST=127.0.0.1 landoop/fast-data-dev:latest
7+
```
8+
_Note: Follow the instructions on fast-data-dev README to customise the container._
9+
10+
Enter the container bash:
11+
12+
```
13+
docker run --rm -it --net=host landoop/fast-data-dev bash
14+
```
15+
16+
Kafka utilities are now available:
17+
18+
# Topics
19+
20+
Creating a New Topic
21+
```
22+
kafka-topics --create --zookeeper localhost:2181 --replication-factor 1 --partitions 3 --topic my-topic
23+
```
24+
Verify the topic
25+
```
26+
kafka-topics --list --zookeeper localhost:2181
27+
```
28+
Adding Partitions
29+
```
30+
kafka-topics --zookeeper localhost:2181 --alter --topic my-topic --partitions 16
31+
```
32+
Deleting a Topic
33+
```
34+
kafka-topics --zookeeper localhost:2181 --delete --topic my-topic
35+
```
36+
Listing All Topics in a Cluster
37+
```
38+
kafka-topics --zookeeper localhost:2181 --list
39+
```
40+
Describing Topic Details
41+
```
42+
kafka-topics --zookeeper localhost:2181/kafka-cluster --describe
43+
```
44+
Show Under-replicated Partitions for topics
45+
```
46+
kafka-topics --zookeeper localhost:2181/kafka-cluster --describe --under-replicated-partitions
47+
```
48+
49+
# Producers
50+
Produce messages standard input
51+
```
52+
kafka-console-producer --broker-list localhost:9092 --topic my-topic
53+
```
54+
Produce messages file
55+
```
56+
kafka-console-producer --broker-list localhost:9092 --topic test < messages.txt
57+
```
58+
Produce Avro messages
59+
```
60+
kafka-avro-console-producer --broker-list localhost:9092 --topic my.Topic --property value.schema='{"type":"record","name":"myrecord","fields":[{"name":"f1","type":"string"}]}' --property schema.registry.url=http://localhost:8081
61+
```
62+
And enter a few values from the console:
63+
```
64+
{"f1": "value1"}
65+
```
66+
67+
# Consumers
68+
69+
## Consume messages
70+
71+
Start a consumer from the beginning of the log
72+
```
73+
kafka-console-consumer --bootstrap-server localhost:9092 --topic my-topic --from-beginning
74+
```
75+
Consume 1 message
76+
```
77+
kafka-console-consumer --bootstrap-server localhost:9092 --topic my-topic --max-messages 1
78+
```
79+
80+
Consume 1 message from `__consumer_offsets`
81+
```
82+
kafka-console-consumer --bootstrap-server localhost:9092 --topic __consumer_offsets --formatter 'kafka.coordinator.GroupMetadataManager$OffsetsMessageFormatter' --max-messages 1
83+
```
84+
85+
Consume, specify consumer group:
86+
```
87+
kafka-console-consumer --topic my-topic --new-consumer --bootstrap-server localhost:9092 --consumer-property group.id=my-group
88+
```
89+
90+
## Consume Avro messages
91+
```
92+
kafka-avro-console-consumer --topic position-reports --new-consumer --bootstrap-server localhost:9092 --from-beginning --property schema.registry.url=localhost:8081 --max-messages 10
93+
```
94+
95+
```
96+
kafka-avro-console-consumer --topic position-reports --new-consumer --bootstrap-server localhost:9092 --from-beginning --property schema.registry.url=localhost:8081
97+
```
98+
99+
## Consumers admin operations
100+
101+
List Groups
102+
```
103+
kafka-consumer-groups --new-consumer --list --bootstrap-server localhost:9092
104+
```
105+
Describe Groups
106+
```
107+
kafka-consumer-groups --bootstrap-server localhost:9092 --describe --group testgroup
108+
```
109+
110+
# Config
111+
Set the retention for the topic
112+
```
113+
kafka-configs --zookeeper localhost:2181 --alter --entity-type topics --entity-name my-topic --add-config retention.ms=3600000
114+
```
115+
Show all configuration overrides for a topic
116+
```
117+
kafka-configs --zookeeper localhost:2181 --describe --entity-type topics --entity-name my-topic
118+
```
119+
Delete a configuration override for retention.ms for a topic
120+
```
121+
kafka-configs --zookeeper localhost:2181 --alter --entity-type topics --entity-name my-topic --delete-config retention.ms
122+
```
123+
124+
# Performance
125+
126+
Producer
127+
```
128+
kafka-producer-perf-test --topic position-reports --throughput 10000 --record-size 300 --num-records 20000 --producer-props bootstrap.servers="localhost:9092"
129+
```
130+
131+
# ACLs
132+
```
133+
kafka-acls --authorizer-properties zookeeper.connect=localhost:2181 --add --allow-principal User:Bob --consumer --topic topicA --group groupA
134+
```
135+
136+
```
137+
kafka-acls --authorizer-properties zookeeper.connect=localhost:2181 --add --allow-principal User:Bob --producer --topic topicA
138+
```
139+
List the ACLs
140+
```
141+
kafka-acls --authorizer-properties zookeeper.connect=localhost:2181 --list --topic topicA
142+
```
143+
144+
# Zookeeper
145+
Enter zookeepr shell:
146+
```
147+
zookeeper-shell localhost:2182 ls /
148+
```
Binary file not shown.

0 commit comments

Comments
 (0)