Kafka startup error & amp; Problem solving
When I went to work early in the morning, I received a notice from my o & M colleagues that a physical machine was down, causing the virtual machine to hang up. The Kafka server had to be restarted
1. Start
Start zookeeper
bin/zkServer.sh start conf/zoo.cfg &
Start Kafka
bin/kafka-server-start.sh config/server.properties &
2. Test
question 1
After Kafka is started, it is found that warn is always printed, as shown in the figure above. In the process, PS – ef| grep Kafka cannot be seen. Obviously Kafka failed to start
Resetting first dirty offset of __consumer_offsets
From the repeated error message, we can know that this is a problem that the cleaning thread has been encountering. The fastest way is to empty Kafka’s data directory. Or regardless of this warn, when a large amount of data flows in and generates segments that can be cleaned up, there will be no more this warn. reference resources https://blog.csdn.net/define_us/article/details/80537186。
question 2
After Kafka is started normally, test whether it can be used to walk a wave
Create topic
bin/kafka-topics.sh --create --zookeeper localhost:2181 --replication-factor 1 --partitions 1 --topic test
Producer production message
[ apps@erp -computation-4 kafka_2.11-1.1.0]$ bin/kafka-console-producer.sh --broker-list 10.17.156.8:9092 --topic test
my name is xiaoqiang
Consumer News
[ apps@erp -computation-4 kafka_2.11-1.1.0]$ bin/kafka-console-consumer.sh --bootstrap-server 10.17.156.8:9092 --topic test --from-beginning
my name is xiaoqiang
At this point, Kafka is started and used normally. CTRL + C, close the xshell window and focus on the code. When testing the application, it is found that a pile of error reports all point to Kafka. Fight the Kafka server again and find that the Kafka process is no longer present
Kafka starts normally
The application starts normally
ctrl+c
Application connection Kafka error
The Kafka process was killed
Finally, the problem is found: when exiting Kafka, do not use Ctrl + C, but use the exit function to exit