Kafka startup error & amp; Problem solving

When I went to work early in the morning, I received a notice from my o & M colleagues that a physical machine was down, causing the virtual machine to hang up. The Kafka server had to be restarted

1. Start

Start zookeeper

bin/zkServer.sh start conf/zoo.cfg &

Start Kafka

bin/kafka-server-start.sh config/server.properties &

2. Test

question 1

After Kafka is started, it is found that warn is always printed, as shown in the figure above. In the process, PS – ef| grep Kafka cannot be seen. Obviously Kafka failed to start

Resetting first dirty offset of __consumer_offsets

From the repeated error message, we can know that this is a problem that the cleaning thread has been encountering. The fastest way is to empty Kafka’s data directory. Or regardless of this warn, when a large amount of data flows in and generates segments that can be cleaned up, there will be no more this warn. reference resources https://blog.csdn.net/define_us/article/details/80537186。

question 2

After Kafka is started normally, test whether it can be used to walk a wave

Create topic

bin/kafka-topics.sh --create --zookeeper localhost:2181 --replication-factor 1 --partitions 1 --topic test

Producer production message

[ apps@erp -computation-4 kafka_2.11-1.1.0]$ bin/kafka-console-producer.sh --broker-list 10.17.156.8:9092 --topic test

my name is xiaoqiang

Consumer News

[ apps@erp -computation-4 kafka_2.11-1.1.0]$ bin/kafka-console-consumer.sh --bootstrap-server 10.17.156.8:9092 --topic test --from-beginning

my name is xiaoqiang

At this point, Kafka is started and used normally. CTRL + C, close the xshell window and focus on the code. When testing the application, it is found that a pile of error reports all point to Kafka. Fight the Kafka server again and find that the Kafka process is no longer present

Kafka starts normally

The application starts normally

ctrl+c

Application connection Kafka error

The Kafka process was killed

Finally, the problem is found: when exiting Kafka, do not use Ctrl + C, but use the exit function to exit