filebeat output   Redis reports an error I/O timeout

Post the error report first

ERROR redis/client. go:214 Failed to RPUSH to redis list with write tcp 172.16.73.32:54822->172.16.73.33:52611:   i/o timeout
ERROR redis/client. go:92  Failed to publish events: write tcp 1172.16.73.32:54822->172.16.73.33:52611: i/o timeout

The error content is that the writing timed out. Analyze the problem. When the redis service information is normal, the first possibility is that the message queue is blocked, and the second possibility is that the network connecting to the redis server is not very good

1. Redis message queue is blocked

This is my mistake. There was a problem with the configuration and parsing of logstash when reading redis, so the message queue was blocked and filebeat could not be written in

Solution:

First check the input configuration of logstash:

Correctly configure the instance:

input {
    redis {
      host => "172.16.73.33"  #redis ip
      port => "52611"         #redis port
      password => "123456" #redis password
      db => 9 # Specify the redis library number
      batch_count => 1 # This default value of 125 refers to how many data to read from redis at once, I will write about why this is set to 1 in another article.
      data_type => "list" # data type
      key => "filebeat" #key value name
　　　}
}

The output configuration also needs to be specified correctly, otherwise the output will be blocked if it does not go out

Check whether input is blocked or output is blocked   If it’s blocked, we can first let the output output to the console, and then see if we can output data. If we can, it’s output   The configuration of is incorrect

On the contrary, our input cannot get the message queue. Check whether the input configuration is correct

  2. It’s not good to connect to the redis network

Due to some network reasons, our connection to redis will timeout, resulting in io   Timeout. We can set the timeout in filebeat to avoid this problem

filebeat.yml

output.redis:
  hosts: ["172.16.73.33:52611"] 
  password: "123456"
  key: "filebeat"
  db: 9
  timeout: 60   #Add this parameter to set the timeout time, the default value of this is 5.

Add timeout   Parameter is used to set the timeout. To mitigate the impact of the network. The specific setting value is set according to your business situation

  3. The redis version is too low

The official recommended redis version is above version 2.6, so if we use redis 2.6   The following versions are most likely caused by the version. We can use versions above 2.6 for testing. See if the problem is caused by the version

Installation document of new version of redis: redis installation and error