Tag Archives: redis

How to SolveRedis adding hash Error

Error message

127.0.0.1:6379> hset ii name ss
(error) MISCONF Redis is configured to save RDB snapshots, but it is currently not able to persist on disk. Commands that may modify the data set are disabled, because this instance is configured to report errors during writes if RDB snapshotting fails (stop-writes-on-bgsave-error option). Please check the Redis logs for details about the RDB error.
127.0.0.1:6379> config set stop-writes-on-bgsave-error no

(error) the wrong redis is configured to save RDB snapshots, but it cannot be persisted on disk at present. Disable commands that may modify datasets because this instance is configured to report errors during writes when RDB snapshots fail (stop writes on the bgsave error option). For details of RDB errors, please check redis logs.

Solution: run the redis command line

config set stop-writes-on-bgsave-error no

Redis: How to Use the command to delete keys in batches

In redis, there is no command to delete keys directly according to regular expressions, only the Del key1 command

However, redis has a command to obtain keys through regular expressions: keys “regular expressions”

You can delete keys in batches with the help of xargs command, and pass the found key value to del as a parameter

redis-cli keys “xxxx*” | xargs del

Full command

redis-cli -h 192.169.1.71 -p 7001 -a 123456 keys xxxx* | xargs -r -t -n1 redis-cli -h 192.169.1.71 -p 7001 -a 123456 del

[note]

The xargs command needs to be followed by the parameter – R, otherwise when the number of keys is 0, an error will be reported err wrong number of arguments for ‘del’ command

The parameter – N1 should be added after the xargs command. Otherwise, when the number of cluster keys is greater than 1, an error may be reported: crosslot keys in request don’t hash to the same slot

It is OK not to add – T. adding – t will output the content deleted each time. If not, the deleted content will not be output, but the number of keys deleted each time will still be output

Redis uses connection pool to solve the problem of error reporting

Redis uses the connection pool to report errors
redis has been reporting exceptions for more than ten hours

redis.clients.jedis.exceptions.JedisConnectionException: Could not get a resource from the pool  
    at redis.clients.util.Pool.getResource(Pool.java:22)  
    at com.derbysoft.jredis.longkeytest.BorrowObject.run(BorrowObject.java:22)  
    at java.lang.Thread.run(Thread.java:662)  
Caused by: java.util.NoSuchElementException: Timeout waiting for idle object  
    at org.apache.commons.pool.impl.GenericObjectPool.borrowObject(GenericObjectPool.java:1134)  
    at redis.clients.util.Pool.getResource(Pool.java:20)  

The reason is that there is no recycling of resources, resulting in
the correct approach is

ShardedJedis jedis = shardedJedisPool.getResource();
try {
    jedis.set(key, value);
} finally {
    shardedJedisPool.returnResource(jedis);
}

 

Update content:

If you use jedis 2.4.2 or earlier, don’t forget to return and connect to the resource pool

However, after version 2.5.0, jedis uses try with resource. When jedis runs out, it will be returned automatically. You don’t have to return every time

The parameters of various link numbers can be adjusted by yourself. For example, maxtotal, maxidle, etc~

If the redis server is not in the LAN, the link may timeout due to network problems, and a socket exception will be thrown. Of course, the write data will also be lost

The connection timeout is 2000ms by default. You can adjust it larger by yourself, so that data will not be lost when the network is slow

A very important point: during development, method a calls method B. method a needs to obtain jedis links, and method B also needs to obtain jedis links. During implementation, try to make them public connections. For example, B uses the link of a, so that efficiency will be much faster and links will not be wasted

the most important thing is to answer from Google foreigners:

1.I test it using a multi-thread code, when the poolSize > maxclients(redis.conf),
it will throw this exception. And it runs well when the maxclients is larger than poolSize.

2.If you set maxclients to 10, and set fixed thread pool size to 20, it prints “.. fail to get connection…”.
If you set maxclients to 10, and set fixed thread pool size to 5, it runs well.

 

Thus, in the redis configuration file and redis.conf file, there is a parameter setting, maxclients (commented out by default. If this value is not set, there is no limit on the number of redis client connections)

When setting, the value of maxclients should be greater than poolsize (the maximum number of links in the connection pool)

 

[Solved] Redis Cluster: (error) CLUSTERDOWN The cluster is down

After replacing the computer, copy the virtual machine on the original computer to the new computer, start the CentOS system on the virtual machine, and then start the redis cluster (redis5 version). It is found that the cluster can be started and the redis process can also be started, but an error is reported when connecting any node in the cluster, as shown below

To view the cluster configuration information of a single node:

It is found that the error is caused by the original cluster configuration information. You need to delete the dump.rdb and nodes.conf files under each redis instance, and then re create the cluster

Recreate the cluster:

Start 6 redis instances first

2. Enter any redis instance and execute the cluster creation command:

./redis-cli –cluster create 192.168.5.108:8001 192.168.5.108:8002 192.168.5.108:8003   192.168.5.108:8004   192.168.5.108:8005 192.168.5.108:8006 –cluster-replicas 1

Enter yes to continue. You can see that the cluster node is created successfully

Test cluster:

 

PHP xampp Windows environment installation extension redis fatal error: class’ redis’ not found solution

Front end developers must read! Starting from scratch, teach you to build a low code website platform in stages>>>

PHP xampp Windows environment installation extension redis fatal error: class’ redis’ not found solution

Reference article:

(1) PHP xampp Windows environment installation extension redis fatal error: class’ redis’ not found solution

(2) https://www.cnblogs.com/zdz8207/p/php-7-xampp-redis.html

Let’s make a note.

[Solved] Redis has java.net.socketexception: permission denied: connect problem during stress test

Recently, we are considering using redis’s message queue to solve the problem of high concurrency business scenario of company’s voucher collection and integral exchange. Therefore, we have encountered many exceptions caused by improper use of redis, so we share the solution process of one of the exceptions

Since I was written in Java, I used jedis, version 2.2. At first, I mainly used lpush, rpop, hget and hset in redis for stress testing. There were 100 threads, and each thread had 1000 cycles, so there were few problems. When I use hincr, hsetnx and other operations, after testing for more than ten seconds, there will be a java.net.socketexception: permission denied: connect error. At first, I searched a lot of data, but I seldom mentioned this error. We have to search the standard writing method of redis on the Internet and improve our own redis tool class. Finally, we mainly optimize the code of redis resource return, which was not used in the previous resource return. In the exception capture processing block, we use returnbrokenresource for processing

According to a blog post on the Internet

a. When getting an instance of jedis, there are actually two types of errors
one is pool. Getresource(), which can’t get available jedis instances
the other is jedis.set/get, which will also throw an exception when there is an error
in order to realize the distinction, it is implemented according to whether the instance is null or not. If it is empty, it means that the instance is not initialized at all, and there is no need to return to the pool; If instance is not null, it needs to be returned to pool

b. When there is an error in instance, you must call returnbrokenresource to return it to pool. Otherwise, there may be data in the buffer of instance obtained by getresource next time, which will cause problems

The operation steps of jedis are as follows:
1 – > To get the jedis instance, you need to get it from the jedispool
2-> When the jedis instance is used up, it needs to be returned to jedispool
3-> If there is an error in the use of jedis, you also need to return it to jedispool

    public long hsetnx(String key, String field,String value) {
    	long isExist =0;
    	ShardedJedis shardedJedis =null;
    	try {
    		shardedJedis = (ShardedJedis) shardedJedisPool.getResource();//shardedJedisPool为redis链接池
			if(shardedJedis != null){
				isExist = shardedJedis.hsetnx(key, field, value);
			}
    	}catch (Exception e) {
    		closeBroken(shardedJedis);//Mainly add this closeBroken method
			e.printStackTrace();
		}finally{
			close(shardedJedis);
		}
		return isExist;
		
    }

    public void closeBroken(ShardedJedis shardedJedis){
		if(shardedJedis!=null&&shardedJedisPool!=null){
			shardedJedisPool.returnBrokenResource(shardedJedis);
		}
	}
	
	public void close(ShardedJedis shardedJedis) {
		try {
			if(shardedJedis!=null&&shardedJedisPool!=null){
			shardedJedisPool.returnResource(shardedJedis);
			}
		} catch (Exception e) {
            e.printStackTrace();
		}
	}

After the stress test, there is no frequent connection rejection error

 

[Solved] Redis Operation exception: java.lang.ClassCastException: java.lang.Long cannot be cast to [B

Background Description:
(1) A project, using redis to do storage, using the set nature of redis to do real-time statistics, but also store other statistics;
(2) the use of a lot of key, value set volume more;
(3) every day at zero time, back to clean up all the data in the current redis;
(4) exceptions occur after the zero point cleanup;

Exception site:

2018/03/13 00:00:20 OSS INFO [com.xxx.RedisDAO] – error
redis.clients.jedis.exceptions.JedisConnectionException: java.net.SocketTimeoutException: Read timed out
at redis.clients.util.RedisInputStream.ensureFill(RedisInputStream.java:202)
at redis.clients.util.RedisInputStream.readByte(RedisInputStream.java:40)
at redis.clients.jedis.Protocol.process(Protocol.java:151)
at redis.clients.jedis.Protocol.read(Protocol.java:215)
at redis.clients.jedis.Connection.readProtocolWithCheckingBroken(Connection.java:340)
at redis.clients.jedis.Connection.getIntegerReply(Connection.java:265)
at redis.clients.jedis.Jedis.sadd(Jedis.java:1109)
at com.xxx.RedisDAO.setXXXKPI(RedisDAO.java:66)
at com.xxx.xxxRunnable.run(xxxRunnable.java:69)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.net.SocketTimeoutException: Read timed out
at java.net.SocketInputStream.socketRead0(Native Method)
at java.net.SocketInputStream.socketRead(SocketInputStream.java:116)
at java.net.SocketInputStream.read(SocketInputStream.java:171)
at java.net.SocketInputStream.read(SocketInputStream.java:141)
at java.net.SocketInputStream.read(SocketInputStream.java:127)
at redis.clients.util.RedisInputStream.ensureFill(RedisInputStream.java:196)
… 9 more
2018/03/13 00:00:20 OSS INFO [com.xxx.RedisDAO] – error
java.lang.ClassCastException: java.lang.Long cannot be cast to [B
at redis.clients.jedis.Connection.getStatusCodeReply(Connection.java:239)
at redis.clients.jedis.Jedis.set(Jedis.java:121)
at com.xxx.RedisDAO.setXXXXXKPI(RedisDAO.java:100)
at com.xxx.xxxRunnable.run(xxxRunnable.java:70)
at java.lang.Thread.run(Thread.java:745)
2018/03/13 00:00:20 OSS INFO [com.xxx.RedisDAO] – error
java.lang.ClassCastException: [B cannot be cast to java.lang.Long
at redis.clients.jedis.Connection.getIntegerReply(Connection.java:265)
at redis.clients.jedis.Jedis.sadd(Jedis.java:1109)
at com.xxx.RedisDAO.setXXXXXKPI(RedisDAO.java:167)
at com.xxx.xxxRunnable.run(xxxRunnable.java:71)
at java.lang.Thread.run(Thread.java:745)

 

(1) If there is a lot of data in the current redis, FLUSHDB operation, it takes a certain amount of time, blocking the response of redis, so the SocketTimeoutException appears first;
(2) After the connection exception, from the logs, then there is a ClassCastException, and the subsequent keep reporting this exception error;

 

In summary, checking the project code, we found several problems:
a. After an exception occurs in a redis connection operation, it should close the close, return the resource, and go to a redis connection again to use it. The program is written as follows: an exception is thrown, a log is recorded, and the redis connection is used again, and the exception is thrown all the time.
b. The connection pool is set too large, when restarting the project, the initialized connection plus the previously unreleased connection (directly kill the process, not released when closing), the instant link is too large;
c. Program design is not reasonable, because the data is displayed in real time statistics every day, you can set the key prefix different every day, the key expiration time, so that you do not have to FLUSHDB every day.

 

 

[Solved] Redis MISCONF ERROR: Redis is configured to save RDB snapshots

Today I ran the Magento2 update command on a new server and found that Redis was having problems

Warning: MISCONF Redis is configured to save RDB snapshots, but it is currently not able to persist on disk. Commands that may modify the data set are disabled, because this instance is configured to report errors during writes if RDB snapshotting fails (stop-writes-on-bgsave-error option). Please check the Redis logs for details about the RDB error.

Solution.

1. Execute: redis-cli, enter redis client;

2, then execute: config set stop-writes-on-bgsave-error no

OK

How to Solve Redis Error: OOM command not allowed when used memory > ‘maxmemory’

exception prompt: the memory is full and no more data can be saved. You can view the specific information of redis through redis cli

127.0.0.1:6379> info memory
# Memory
used_memory:168363320
used_memory_human:160.56M
used_memory_rss:150208512
used_memory_rss_human:143.25M
used_memory_peak:168425144
used_memory_peak_human:160.62M
used_memory_peak_perc:99.96%
used_memory_overhead:139522822
used_memory_startup:487096
used_memory_dataset:28840498
used_memory_dataset_perc:17.18%
total_system_memory:4143534080
total_system_memory_human:3.86G
used_memory_lua:37888
used_memory_lua_human:37.00K
maxmemory:124000000
maxmemory_human:118.26M
maxmemory_policy:noeviction
mem_fragmentation_ratio:0.89
mem_allocator:jemalloc-4.0.3
active_defrag_running:0
lazyfree_pending_objects:0

used_ memory_ Human represents used memory

used_ memory_ RSS represents the memory allocated by the system to redis (i.e. resident memory)

mem_ fragmentation_ ratio=used_ memory_ rss/used_ Memory ratio, in general, used_ memory_ RSS is slightly higher than used_ Memory, when there are many memory fragments, then memory_ fragmentation_ The ratio will be large, which can reflect whether there are many memory fragments

solutions:

1. Add redis memory and modify redis.conf

maxmemory 2gb

2. Modify the redis storage policy

The default redis setting is very conservative, that is, it will not be stored after memory overrun. You can change the policy to LRU algorithm (the least recently used algorithm) – the new stored information will replace the old information, so that the memory will not cross the line and redis.conf will not be modified

maxmemory-policy volatile-lru

3. Save unnecessary data in redis

After modifying the configuration, you must restart redis to take effect

Redis service failed to start, prompt: redis server: command not found

Under the Linux operating system, I download and install the Linux version of redis. Since the first installation of redis is based on the tutorial, each step is operated step by step according to the tutorial. Finally, the installation is successful, and the command execution is also very smooth. Although the execution is complete, I don’t know much about the operation. There was a problem when installing redis today. The screenshot is as follows:

I’m talking about my installation steps:

Step 1: create a new directory and enter it

[root@linux ~]# mkdir -p software/download/redis

               [root@linux ~]# cd software/download/redis

               [root@linux redis]#

Step 2: in the current directory, execute WGet command to start downloading redis package

             [root@linux redis]# pwd
             /root/software/download/redis

             [root@linux redis]# wget http://redis.io/download/redis-4.0.tar.gz
             //.....
             //start to download
             //...
             //download finish

Step 3: decompress the redis file in the current directory

             [root@linux redis]# pwd
             /root/software/download/redis

             [root@linux redis]# tar zxvf redis-4.0.6.tar.gz
             OK

             [root@linux redis]# ls
             redis-4.0.6   redis-4.0.6.tar.gz

Step 4: enter the redis root directory, execute the make command, and start compiling

[root@linux redis]# cd redis-4.0.6
             [root@linux redis-4.0.6]# make
             //Start compiling
             //....
             //Complete compilation

Step 5: copy the relevant files to the specified directory with the directory address of/root/application/program/redis to complete the operation

             [root@linux redis]# pwd
             /root/software/download/redis

             [root@linux redis]# cp redis.conf /root/application/program/redis/
             [root@linux redis]# cd src

             [root@linux src]# cp redis-cli /root/application/program/redis/
             OK

             [root@linux src]# cp redis-server /root/application/program/redis/
             OK

             [root@linux src]# cp redis-benchmark /root/application/program/redis/
             OK

after the above five steps are completed, and the redis.conf configuration file has been modified, I think redis server can be started. When I execute the following command, I will prompt an error

[root@linux~]# cd application/program/redis

          [root@linux redis]# redis-server redis.conf
          -bash:redis-server:command not found

          //It is also the screenshot above

After debugging, I found that I had omitted the sixth step. I thought the make command could be executed and compiled. There was no need to execute the make install command. I thought that the function of the make install command was to copy the redis related files to the/usr/local/bin directory. It turned out that if it was not executed, the redis server redis.conf command would not be executed

The following is the sixth step. After this process, we will have a deeper understanding

Step 6: go back to the/root/software/download/redis/redis-4.0.6/src directory and execute the make install command to complete all the installation

[root@linux~]# cd software/download/redis/resis-4.0.6/src
          [root@linux src]# make install
          //Start installation
          //...
          //Installation complete

Well, the problem has been solved. I find a truth that “failure is the mother of success”. Through continuous operation and debugging, the understanding will be different, especially for software development. Start to test the cluster mode of master-slave replication of two Linux versions of redis