Recently, I encountered mysqldump: error 2013 in mysqldump. I think it’s a common problem with parameter setting, and after adjustment, there is no effect. It turns out that oom happened. Here is a detailed description

1. Fault phenomenon

# more /etc/redhat-release 
CentOS Linux release 7.4.1708 (Core) 

# mysql -V           ##PXC 5.7
mysql  Ver 14.14 Distrib 5.7.20-18, for Linux (x86_64) using 6.2 # mysqldump -hlocalhost -uroot -p --port=33006 --default-character-set=utf8 -F -R -E --triggers -e \ > --single-transaction --all-databases >/tmp/alldb.sql Enter password: mysqldump: Error 2013: Lost connection to MySQL server during query when dumping table `tel_send_log` at row: 390486 The table isn't very large, and it's only about 200w rows of data. 

2. Fault analysis

At first I thought there was a problem with the relevant variable settings, so I adjusted the relevant parameters
mysql > set global connect_timeout=1800;
mysql > set global net_read_timeout=1800;
mysql > set global net_write_timeout=1800; mysql > set global max_allowed_packet=1024*1024*1024; mysql > set global net_ buffer_length=1024*1024; After exporting again, the problem remained, and after the error was reported, mysqld just hung up Checking mysqld's error log also did not get the specific information about the error, and thought it was too new version encountered bugs later added an option - skip-extended- insert (this is not enabled mysql generate insert contains multiple values) also does not work after switching to another node, the export is normal. So I opened the error system log message and saw thatout of memory # tail -fn 50 /var/log/message Apr 26 15:13:10 zcd kernel: Out of memory: Kill process 10274 (firewalld) score 5 or sacrifice child Apr 26 15:13:10 zcd kernel: Killed process 10274 (firewalld) total-vm:333984kB, anon-rss:21980kB, file-rss:44kB, shmem-rss:0kB Apr 26 15:13:10 zcd systemd: firewalld.service: main process exited, code=killed, status=9/KILL Apr 26 15:13:10 zcd mysqld_safe: /usr/bin/mysqld_safe: line 220: 9658 Killed nohup /usr/sbin/mysqld --basedir=/usr --datadir=/u02/pxcdata --plugin-dir=/usr/lib64/mysql/plugin --user=mysql --wsrep-provider=/usr/lib64/galera3/libgalera_smm.so --log-error=/var/log/mysqld.log --pid-file=/var/run/mysqld/mysqld.pid --socket=/var/lib/mysql/mysql.sock --port=33006 --wsrep_start_position=e12b32d1-06f1-11e8-b907-d7b0846b449d:68722 < /dev/null > /dev/null 2>&1 Apr 26 15:13:10 zcd mysqld_safe: 2018-04-26T07:13:10.742581Z mysqld_safe Number of processes running now: 0 Apr 26 15:13:10 zcd mysqld_safe: 2018-04-26T07:13:10.831232Z mysqld_safe WSREP: not restarting wsrep node automatically Apr 26 15:13:10 zcd mysqld_safe: 2018-04-26T07:13:10.835441Z mysqld_safe mysqld from pid file /var/run/mysqld/mysqld.pid ended So restart mysql, then export again and observe the memory usage, as follows, you can see that the free column value is decreasing procs -----------memory---------- ---swap-- -----io---- -system-- ------cpu----- r b swpd free buff cache si so bi bo in cs us sy id wa st 1 1 0 625288 31836 1347284 0 0 19840 372 3181 3394 9 3 42 46 0 0 1 0 586820 31844 1365304 0 0 19840 60 2385 3264 7 3 46 44 0 0 1 0 515076 31956 1400024 0 0 19312 564 2681 3327 7 4 43 46 0 0 1 0 478708 31964 1417948 0 0 20368 24 3057 3301 8 4 43 46 0 0 1 0 400960 31964 1452688 0 0 19840 0 2213 3278 7 3 44 47 0 0 1 0 364868 31964 1468268 0 0 19840 24648 3053 3500 7 4 40 50 0 0 2 0 324640 31976 1488112 0 0 4544 43656 1079 1688 2 2 14 83 0 0 1 0 286768 32116 1505272 0 0 19840 760 3412 3877 7 5 25 64 0 0 1 0 248916 32132 1520528 0 0 19840 36 4065 5746 13 4 43 41 0 0 1 0