Geeks, please accept the hero post of 2021 Microsoft x Intel hacking contest>>>

Today, a nginx is deployed. Web page access always reports 500 errors (500 internal server error). After entering the error.log of nginx, it is found that it has been reporting too many open files

2018/08/21 10:07:35 [crit] 13508#13508: accept4() failed (24: Too many open files)
2018/08/21 10:07:35 [crit] 13508#13508: accept4() failed (24: Too many open files)
2018/08/21 10:07:36 [crit] 13508#13508: accept4() failed (24: Too many open files)
2018/08/21 10:07:36 [crit] 13508#13508: accept4() failed (24: Too many open files)
2018/08/21 10:07:37 [crit] 13508#13508: accept4() failed (24: Too many open files)
2018/08/21 10:07:37 [crit] 13508#13508: accept4() failed (24: Too many open files)
2018/08/21 10:07:38 [crit] 13508#13508: accept4() failed (24: Too many open files)
2018/08/21 10:07:38 [crit] 13508#13508: accept4() failed (24: Too many open files)
2018/08/21 10:07:39 [crit] 13508#13508: accept4() failed (24: Too many open files)
2018/08/21 10:07:39 [crit] 13508#13508: accept4() failed (24: Too many open files)
2018/08/21 10:07:40 [crit] 13508#13508: accept4() failed (24: Too many open files)
2018/08/21 10:07:40 [crit] 13508#13508: accept4() failed (24: Too many open files)
2018/08/21 10:07:41 [crit] 13508#13508: accept4() failed (24: Too many open files)
2018/08/21 10:07:41 [crit] 13508#13508: accept4() failed (24: Too many open files)
2018/08/21 10:07:42 [crit] 13508#13508: accept4() failed (24: Too many open files)
2018/08/21 10:07:42 [crit] 13508#13508: accept4() failed (24: Too many open files)
2018/08/21 10:07:43 [crit] 13508#13508: accept4() failed (24: Too many open files)
2018/08/21 10:07:43 [crit] 13508#13508: accept4() failed (24: Too many open files)
2018/08/21 10:07:44 [crit] 13508#13508: accept4() failed (24: Too many open files)
2018/08/21 10:07:44 [crit] 13508#13508: accept4() failed (24: Too many open files)
2018/08/21 10:07:45 [crit] 13508#13508: accept4() failed (24: Too many open files)
2018/08/21 10:07:45 [crit] 13508#13508: accept4() failed (24: Too many open files)
2018/08/21 10:07:46 [crit] 13508#13508: accept4() failed (24: Too many open files)
2018/08/21 10:07:46 [crit] 13508#13508: accept4() failed (24: Too many open files)
2018/08/21 10:07:47 [crit] 13508#13508: accept4() failed (24: Too many open files)
2018/08/21 10:07:47 [crit] 13508#13508: accept4() failed (24: Too many open files)
2018/08/21 10:07:48 [crit] 13508#13508: accept4() failed (24: Too many open files)

I guess the file open is beyond the upper limit set by the system, so I log in to the server to check

Run the following command to view

[root@server nginx]$ ulimit -n  
1024
[root@server nginx]$ 

It’s only 1024. Since it’s so small, let’s revise it

But before we modify it, let’s confirm the upper limit of the system

[root@server nginx]# sysctl -n -e fs.file-max
13057331
[root@server nginx]# 

Since there are so many, I don’t worry. I don’t want many

Start to modify the maximum number of open files, in the/etc/security/limits. Conf file, and add

* soft nofile 10240
* hard nofile 10240

# /etc/security/limits.conf
#
#This file sets the resource limits for the users logged in via PAM.
#It does not affect resource limits of the system services.
#
#Also note that configuration files in /etc/security/limits.d directory,
#which are read in alphabetical order, override the settings in this
#file in case the domain is the same or more specific.
#That means for example that setting a limit for wildcard domain here
#can be overriden with a wildcard setting in a config file in the
#subdirectory, but a user specific setting here can be overriden only
#with a user specific setting in the subdirectory.
#
#Each line describes a limit for a user in the form:
#
#<domain>        <type>  <item>  <value>
#
#Where:
#<domain> can be:
#        - a user name
#        - a group name, with @group syntax
#        - the wildcard *, for default entry
#        - the wildcard %, can be also used with %group syntax,
#                 for maxlogin limit
#
#<type> can have the two values:
#        - "soft" for enforcing the soft limits
#        - "hard" for enforcing hard limits
#
#<item> can be one of the following:
#        - core - limits the core file size (KB)
#        - data - max data size (KB)
#        - fsize - maximum filesize (KB)
#        - memlock - max locked-in-memory address space (KB)
#        - nofile - max number of open file descriptors
#        - rss - max resident set size (KB)
#        - stack - max stack size (KB)
#        - cpu - max CPU time (MIN)
#        - nproc - max number of processes
#        - as - address space limit (KB)
#        - maxlogins - max number of logins for this user
#        - maxsyslogins - max number of logins on the system
#        - priority - the priority to run user process with
#        - locks - max number of file locks the user can hold
#        - sigpending - max number of pending signals
#        - msgqueue - max memory used by POSIX message queues (bytes)
#        - nice - max nice priority allowed to raise to values: [-20, 19]
#        - rtprio - max realtime priority
#
#<domain>      <type>  <item>         <value>
#

#*               soft    core            0
#*               hard    rss             10000
#@student        hard    nproc           20
#@faculty        soft    nproc           20
#@faculty        hard    nproc           50
#ftp             hard    nproc           0
#@student        -       maxlogins       4

* soft nofile 10240
* hard nofile 10240
# End of file

Then log out, go in and check it, and find that it has been set

[root@server nginx]$ ulimit -n  
10240
[root@server nginx]$ 

Then restart nginx, it should be OK