Tag Archives: celery

Error reported in the celery log of the project: redis.exceptions.connectionerror failed to connect with redis

When the project in the test environment is running, check uwsgi.log. If the scheduled task does not run, check the cell_ Beat.log log shows: redis.exceptions.connectionerror: error 24 connecting to 127.0.0.1:6379. Too many open files

On the command line, enter redis to check that the data is empty and there is no data.

If the celery is restarted regularly, it can be effective for a while and cannot be stable for a long time.

Compared with the online environment, the redis version is redis cli 2.8.18   The redis version of the test environment reporting an error is 4.0.10, which is too high.

Reinstall the lower version of redis     On the official website https://redis.io/ Find redis version 2.8.18 in the, extract and install it to the redis server version 2.8.18

After downloading, I uploaded the installation package to/usr/local/for installation

/Redis.conf in etc/   / In bin/is mainly mkreleasehdr.sh     redis-benchmark   redis-check-aof     redis-cli     redis-server     It’s all moved by MV command,

The original location is in/usr/local/redis-2.8.18/src/  After moving, it is convenient for subsequent redis startup

Stop the old version of 4.0.10 redis and kill, and delete the corresponding directory files

Start the newly installed version 2.8.18 redis service:/usr/local/redis-2.8.18/bin/redis-server      / usr/local/redis-2.8.18/etc/redis.conf

 

  The version of redis has been reduced. Then enter the project directory to check the log. It runs normally at regular intervals. There is also data in redis

Error report of cell timed task startup: KeyError: ‘scheduler’ KeyError: ‘entries’

An error was reported when starting celery:KeyError: 'scheduler' KeyError: 'entries'

[2021-08-25 17:23:50,410: INFO/MainProcess] beat: Starting...
[2021-08-25 17:23:50,418: CRITICAL/MainProcess] beat raised exception <class 'EOFError'>: EOFError('Ran out of input')
Traceback (most recent call last):
  File "f:\python_envs\jiankong\lib\site-packages\kombu\utils\objects.py", line 42, in __get__
    return obj.__dict__[self.__name__]
KeyError: 'scheduler'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "f:\python376\lib\shelve.py", line 111, in __getitem__
    value = self.cache[key]
KeyError: 'entries'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "f:\python_envs\jiankong\lib\site-packages\celery\apps\beat.py", line 109, in start_scheduler
    service.start()
  File "f:\python_envs\jiankong\lib\site-packages\celery\beat.py", line 622, in start
    humanize_seconds(self.scheduler.max_interval))
  File "f:\python_envs\jiankong\lib\site-packages\kombu\utils\objects.py", line 44, in __get__
    value = obj.__dict__[self.__name__] = self.__get(obj)
  File "f:\python_envs\jiankong\lib\site-packages\celery\beat.py", line 666, in scheduler
    return self.get_scheduler()
  File "f:\python_envs\jiankong\lib\site-packages\celery\beat.py", line 661, in get_scheduler
    lazy=lazy,
  File "f:\python_envs\jiankong\lib\site-packages\celery\beat.py", line 501, in __init__
    Scheduler.__init__(self, *args, **kwargs)
  File "f:\python_envs\jiankong\lib\site-packages\celery\beat.py", line 257, in __init__
    self.setup_schedule()
  File "f:\python_envs\jiankong\lib\site-packages\celery\beat.py", line 529, in setup_schedule
    self._create_schedule()
  File "f:\python_envs\jiankong\lib\site-packages\celery\beat.py", line 558, in _create_schedule
    self._store[str('entries')]
  File "f:\python376\lib\shelve.py", line 114, in __getitem__
    value = Unpickler(f).load()
EOFError: Ran out of input
[2021-08-25 17:23:50,421: WARNING/MainProcess] Traceback (most recent call last):
[2021-08-25 17:23:50,422: WARNING/MainProcess] File "f:\python_envs\jiankong\lib\site-packages\kombu\utils\objects.py", line 42, in __get__
[2021-08-25 17:23:50,423: WARNING/MainProcess] return obj.__dict__[self.__name__]
[2021-08-25 17:23:50,423: WARNING/MainProcess] KeyError
[2021-08-25 17:23:50,424: WARNING/MainProcess] :
[2021-08-25 17:23:50,425: WARNING/MainProcess] 'scheduler'
[2021-08-25 17:23:50,426: WARNING/MainProcess] During handling of the above exception, another exception occurred:
[2021-08-25 17:23:50,427: WARNING/MainProcess] Traceback (most recent call last):
[2021-08-25 17:23:50,427: WARNING/MainProcess] File "f:\python376\lib\shelve.py", line 111, in __getitem__
[2021-08-25 17:23:50,428: WARNING/MainProcess] value = self.cache[key]
[2021-08-25 17:23:50,429: WARNING/MainProcess] KeyError
[2021-08-25 17:23:50,431: WARNING/MainProcess] :
[2021-08-25 17:23:50,431: WARNING/MainProcess] 'entries'
[2021-08-25 17:23:50,432: WARNING/MainProcess] During handling of the above exception, another exception occurred:
[2021-08-25 17:23:50,432: WARNING/MainProcess] Traceback (most recent call last):
[2021-08-25 17:23:50,433: WARNING/MainProcess] File "f:\python376\lib\runpy.py", line 193, in _run_module_as_main
[2021-08-25 17:23:50,434: WARNING/MainProcess] "__main__", mod_spec)
[2021-08-25 17:23:50,434: WARNING/MainProcess] File "f:\python376\lib\runpy.py", line 85, in _run_code
[2021-08-25 17:23:50,435: WARNING/MainProcess] exec(code, run_globals)
[2021-08-25 17:23:50,435: WARNING/MainProcess] File "F:\Python_Envs\jiankong\Scripts\celery.exe\__main__.py", line 7, in <module>
[2021-08-25 17:23:50,436: WARNING/MainProcess] File "f:\python_envs\jiankong\lib\site-packages\celery\__main__.py", line 16, in main
[2021-08-25 17:23:50,437: WARNING/MainProcess] _main()
[2021-08-25 17:23:50,437: WARNING/MainProcess] File "f:\python_envs\jiankong\lib\site-packages\celery\bin\celery.py", line 322, in main
[2021-08-25 17:23:50,438: WARNING/MainProcess] cmd.execute_from_commandline(argv)
[2021-08-25 17:23:50,438: WARNING/MainProcess] File "f:\python_envs\jiankong\lib\site-packages\celery\bin\celery.py", line 499, in execute_from_commandline
[2021-08-25 17:23:50,439: WARNING/MainProcess] super(CeleryCommand, self).execute_from_commandline(argv)))
[2021-08-25 17:23:50,439: WARNING/MainProcess] File "f:\python_envs\jiankong\lib\site-packages\celery\bin\base.py", line 305, in execute_from_commandline
[2021-08-25 17:23:50,440: WARNING/MainProcess] return self.handle_argv(self.prog_name, argv[1:])
[2021-08-25 17:23:50,440: WARNING/MainProcess] File "f:\python_envs\jiankong\lib\site-packages\celery\bin\celery.py", line 491, in handle_argv
[2021-08-25 17:23:50,441: WARNING/MainProcess] return self.execute(command, argv)
[2021-08-25 17:23:50,441: WARNING/MainProcess] File "f:\python_envs\jiankong\lib\site-packages\celery\bin\celery.py", line 419, in execute
[2021-08-25 17:23:50,442: WARNING/MainProcess] ).run_from_argv(self.prog_name, argv[1:], command=argv[0])
[2021-08-25 17:23:50,442: WARNING/MainProcess] File "f:\python_envs\jiankong\lib\site-packages\celery\bin\base.py", line 309, in run_from_argv
[2021-08-25 17:23:50,443: WARNING/MainProcess] sys.argv if argv is None else argv, command)
[2021-08-25 17:23:50,443: WARNING/MainProcess] File "f:\python_envs\jiankong\lib\site-packages\celery\bin\base.py", line 393, in handle_argv
[2021-08-25 17:23:50,443: WARNING/MainProcess] return self(*args, **options)
[2021-08-25 17:23:50,444: WARNING/MainProcess] File "f:\python_envs\jiankong\lib\site-packages\celery\bin\base.py", line 253, in __call__
[2021-08-25 17:23:50,445: WARNING/MainProcess] ret = self.run(*args, **kwargs)
[2021-08-25 17:23:50,445: WARNING/MainProcess] File "f:\python_envs\jiankong\lib\site-packages\celery\bin\beat.py", line 109, in run
[2021-08-25 17:23:50,446: WARNING/MainProcess] return beat().run()
[2021-08-25 17:23:50,446: WARNING/MainProcess] File "f:\python_envs\jiankong\lib\site-packages\celery\apps\beat.py", line 81, in run
[2021-08-25 17:23:50,447: WARNING/MainProcess] self.start_scheduler()
[2021-08-25 17:23:50,447: WARNING/MainProcess] File "f:\python_envs\jiankong\lib\site-packages\celery\apps\beat.py", line 109, in start_scheduler
[2021-08-25 17:23:50,448: WARNING/MainProcess] service.start()
[2021-08-25 17:23:50,449: WARNING/MainProcess] File "f:\python_envs\jiankong\lib\site-packages\celery\beat.py", line 622, in start
[2021-08-25 17:23:50,450: WARNING/MainProcess] humanize_seconds(self.scheduler.max_interval))
[2021-08-25 17:23:50,451: WARNING/MainProcess] File "f:\python_envs\jiankong\lib\site-packages\kombu\utils\objects.py", line 44, in __get__
[2021-08-25 17:23:50,452: WARNING/MainProcess] value = obj.__dict__[self.__name__] = self.__get(obj)
[2021-08-25 17:23:50,452: WARNING/MainProcess] File "f:\python_envs\jiankong\lib\site-packages\celery\beat.py", line 666, in scheduler
[2021-08-25 17:23:50,454: WARNING/MainProcess] return self.get_scheduler()
[2021-08-25 17:23:50,454: WARNING/MainProcess] File "f:\python_envs\jiankong\lib\site-packages\celery\beat.py", line 661, in get_scheduler
[2021-08-25 17:23:50,455: WARNING/MainProcess] lazy=lazy,
[2021-08-25 17:23:50,456: WARNING/MainProcess] File "f:\python_envs\jiankong\lib\site-packages\celery\beat.py", line 501, in __init__
[2021-08-25 17:23:50,458: WARNING/MainProcess] Scheduler.__init__(self, *args, **kwargs)
[2021-08-25 17:23:50,458: WARNING/MainProcess] File "f:\python_envs\jiankong\lib\site-packages\celery\beat.py", line 257, in __init__
[2021-08-25 17:23:50,461: WARNING/MainProcess] self.setup_schedule()
[2021-08-25 17:23:50,462: WARNING/MainProcess] File "f:\python_envs\jiankong\lib\site-packages\celery\beat.py", line 529, in setup_schedule
[2021-08-25 17:23:50,463: WARNING/MainProcess] self._create_schedule()
[2021-08-25 17:23:50,464: WARNING/MainProcess] File "f:\python_envs\jiankong\lib\site-packages\celery\beat.py", line 558, in _create_schedule
[2021-08-25 17:23:50,465: WARNING/MainProcess] self._store[str('entries')]
[2021-08-25 17:23:50,465: WARNING/MainProcess] File "f:\python376\lib\shelve.py", line 114, in __getitem__
[2021-08-25 17:23:50,466: WARNING/MainProcess] value = Unpickler(f).load()
[2021-08-25 17:23:50,467: WARNING/MainProcess] EOFError
[2021-08-25 17:23:50,467: WARNING/MainProcess] :
[2021-08-25 17:23:50,467: WARNING/MainProcess] Ran out of input

Solution:

Delete the timing file automatically generated during startup in the celery startup directory

celerybeat-schedule.bak
celerybeat-schedule.dat
celerybeat-schedule.dir