Skip to content

Celery Cleanup Task Excluded In Schedule #902

Open
@ghost

Description

Summary

When upgrading to 2.8.1 we've had a unit test fail for our custom Scheduler class, due to celery.backend_cleanup missing from the scheduler's schedule.

Detailed Information

It appears this is intended to be included, as it is explicitly added to the hours_to_include:

# Window of +/- 2 hours around the current hour in server tz.
hours_to_include = [
(server_hour + offset) % 24 for offset in range(-2, 3)
]
hours_to_include += [4] # celery's default cleanup task

Yet the default schedule in DatabaseScheduler.install_default_entries() does not specify a timezone for celery.backend_cleanup, so the crontab will default to CELERY_TIMEZONE:

def crontab_schedule_celery_timezone():
"""Return timezone string from Django settings ``CELERY_TIMEZONE`` variable.
If is not defined or is not a valid timezone, return ``"UTC"`` instead.
"""
try:
CELERY_TIMEZONE = getattr(
settings, '%s_TIMEZONE' % current_app.namespace)
except AttributeError:
return 'UTC'
if CELERY_TIMEZONE in available_timezones():
return CELERY_TIMEZONE
return 'UTC'

This then may be excluded, as the server_hour for the cleanup task may not be within the hours_to_include:

# Calculate server-hour based on timezone offset
server_hour=Case(
# Handle each timezone specifically
*[
When(
timezone=timezone_name,
then=(
F('hour_int')
+ self._get_timezone_offset(timezone_name)
+ 24
) % 24
)
for timezone_name in self._get_unique_timezone_names()
],
# Default case - use hour as is
default=F('hour_int')
)

e.g.
Expected hour: 4
sever_hour in Australia/Melbourne: 17

All tasks appear to still be running fine, but just wanted to point this out as it seems unintentional.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions