Abortable tasks overview¶
Task’s, it can be desirable to support
aborting during execution. Of course, these tasks should be built to
support abortion specifically.
AbortableTask serves as a base class for all
objects that should support abortion by producers.
- Producers may invoke the
AbortableAsyncResultinstances, to request abortion.
- Consumers (workers) should periodically check (and honor!) the
is_aborted()method at controlled points in their task’s
run()method. The more often, the better.
The necessary intermediate communication is dealt with by the
In the consumer:
from __future__ import absolute_import from celery.contrib.abortable import AbortableTask from celery.utils.log import get_task_logger from proj.celery import app logger = get_logger(__name__) @app.task(bind=True, base=AbortableTask) def long_running_task(self): results =  for i in range(100): # check after every 5 iterations... # (or alternatively, check when some timer is due) if not i % 5: if self.is_aborted(): # respect aborted state, and terminate gracefully. logger.warning('Task aborted') return value = do_something_expensive(i) results.append(y) logger.info('Task complete') return results
In the producer:
from __future__ import absolute_import import time from proj.tasks import MyLongRunningTask def myview(request): # result is of type AbortableAsyncResult result = long_running_task.delay() # abort the task after 10 seconds time.sleep(10) result.abort()
After the result.abort() call, the task execution isn’t aborted immediately. In fact, it’s not guaranteed to abort at all. Keep checking result.state status, or call result.get(timeout=) to have it block until the task is finished.
In order to abort tasks, there needs to be communication between the producer and the consumer. This is currently implemented through the database backend. Therefore, this class will only work with the database backends.
||Represents an abortable result.|
||Task that can be aborted.|
Represents an abortable result.
Specifically, this gives the AsyncResult a
abort()method, that sets the state of the underlying Task to ‘ABORTED’.
Trueif the task is (being) aborted.
Set the state of the task to
Abortable tasks monitor their state at regular intervals and terminate execution if so.
- Be aware that invoking this method does not guarantee when the task will be aborted (or even if the task will be aborted at all).
Task that can be aborted.
This serves as a base class for all
Task’s that support aborting during execution.
Return the accompanying AbortableAsyncResult instance.
Return true if task is aborted.
Checks against the backend whether this
Falsein case the task_id parameter refers to a regular (non-abortable)
Be aware that invoking this method will cause a hit in the backend (for example a database query), so find a good balance between calling it regularly (for responsiveness), but not too often (for performance).