bin.worker

Program used to start a Celery worker instance.

The celery worker command (previously known as celeryd)

See also

See preload-options.

-c, --concurrency

Number of child processes processing the queue. The default is the number of CPUs available on your system.

-P, --pool

Pool implementation:

prefork (default), eventlet, gevent or solo.

-n, --hostname

Set custom hostname (e.g., ‘w1@%%h’). Expands: %%h (hostname), %%n (name) and %%d, (domain).

-B, --beat

Also run the celery beat periodic task scheduler. Please note that there must only be one instance of this service.

Note

-B is meant to be used for development purposes. For production environment, you need to start celery beat separately.

-Q, --queues

List of queues to enable for this worker, separated by comma. By default all configured queues are enabled. Example: -Q video,image

-X, --exclude-queues

List of queues to disable for this worker, separated by comma. By default all configured queues are enabled. Example: -X video,image.

-I, --include

Comma separated list of additional modules to import. Example: -I foo.tasks,bar.tasks

-s, --schedule

Path to the schedule database if running with the -B option. Defaults to celerybeat-schedule. The extension “.db” may be appended to the filename.

-O

Apply optimization profile. Supported: default, fair

--prefetch-multiplier

Set custom prefetch multiplier value for this worker instance.

--scheduler

Scheduler class to use. Default is celery.beat.PersistentScheduler

-S, --statedb

Path to the state database. The extension ‘.db’ may be appended to the filename. Default: {default}

-E, --task-events

Send task-related events that can be captured by monitors like celery events, celerymon, and others.

--without-gossip

Don’t subscribe to other workers events.

--without-mingle

Don’t synchronize with other workers at start-up.

--without-heartbeat

Don’t send event heartbeats.

--heartbeat-interval

Interval in seconds at which to send worker heartbeat

--purge

Purges all waiting tasks before the daemon is started. WARNING: This is unrecoverable, and the tasks will be deleted from the messaging server.

--time-limit

Enables a hard time limit (in seconds int/float) for tasks.

--soft-time-limit

Enables a soft time limit (in seconds int/float) for tasks.

--max-tasks-per-child

Maximum number of tasks a pool worker can execute before it’s terminated and replaced by a new worker.

--max-memory-per-child

Maximum amount of resident memory, in KiB, that may be consumed by a child process before it will be replaced by a new one. If a single task causes a child process to exceed this limit, the task will be completed and the child process will be replaced afterwards. Default: no limit.

--autoscale

Enable autoscaling by providing max_concurrency, min_concurrency. Example:

--autoscale=10,3

(always keep 3 processes, but grow to 10 if necessary)

--detach

Start worker as a background process.

-f, --logfile

Path to log file. If no logfile is specified, stderr is used.

-l, --loglevel

Logging level, choose between DEBUG, INFO, WARNING, ERROR, CRITICAL, or FATAL.

--pidfile

Optional file used to store the process pid.

The program won’t start if this file already exists and the pid is still alive.

--uid

User id, or user name of the user to run as after detaching.

--gid

Group id, or group name of the main group to change to after detaching.

--umask

Effective umask(1) (in octal) of the process after detaching. Inherits the umask(1) of the parent process by default.

--workdir

Optional directory to change to after detaching.

--executable

Executable to use for the detached process.

Module Contents

Classes

worker() Start worker instance.

Functions

main(app=None) Start worker.
class worker

Start worker instance.

Examples:
$ celery worker --app=proj -l info
$ celery worker -A proj -l info -Q hipri,lopri

$ celery worker -A proj --concurrency=4
$ celery worker -A proj --concurrency=1000 -P eventlet
$ celery worker --autoscale=10,0
run_from_argv(prog_name, argv=None, command=None)
maybe_detach(argv, dopts=list)
run(hostname=None, pool_cls=None, app=None, uid=None, gid=None, loglevel=None, logfile=None, pidfile=None, statedb=None, **kwargs)
with_pool_option(argv)
add_arguments(parser)
main(app=None)

Start worker.