Written in Python Available in Python | Platform Cross-platform | |
Stable release 4.0.0 / November 4, 2016; 3 months ago (2016-11-04) Repository github.com/celery/celery Type Message-oriented middleware | ||
Celery is an open source asynchronous task queue or job queue which is based on distributed message passing. While it supports scheduling, its focus is on operations in real time.
Contents
Overview
The execution units, called tasks, are executed concurrently on one or more worker nodes using multiprocessing, eventlet or gevent. Tasks can execute asynchronously (in the background) or synchronously (wait until ready). Celery is used in production systems, for instance Instagram, to process millions of tasks every day.
Technology
Celery is written in Python, but the protocol can be implemented in any language. It can also operate with other languages using webhooks. There is also a Ruby-Client called RCelery, a PHP client, and a Node.js client.
The recommended message broker is RabbitMQ, or Redis. Additionally, MongoDB, Beanstalk, Amazon SQS, CouchDB, IronMQ, and databases (using SQLAlchemy or the Django ORM) are supported in status experimental.