I'm using the combination of Celery and django-celery with RabbitMQ to do background video transcoding of videos uploaded to a Django web site. I use Supervisord to keep my gunicorn processes running, and I wanted to use to keep Celery running too. Unfortunately, it didn't work out of the box.
<!-- SUMMARY_END -->
RabbitMQ just worked, no need for further comment there, except to say that it has a cool logo and is written in Erlang.
Celery comes with a supervisord/celeryd.conf file, but recommends that you use django-celery's celeryd.conf if you're using Django. This is my very slightly edited version of the file, with paths in directory and command to match the location of my project:
[program:celery] command=/var/www/virtualenv/project/manage.py celeryd --loglevel=INFO directory=/var/www/virtualenv/project user=nobody numprocs=1 ...
When I reloaded the supervisord configs and tried to start celery it wouldn't work. I'd get one of the following error messages:
$ sudo supervisorctl status celery FATAL Exited too quickly (process log may have details) ... $ sudo supervisorctl status celery BACKOFF Exited too quickly (process log may have details) ...
tail``ing ``/var/log/celeryd.log revealed the following:
...
Traceback (most recent call last):
File "/var/www/virtualenv/project/manage.py", line 3, in <module>
from django.core.management import execute_manager
ImportError: No module named django.core.management
...
I fixed if by customizing the Python path in manage.py so that it points to the Python bin in the virtualenv. This line:
#!/usr/bin/env python
becomes this line:
#!/var/www/stage/nikkeienv/bin/python
With that change and another sudo supervisorctl reload everything worked.