california app design company

SF Django Meetup: Asynchronous Events and Architecture

February 04, 2015

Last month, we met on January 28th at InfoScout's offices to hear from their head engineer and co-founder, Jon Brelig about how they architected their asynchronous operations.

Usual Suspects

InfoScout, along with many other companies needing to implement asynchronous functions with Django use Celery and RabbitMQ. We got a quick walkthrough of the parts needed to set up celery with Django and how do you denote a method that should be run asynchronously.

Takeaways

There was some great insight if you plan on implementing celery for your Django application soon:

  • Set up different boxes for running asynchronous tasks based on what those tasks are doing
  • Create separate high priority and low priority queues
  • Do not pass ORM objects to asynchronous tasks, preferrably send the primary key and let the task fetch it from the database
  • Use Flower to monitor the status of your queue
  • Keep your tasks small so if they fail, they will fail fast. Retried failed tasks, won't have to redo a lot of work.

MeetupSlidesInfoScout

is a EVP of Tech | Founding Partner at Yeti. He found his passion for technology as a youth, spending his childhood developing games and coding websites. Rudy now resides in the Yeti Cave where he architects Yeti’s system and heads up project production. Follow Rudy on Twitter.

blog comments powered by Disqus
SF Django Meetup: Asynchronous Events and Architecture https://s3-us-west-1.amazonaws.com/yeti-site-media/uploads/blog/.thumbnails/cropped-meetup.jpg/cropped-meetup-360x0.jpg
Yeti (415) 766-4198 https://s3-us-west-1.amazonaws.com/yeti-site-static/img/yeti-head-blue.png