of traffic. Rate limits can be set for each task type, or globally for all. Routing Using AMQP’s flexible routing model you can route tasks to different workers, or select different message topologies, Routing Figure 2.2: Figure 2: Worker bindings. Go to Routing Tasks. Celery takes advantage of AMQPs flexible routing model. Tasks can be routed to specific servers, or a cluster of servers by binding workers Let’s pretend we have an application with lot of different tasks: some process video, others process images, and some gather collective intelligence about its users. Some of these tasks are more important

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 93 5.9 Downloading Item Images . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 97 5.10 Ubuntu packages glance 7 Scrapy Documentation, Release 0.16.5 • A media pipeline for automatically downloading images (or any other media) associated with the scraped items • Support for extending Scrapy by plugging