georss.org/georss" xmlns:gd="http://schemas.google.com/g/2005" xmlns:thr="http://purl.org/syndication/thread/1.0" xmlns:feedburner="http://rssnamespace.org/feedburner/ext/1.0"> ... You can see several namespace set_event_loop(), which will set the specified event loop as the current loop for the current OS thread. BOT_NAME Default: 'scrapybot' The name of the bot implemented by this Scrapy project (also known REACTOR_THREADPOOL_MAXSIZE Default: 10 The maximum limit for Twisted Reactor thread pool size. This is common multi-purpose thread pool used by various Scrapy components. Threaded DNS Resolver, BlockingFeedStorage

georss.org/georss" xmlns:gd="http://schemas.google.com/g/2005" xmlns:thr="http://purl.org/syndication/thread/1.0" xmlns:feedburner="http://rssnamespace.org/feedburner/ext/1.0"> ... 3.3. Selectors 53 Scrapy REACTOR_THREADPOOL_MAXSIZE Default: 10 The maximum limit for Twisted Reactor thread pool size. This is common multi-purpose thread pool used by various Scrapy components. Threaded DNS Resolver, BlockingFeedStorage CONCURRENT_REQUESTS = 100 5.5.2 Increase Twisted IO thread pool maximum size Currently Scrapy does DNS resolution in a blocking way with usage of thread pool. With higher concurrency levels the crawling

REACTOR_THREADPOOL_MAXSIZE Default: 10 The maximum limit for Twisted Reactor thread pool size. This is common multi-purpose thread pool used by various Scrapy components. Threaded DNS Resolver, BlockingFeedStorage Documentation, Release 1.0.7 Increase Twisted IO thread pool maximum size Currently Scrapy does DNS resolution in a blocking way with usage of thread pool. With higher concurrency levels the crawling processed faster speeding up establishing of connection and crawling overall. To increase maximum thread pool size use: REACTOR_THREADPOOL_MAXSIZE = 20 Setup your own DNS If you have multiple crawling

georss.org/georss" xmlns:gd="http://schemas.google.com/g/2005" xmlns:thr="http://purl.org/syndication/thread/1.0" xmlns:feedburner="http://rssnamespace.org/feedburner/ext/1.0"> ... You can see several namespace set_event_loop(), which will set the specified event loop as the current loop for the current OS thread. 124 Chapter 3. Basic concepts Scrapy Documentation, Release 2.6.3 BOT_NAME Default: 'scrapybot' REACTOR_THREADPOOL_MAXSIZE Default: 10 The maximum limit for Twisted Reactor thread pool size. This is common multi-purpose thread pool used by various Scrapy components. Threaded DNS Resolver, BlockingFeedStorage

georss.org/georss" xmlns:gd="http://schemas.google.com/g/2005" xmlns:thr="http://purl.org/syndication/thread/1.0" xmlns:feedburner="http://rssnamespace.org/feedburner/ext/1.0"> ... You can see several namespace set_event_loop(), which will set the specified event loop as the current loop for the current OS thread. BOT_NAME Default: 'scrapybot' The name of the bot implemented by this Scrapy project (also known REACTOR_THREADPOOL_MAXSIZE Default: 10 The maximum limit for Twisted Reactor thread pool size. This is common multi-purpose thread pool used by various Scrapy components. Threaded DNS Resolver, BlockingFeedStorage

georss.org/georss" xmlns:gd="http://schemas.google.com/g/2005" xmlns:thr="http://purl.org/syndication/thread/1.0" xmlns:feedburner="http://rssnamespace.org/feedburner/ext/1.0"> ... You can see several namespace REACTOR_THREADPOOL_MAXSIZE Default: 10 The maximum limit for Twisted Reactor thread pool size. This is common multi-purpose thread pool used by various Scrapy components. Threaded DNS Resolver, BlockingFeedStorage concurrency limit accordingly. 5.5.3 Increase Twisted IO thread pool maximum size Currently Scrapy does DNS resolution in a blocking way with usage of thread pool. With higher concurrency levels the crawling

georss.org/georss" xmlns:gd="http://schemas.google.com/g/2005" xmlns:thr="http://purl.org/syndication/thread/1.0" xmlns:feedburner="http://rssnamespace.org/feedburner/ext/1.0"> ... You can see several namespace REACTOR_THREADPOOL_MAXSIZE Default: 10 The maximum limit for Twisted Reactor thread pool size. This is common multi-purpose thread pool used by various Scrapy components. Threaded DNS Resolver, BlockingFeedStorage concurrency limit accordingly. 5.5.3 Increase Twisted IO thread pool maximum size Currently Scrapy does DNS resolution in a blocking way with usage of thread pool. With higher concurrency levels the crawling

REACTOR_THREADPOOL_MAXSIZE Default: 10 The maximum limit for Twisted Reactor thread pool size. This is common multi-purpose thread pool used by various Scrapy components. Threaded DNS Resolver, BlockingFeedStorage use: CONCURRENT_REQUESTS = 100 Increase Twisted IO thread pool maximum size Currently Scrapy does DNS resolution in a blocking way with usage of thread pool. With higher concurrency levels the crawling processed faster speeding up establishing of connection and crawling overall. To increase maximum thread pool size use: REACTOR_THREADPOOL_MAXSIZE = 20 Setup your own DNS If you have multiple crawling

REACTOR_THREADPOOL_MAXSIZE Default: 10 The maximum limit for Twisted Reactor thread pool size. This is common multi-purpose thread pool used by various Scrapy components. Threaded DNS Resolver, BlockingFeedStorage use: CONCURRENT_REQUESTS = 100 Increase Twisted IO thread pool maximum size Currently Scrapy does DNS resolution in a blocking way with usage of thread pool. With higher concurrency levels the crawling processed faster speeding up establishing of connection and crawling overall. To increase maximum thread pool size use: REACTOR_THREADPOOL_MAXSIZE = 20 Setup your own DNS If you have multiple crawling

REACTOR_THREADPOOL_MAXSIZE Default: 10 The maximum limit for Twisted Reactor thread pool size. This is common multi- purpose thread pool used by various Scrapy components. Threaded DNS Resolver, BlockingFeedStorage use: CONCURRENT_REQUESTS = 100 Increase Twisted IO thread pool maximum size Currently Scrapy does DNS resolution in a blocking way with usage of thread pool. With higher concurrency levels the crawling processed faster speeding up establishing of connection and crawling overall. To increase maximum thread pool size use: REACTOR_THREADPOOL_MAXSIZE = 20 Setup your own DNS If you have multiple crawling