shell fetch view version Project-only commands: crawl check list edit parse bench startproject Syntax: scrapy startproject [project_dir] Requires project: no Creates a new Scrapy project project_dir will be the same as myproject. Usage example: $ scrapy startproject myproject genspider Syntax: scrapy genspider [-t template] Requires project: no Create a new spider in the command. crawl Syntax: scrapy crawl Requires project: yes Start crawling using a spider. Usage examples: $ scrapy crawl myspider [ ... myspider starts crawling ... ] check Syntax: scrapy check

version Project-only commands: • crawl • check • list • edit • parse • bench startproject • Syntax: scrapy startproject [project_dir] • Requires project: no Creates a new Scrapy project will be the same as project_name. Usage example: $ scrapy startproject myproject genspider • Syntax: scrapy genspider [-t template] • Requires project: no Create a new spider in the crawl • Syntax: scrapy crawl • Requires project: yes Start crawling using a spider. Usage examples: $ scrapy crawl myspider [ ... myspider starts crawling ... ] check • Syntax: scrapy check

version Project-only commands: • crawl • check • list • edit • parse • bench startproject • Syntax: scrapy startproject [project_dir] • Requires project: no Creates a new Scrapy project startproject myproject 28 Chapter 3. Basic concepts Scrapy Documentation, Release 2.3.0 genspider • Syntax: scrapy genspider [-t template] • Requires project: no Create a new spider in the crawl • Syntax: scrapy crawl • Requires project: yes Start crawling using a spider. Usage examples: $ scrapy crawl myspider [ ... myspider starts crawling ... ] check • Syntax: scrapy check

access the values of their fields (attributes of the class we defined earlier) using the standard dict syntax like: >>> item = DmozItem() >>> item['title'] = 'Example title' >>> item['title'] 'Example title' view version Project-only commands: crawl list edit parse genspider server deploy startproject Syntax: scrapy startproject Requires project: no Creates a new Scrapy project named project_name project_name, under the project_name directory. Usage example: $ scrapy startproject myproject genspider Syntax: scrapy genspider [-t template] Requires project: yes Create a new spider in the

access the values of their fields (attributes of the class we defined earlier) using the standard dict syntax like: >>> item = DmozItem() >>> item['title'] = 'Example title' >>> item['title'] 'Example title' Project-only commands: • crawl • list • edit • parse • genspider • server • deploy startproject • Syntax: scrapy startproject • Requires project: no Creates a new Scrapy project named project_name under the project_name directory. Usage example: $ scrapy startproject myproject genspider • Syntax: scrapy genspider [-t template] • Requires project: yes Create a new spider in

version Project-only commands: • crawl • check • list • edit • parse • bench startproject • Syntax: scrapy startproject [project_dir] • Requires project: no Creates a new Scrapy project startproject myproject 26 Chapter 3. Basic concepts Scrapy Documentation, Release 2.2.1 genspider • Syntax: scrapy genspider [-t template] • Requires project: no Create a new spider in the crawl • Syntax: scrapy crawl • Requires project: yes Start crawling using a spider. Usage examples: $ scrapy crawl myspider [ ... myspider starts crawling ... ] check • Syntax: scrapy check

version Project-only commands: • crawl • check • list • edit • parse • bench startproject • Syntax: scrapy startproject [project_dir] • Requires project: no Creates a new Scrapy project startproject myproject 28 Chapter 3. Basic concepts Scrapy Documentation, Release 2.6.3 genspider • Syntax: scrapy genspider [-t template] • Requires project: no New in version 2 You can just create the spider source code files yourself, instead of using this command. crawl • Syntax: scrapy crawl • Requires project: yes Start crawling using a spider. Usage examples:

version Project-only commands: • crawl • check • list • edit • parse • bench startproject • Syntax: scrapy startproject [project_dir] • Requires project: no Creates a new Scrapy project startproject myproject 28 Chapter 3. Basic concepts Scrapy Documentation, Release 2.5.1 genspider • Syntax: scrapy genspider [-t template] • Requires project: no Create a new spider in the crawl • Syntax: scrapy crawl • Requires project: yes Start crawling using a spider. Usage examples: $ scrapy crawl myspider [ ... myspider starts crawling ... ] check • Syntax: scrapy check

access the values of their fields (attributes of the class we defined earlier) using the standard dict syntax like: >>> item = DmozItem() >>> item['title'] = 'Example title' >>> item['title'] 'Example title' Project-only commands: • crawl • list • parse • genspider • server • deploy startproject • Syntax: scrapy startproject • Requires project: no Creates a new Scrapy project named project_name under the project_name directory. Usage example: $ scrapy startproject myproject genspider • Syntax: scrapy genspider [-t template] • Requires project: yes 3.1. Command line tool

access the values of their fields (attributes of the class we defined earlier) using the standard dict syntax like: >>> item = DmozItem() >>> item['title'] = 'Example title' >>> item['title'] 'Example title' fetch view version Project-only commands: crawl list parse genspider server deploy startproject Syntax: scrapy startproject Requires project: no Creates a new Scrapy project named project_name project_name, under the project_name directory. Usage example: $ scrapy startproject myproject genspider Syntax: scrapy genspider [-t template] Requires project: yes Create a new spider in the