and have also been reported to contain leaks, so 2.6.28 or above is highly recommended) Optional: • pyopenssl (for HTTPS support, highly recommended) • simplejson (for (de)serializing JSON) 2.2.2 Step use the Control Panel): set PYTHONPATH=C:\path\to\scrapy-trunk 3. Make the scrapy-ctl.py script available On Unix-like systems, create a symbolic link to the file scrapy-trunk/bin/scrapy-ctl.py in a direc- hon/Books/>... ------------------------------------------------------------------------------ Available Scrapy variables: xxs: url: http://www.dmoz.org/Comp

Learn how to configure Scrapy and see all available settings. Signals See all available signals and how to work with them. Exceptions See all available exceptions and their meaning. Item Exporters reported to contain leaks, so 2.6.28 or above is highly recommended) Optional: pyopenssl [http://pyopenssl.sourceforge.net] (for HTTPS support, highly recommended) simplejson [http://undefined.org/python/#simplejson] nge_variable.mspx]): set PYTHONPATH=C:\path\to\scrapy-trunk 3. Make the scrapy-ctl.py script available On Unix-like systems, create a symbolic link to the file scrapy- trunk/bin/scrapy-ctl.py in a directory

and see all available commands. Requests and Responses Understand the classes used to represent HTTP requests and responses. Settings Learn how to configure Scrapy and see all available settings. Signals Signals See all available signals and how to work with them. Exceptions See all available exceptions and their meaning. Item Exporters Quickly export your scraped items to a file (XML, CSV, etc). libxml2 [http://xmlsoft.org] (if using libxml2 [http://xmlsoft.org], version 2.6.28 or above is highly recommended) simplejson [http://pypi.python.org/pypi/simplejson/] (not required if using Python 2

libxml2, version 2.6.28 or above is highly recommended) • simplejson (not required if using Python 2.6 or above) • pyopenssl (for HTTPS support. Optional, but highly recommended) 2.2.2 Install Python /Languages/Python/Books/ This is what the shell looks like: [ ... Scrapy log here ... ] [s] Available Scrapy objects: [s] 2010-08-19 21:45:59-0300 [default] INFO: Spider closed (finished) [s] hxs available at https://github.com/scrapy/dirbot Now doing a crawl on the dmoz.org domain yields DmozItem‘s:

libxml2, version 2.6.28 or above is highly recommended) • simplejson (not required if using Python 2.6 or above) • pyopenssl (for HTTPS support. Optional, but highly recommended) 8 Chapter 2. First steps /Languages/Python/Books/ This is what the shell looks like: [ ... Scrapy log here ... ] [s] Available Scrapy objects: [s] 2010-08-19 21:45:59-0300 [default] INFO: Spider closed (finished) [s] hxs available commands: Scrapy X.Y - no active project Usage: scrapy [options] [args] Available commands: crawl Start crawling a spider or URL fetch

and see all available commands. Requests and Responses Understand the classes used to represent HTTP requests and responses. Settings Learn how to configure Scrapy and see all available settings. Signals Signals See all available signals and how to work with them. Exceptions See all available exceptions and their meaning. Item Exporters Quickly export your scraped items to a file (XML, CSV, etc). libxml2 [http://xmlsoft.org] (if using libxml2 [http://xmlsoft.org], version 2.6.28 or above is highly recommended) simplejson [http://pypi.python.org/pypi/simplejson/] (not required if using Python 2

some binary packages that Scrapy depends on (like Twisted, lxml and pyOpenSSL) require a compiler available to install, and fail if you don’t have Visual Studio installed. You can find Windows installers character) will not work. This is what the shell looks like: [ ... Scrapy log here ... ] [s] Available Scrapy objects: [s] 2010-08-19 21:45:59-0300 [default] INFO: Spider closed (finished) [s] sel available at https://github.com/scrapy/dirbot Now doing a crawl on the dmoz.org domain yields DmozItem‘s:

some binary packages that Scrapy depends on (like Twisted, lxml and pyOpenSSL) require a compiler available to install, and fail if you don’t have Visual Studio installed. You can find Windows installers character) will not work. This is what the shell looks like: [ ... Scrapy log here ... ] [s] Available Scrapy objects: [s] 2010-08-19 21:45:59-0300 [default] INFO: Spider closed (finished) [s] sel available at https://github.com/scrapy/dirbot Now doing a crawl on the dmoz.org domain yields DmozItem‘s:

some binary packages that Scrapy depends on (like Twisted, lxml and pyOpenSSL) require a compiler available to install, and fail if you don’t have Visual Studio installed. You can find Windows installers /Languages/Python/Books/ This is what the shell looks like: [ ... Scrapy log here ... ] [s] Available Scrapy objects: [s] 2010-08-19 21:45:59-0300 [default] INFO: Spider closed (finished) [s] hxs available at https://github.com/scrapy/dirbot Now doing a crawl on the dmoz.org domain yields DmozItem‘s:

and see all available commands. Requests and Responses Understand the classes used to represent HTTP requests and responses. Settings Learn how to configure Scrapy and see all available settings. Signals Signals See all available signals and how to work with them. Exceptions See all available exceptions and their meaning. Item Exporters Quickly export your scraped items to a file (XML, CSV, etc). some binary packages that Scrapy depends on (like Twisted, lxml and pyOpenSSL) require a compiler available to install, and fail if you don’t have Visual Studio installed. You can find Windows installers