Python web scraping framework : Scrapy

1. Abstract

2. Introduction

The real project usually start from obtaining data. No matter text mining, machine learning and data mining, all need data. In addition to the professional data purchased or downloaded through some channels, we often need to climb the data by ourselves. At this time, the crawler is particularly important. Fortunately, python provides a very good web crawler tool frameworks -- Scrapy, which can not only crawl the data, but also get and clean the data. Scrapy is an application framework written by Python for crawling website data and extracting structural data. Use twisted efficient asynchronous network framework to handle network communication.

3. Findings

4. Conclusions