I am running fbcrawl from VMWare Windows 10 image. I have python 3.7, and I am seeing following error when I run the command:
2020-03-11 19:31:57 [fb] INFO: Going through the "save-device" checkpoint
2020-03-11 19:32:03 [fb] INFO: Scraping facebook page https://mbasic.facebook.com/cnn
2020-03-11 19:32:07 [scrapy.core.scraper] ERROR: Spider error processing <GET https://mbasic.facebook.com/cnn> (referer: https://mbasic.facebook.com/?_rdr)
Traceback (most recent call last):
File "c:\users\user\appdata\local\programs\python\python37\lib\site-packages\twisted\internet\defer.py", line 1418, in _inlineCallbacks
result = g.send(result)
File "c:\users\user\appdata\local\programs\python\python37\lib\site-packages\scrapy\core\downloader\middleware.py", line 42, in process_request
defer.returnValue((yield download_func(request=request, spider=spider)))
File "c:\users\user\appdata\local\programs\python\python37\lib\site-packages\twisted\internet\defer.py", line 1362, in returnValue
raise _DefGen_Return(val)
twisted.internet.defer._DefGen_Return: <200 https://mbasic.facebook.com/cnn>
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "c:\users\user\appdata\local\programs\python\python37\lib\site-packages\scrapy\utils\defer.py", line 55, in mustbe_deferred
result = f(*args, **kw)
File "c:\users\user\appdata\local\programs\python\python37\lib\site-packages\scrapy\core\spidermw.py", line 60, in process_spider_input
return scrape_func(response, request, spider)
File "c:\users\user\appdata\local\programs\python\python37\lib\site-packages\scrapy\core\scraper.py", line 148, in call_spider
warn_on_generator_with_return_value(spider, callback)
File "c:\users\user\appdata\local\programs\python\python37\lib\site-packages\scrapy\utils\misc.py", line 202, in warn_on_generator_with_return_value
if is_generator_with_return_value(callable):
File "c:\users\user\appdata\local\programs\python\python37\lib\site-packages\scrapy\utils\misc.py", line 187, in is_generator_with_return_value
tree = ast.parse(dedent(inspect.getsource(callable)))
File "c:\users\user\appdata\local\programs\python\python37\lib\ast.py", line 35, in parse
return compile(source, filename, mode, PyCF_ONLY_AST)
File "", line 1
def parse_page(self, response):
^
IndentationError: unexpected indent
2020-03-11 19:32:07 [scrapy.core.engine] INFO: Closing spider (finished)
I am running fbcrawl from VMWare Windows 10 image. I have python 3.7, and I am seeing following error when I run the command:
2020-03-11 19:31:57 [fb] INFO: Going through the "save-device" checkpoint
2020-03-11 19:32:03 [fb] INFO: Scraping facebook page https://mbasic.facebook.com/cnn
2020-03-11 19:32:07 [scrapy.core.scraper] ERROR: Spider error processing <GET https://mbasic.facebook.com/cnn> (referer: https://mbasic.facebook.com/?_rdr)
Traceback (most recent call last):
File "c:\users\user\appdata\local\programs\python\python37\lib\site-packages\twisted\internet\defer.py", line 1418, in _inlineCallbacks
result = g.send(result)
File "c:\users\user\appdata\local\programs\python\python37\lib\site-packages\scrapy\core\downloader\middleware.py", line 42, in process_request
defer.returnValue((yield download_func(request=request, spider=spider)))
File "c:\users\user\appdata\local\programs\python\python37\lib\site-packages\twisted\internet\defer.py", line 1362, in returnValue
raise _DefGen_Return(val)
twisted.internet.defer._DefGen_Return: <200 https://mbasic.facebook.com/cnn>
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "c:\users\user\appdata\local\programs\python\python37\lib\site-packages\scrapy\utils\defer.py", line 55, in mustbe_deferred
result = f(*args, **kw)
File "c:\users\user\appdata\local\programs\python\python37\lib\site-packages\scrapy\core\spidermw.py", line 60, in process_spider_input
return scrape_func(response, request, spider)
File "c:\users\user\appdata\local\programs\python\python37\lib\site-packages\scrapy\core\scraper.py", line 148, in call_spider
warn_on_generator_with_return_value(spider, callback)
File "c:\users\user\appdata\local\programs\python\python37\lib\site-packages\scrapy\utils\misc.py", line 202, in warn_on_generator_with_return_value
if is_generator_with_return_value(callable):
File "c:\users\user\appdata\local\programs\python\python37\lib\site-packages\scrapy\utils\misc.py", line 187, in is_generator_with_return_value
tree = ast.parse(dedent(inspect.getsource(callable)))
File "c:\users\user\appdata\local\programs\python\python37\lib\ast.py", line 35, in parse
return compile(source, filename, mode, PyCF_ONLY_AST)
File "", line 1
def parse_page(self, response):
^
IndentationError: unexpected indent
2020-03-11 19:32:07 [scrapy.core.engine] INFO: Closing spider (finished)