It uses a dummy parser that simply returns the HTTP response body as-is. Use this only if you know what you are
doing. In most cases, using an HTML parser would be more beneficial. For such scenarios, consider using
BeautifulSoupCrawler, ParselCrawler, or writing your own subclass of AbstractHttpCrawler.
Usage
from crawlee.crawlers import HttpCrawler, HttpCrawlingContext
crawler = HttpCrawler()
# Define the default request handler, which will be called for every request.
Specific version of generic
AbstractHttpCrawler.It uses a dummy parser that simply returns the HTTP response body as-is. Use this only if you know what you are doing. In most cases, using an HTML parser would be more beneficial. For such scenarios, consider using
BeautifulSoupCrawler,ParselCrawler, or writing your own subclass ofAbstractHttpCrawler.Usage