non-200 response will been regarded as fetch failed and will not pass to callback. use this decorator to override this feature.
def on_start(self): self.crawl('http://httpbin.org/status/404', self.callback) @catch_status_code_error def callback(self, response): ...
callbackwould not be executed as the request is failed (with status code 404). With the
callbackwould be executed even if the request failed.