Changelog

All notable changes to this project will be documented in this file.

The format is based on Keep a Changelog, and this project adheres to Semantic Versioning.

1.1.1 — 2022-09-09

Changed

ExecuteJavascriptMiddleware now uses waitUntilNetworkIdle before returning the response body (#56)

1.1.0 — 2022-06-22

Added

Added a way to define custom item classes as well as item processors which only process certain types of items (#47)

Changed

Fixed deprecation warning in console commands for symfony/console:^6.1 (#44)

1.0.0 — 2022-04-19

Added

Added Roach::collectSpider method to start a spider run and return all scraped items.
Added array $context parameter to Roach::startSpider and Roach::collectSpider to pass arbitrary context data to a spider when starting a run.
Added roach:run <spider> command to start a spider through the CLI.
Added Roach::fake() method to test that a run for a given spider was started

Changed

Requests dropped by downloader middleware are no longer affected by requestDelay (fixes #27)
Move spatie/browsershot from a require to suggest as it's only necessary if the ExecuteJavascriptMiddleware is used. Remove ext-exif as a dependency for the same reason.

Removed

Removed default command from CLI. To start the REPL, you now need to explicitly invoke the roach:shell <url> command, instead.

0.2.0 - 2021-12-28

Added

Added ExecuteJavascriptMiddleware to retrieve a page’s body after executing Javascript (#7)

[0.1.0] - 2021-12-27

Added

Initial release