| Producer | DigitalPebble Ltd  |
| Bot URL | https://github.com/DigitalPebble/storm-crawler |
| Status |  Active |
Known variants
|
| Useragentstring | StormCrawler Debug Crawl/1.0 (built with StormCrawler Opensearch Archetype 2.7-SNAPSHOT; http://digitalpebble.com/; crawler@digitalpebble.com)
|
| Category | -- Uncategorised -- |
| First seen | 2022-12-17 11:54:45 |
| Last seen | 2023-02-10 12:55:34 |
| IP addresses | 2 |
| Walk from |
|
|
| Useragentstring | StormCrawler+URLFrontier
|
| Category | -- Uncategorised -- |
| Respects robots.txt | No |
| First seen | 2022-05-23 16:45:37 |
| Last seen | 2022-07-20 12:05:11 |
| IP addresses | 1 |
| Walk from |
|
|
| Useragentstring | StormCrawler/2.1 (benchmark crawl to test URLFrontier on a large scale; http://digitalpebble.com/; julien@digitalpebble.com)
|
| Category | -- Uncategorised -- |
| First seen | 2021-07-06 13:54:58 |
| Last seen | 2021-07-06 13:54:58 |
| IP addresses | 1 |
| Walk from |
|
|
| Useragentstring | DigitalPebble Crawler/1.5-SNAPSHOT (StormCrawler Benchmark; http://digitalpebble.com/; crawler@digitalpebble.com)
|
| Category | -- Uncategorised -- |
| First seen | 2017-05-28 00:10:53 |
| Last seen | 2017-05-28 00:10:53 |
| IP addresses | 0 |
| Walk from | |
|
| Useragentstring | DigitalPebble Crawler/1.4-SNAPSHOT (StormCrawler Benchmark; http://digitalpebble.com/; crawler@digitalpebble.com)
|
| Category | -- Uncategorised -- |
| First seen | 2017-05-14 20:30:04 |
| Last seen | 2017-05-14 20:30:04 |
| IP addresses | 0 |
| Walk from | |
|
| Useragentstring | Blackboard Safeassign/0.1 (a Storm-based Blackboard Safeassign web-crawler; https://github.com/DigitalPebble/storm-crawler; stormcrawler@digitalpebble.com)
|
| Category | -- Uncategorised -- |
| First seen | 2016-10-27 11:09:27 |
| Last seen | 2016-10-27 11:09:27 |
| IP addresses | 1 |
| Walk from |
|
|
| Useragentstring | StormCrawler/1.0 (a Storm-based crawler; https://github.com/DigitalPebble/storm-crawler; stormcrawler@digitalpebble.com)
|
| Category | -- Uncategorised -- |
| First seen | 2016-08-12 14:13:02 |
| Last seen | 2016-08-12 14:13:03 |
| IP addresses | 1 |
| Walk from |
|
|
| Useragentstring | anonymous coward/1.0 (a Storm-based crawler; https://github.com/DigitalPebble/storm-crawler; someone@company.com)
|
| Category | -- Uncategorised -- |
| First seen | 2015-02-01 19:28:57 |
| Last seen | 2015-07-24 21:47:13 |
| IP addresses | 3 |
| Walk from |
|
DigitalPebble bot
