Resources > Crawlers list > DigitalPebble bot

logo DigitalPebble bot

ID: 15650
ProducerDigitalPebble Ltd External link
Bot URLhttps://github.com/DigitalPebble/storm-crawler External link
StatusActive Active

Known variants

DigitalPebble Crawler

UseragentstringStormCrawler+URLFrontier
Category-- Uncategorised --
Respects robots.txtNo
First seen2022-05-23 16:45:37
Last seen2022-07-06 14:52:34
IP addresses1
Walk from
193.191.148.194wall.nat.ilabt.imec.be BE
 

DigitalPebble Crawler

UseragentstringStormCrawler/2.1 (benchmark crawl to test URLFrontier on a large scale; http://digitalpebble.com/; julien@digitalpebble.com)
Category-- Uncategorised --
First seen2021-07-06 13:54:58
Last seen2021-07-06 13:54:58
IP addresses1
Walk from
193.191.148.194wall.nat.iminds.be BE
 

DigitalPebble Crawler

UseragentstringDigitalPebble Crawler/1.5-SNAPSHOT (StormCrawler Benchmark; http://digitalpebble.com/; crawler@digitalpebble.com)
Category-- Uncategorised --
First seen2017-05-28 00:10:53
Last seen2017-05-28 00:10:53
IP addresses0
Walk from 

DigitalPebble Crawler

UseragentstringDigitalPebble Crawler/1.4-SNAPSHOT (StormCrawler Benchmark; http://digitalpebble.com/; crawler@digitalpebble.com)
Category-- Uncategorised --
First seen2017-05-14 20:30:04
Last seen2017-05-14 20:30:04
IP addresses0
Walk from 

DigitalPebble Crawler

UseragentstringBlackboard Safeassign/0.1 (a Storm-based Blackboard Safeassign web-crawler; https://github.com/DigitalPebble/storm-crawler; stormcrawler@digitalpebble.com)
Category-- Uncategorised --
First seen2016-10-27 11:09:27
Last seen2016-10-27 11:09:27
IP addresses1
Walk from
69.196.244.10bb-va3-tsg-nat-01.blackboard.comUS
 

DigitalPebble Crawler

UseragentstringStormCrawler/1.0 (a Storm-based crawler; https://github.com/DigitalPebble/storm-crawler; stormcrawler@digitalpebble.com)
Category-- Uncategorised --
First seen2016-08-12 14:13:02
Last seen2016-08-12 14:13:03
IP addresses1
Walk from
69.196.244.10bb-va3-tsg-nat-01.blackboard.comUS
 

DigitalPebble Crawler

Useragentstringanonymous coward/1.0 (a Storm-based crawler; https://github.com/DigitalPebble/storm-crawler; someone@company.com)
Category-- Uncategorised --
First seen2015-02-01 19:28:57
Last seen2015-07-24 21:47:13
IP addresses3
Walk from
52.0.19.0ec2-52-0-19-0.compute-1.amazonaws.comUS
5.9.151.82?DE
5.9.78.211?DE
 
Among our clients
View more...
 salesforce.com, inc.  
 MailChimp  
 Dailymotion SA  
 Akamai Technologies, Inc.  
 Oracle  
 PayPal Holdings, Inc.