Resources > Crawlers list > Scrubby

logo Scrubby

ID: 14157
ProducerScrub The Web External link
Bot URLhttp://www.scrubtheweb.com/help/technology.html External link
StatusInactive Inactive

Known variants

Scrubby/3.2

UseragentstringMozilla/5.0 (compatible; Scrubby/3.2; +http://www.scrubtheweb.com/tools/metatag-analyzer.html)
CategoryTool
First seen2017-02-28 09:33:33
Last seen2017-06-19 14:11:21
IP addresses1
Walk from
66.93.156.36scrubby.scrubtheweb.comUS
 

Scrubby/3.2

UseragentstringMozilla/5.0 (compatible; Scrubby/3.2; +http://seotools.scrubtheweb.com/webpage-analyzer.html)
CategoryTool
Respects robots.txtNo
First seen2014-07-28 07:30:00
Last seen2016-05-12 09:39:45
IP addresses4
Walk from
66.93.156.43spider-5.scrubtheweb.comUS
66.93.156.52ip-93.156.52.scrubtheweb.comUS
66.93.156.41spider-3.scrubtheweb.comUS
 

Scrubby/2.2

UseragentstringScrubby/2.2 (http://www.scrubtheweb.com/)
Category-- Uncategorised --
First seen2016-02-29 06:02:00
Last seen2016-02-24 13:36:26
IP addresses1
Walk from
188.165.238.96ns3280143.ip-188-165-238.euFR
 

Scrubby/2.2

UseragentstringMozilla/5.0 (compatible; Scrubby/2.2; http://www.scrubtheweb.com/)
Category-- Uncategorised --
First seen2015-02-19 19:44:54
Last seen2016-02-19 19:44:54
IP addresses0
Walk from 

Scrubby/2.1

UseragentstringScrubby/2.1 (http://www.scrubtheweb.com/)
Category-- Uncategorised --
First seen2014-12-27 12:03:44
Last seen2015-12-27 12:03:44
IP addresses0
Walk from 

Scrubby/3.0

UseragentstringScrubby/3.0 (+http://www.scrubtheweb.com/help/technology.html)
Category-- Uncategorised --
First seen2014-08-08 09:30:00
Last seen2014-09-07 03:34:50
IP addresses1
Walk from
66.93.156.51ip-93.156.51.scrubtheweb.comUS
 

Scrubby/3.1

UseragentstringMozilla/5.0 (compatible; Scrubby/3.1; +http://www.scrubtheweb.com/help/technology.html)
Category-- Uncategorised --
First seen2014-07-28 07:25:00
Last seen2014-07-25 20:33:38
IP addresses1
Walk from
66.93.156.41spider-3.scrubtheweb.comUS
 
Among our clients
View more...
 salesforce.com, inc.  
 MailChimp  
 Dailymotion SA  
 Akamai Technologies, Inc.  
 Oracle  
 PayPal Holdings, Inc.