Resources > UA list: crawlers

Udger database includes detailed information about every single user agent and operating system

Select list type: Browsers, email clients ... | Operating systems | Devices| Crawlers (Robots)

Category: All - Search engines - Marketing - Screenshots - Monitors - Feeds - Link checkers - Scrapers - Tools - Speed test - Vulnerability/Virus

Crawlers (Robots)Last seenbot URLProducer URL
logo Monibot2020-01-20 19:08:55
logo TMM Crawler2020-01-20 13:39:46
logo Fake bot2020-01-20 07:08:35
logo Scrapy2020-01-19 11:43:59UPUP
logo Diffbot2020-01-18 23:28:19UP
logo DF Bot2020-01-18 18:53:29
logo bot2020-01-18 13:45:25
logo The Knowledge AI2020-01-18 11:00:01
logo Splash2020-01-18 09:07:54UPUP
logo MetaInspector2020-01-10 10:04:55UPUP
logo DownloaderChrome2020-01-08 00:41:20
logo YFF352020-01-02 15:46:17
logo centurybot92020-01-01 10:50:22
logo nbertaupete952019-12-18 09:28:47
logo WebBot2019-11-25 14:49:31
logo WebCrawler2019-11-25 14:49:30
logo Fuze_Bot2019-11-12 16:04:52
logo AdsrvrBot2019-10-28 08:30:19
logo CLcrawler2019-10-12 04:10:48
logo PleskBot2019-08-29 09:05:20
logo MauiBot2019-08-24 09:18:48
logo arachnode.net2019-08-11 09:00:54UPUP
logo DaveCrawler2019-07-07 11:25:07
logo aylienbot2019-06-22 07:50:43DownUP
logo Frontera2019-05-23 13:42:37UPUP
logo crawler2019-05-07 17:52:28UP
logo SemanticJuice crawler2019-05-02 13:56:38UPUP

Maybe inactive robots (One year we have not heard about them)
logo Mixnode2018-08-20 01:06:54UP
logo 0082018-08-14 06:14:32DownUP
logo Contacts Crawler2018-06-28 00:35:28UP
logo Quick-Crawler2018-06-08 07:47:17UP
logo voltron2018-04-01 12:07:39DownUP
logo MailDBBot2017-04-06 20:08:19DownUP
logo scrapy-redis2017-03-06 12:58:15UP
logo Profound scrapyproject2017-01-05 14:39:19UPUP
logo Crawlera2016-02-08 16:08:25UP
logo import.io2016-02-03 18:34:09UP

Historical (Old robots, Inactive cravlers)
logo Saleslift2017-12-23 05:57:41Down
logo Spinn3r2015-12-16 04:26:23DownDown
logo KimonoLabs2015-06-22 04:28:40UP
Among our clients
View more...
 Dailymotion SA  
 Akamai Technologies, Inc.  
 PayPal Holdings, Inc.