Resources > UA list: crawlers

Udger database includes detailed information about every single user agent and operating system

Select list type: Browsers, email clients ... | Operating systems | Devices| Crawlers (Robots)


Category: All - Search engines - Marketing - Screenshots - Monitors - Feeds - Link checkers - Scrapers - Tools - Speed test - Vulnerability/Virus


Crawlers (Robots)Last seenbot URLProducer URL
logo cron-job.org2020-07-08 13:30:40UP
logo Domains Project bot2020-07-08 13:02:04UPUP
logo Ghost Inspector2020-07-08 09:47:33UPUP
logo monitorbacklinks.com bot2020-07-08 08:15:45UPUP
logo Chrome-Lighthouse2020-07-08 07:09:11UPUP
logo s4a2020-07-08 03:51:27UP
logo WhoAPI2020-07-08 03:51:27UP
logo CronDroid2020-07-08 03:51:26UP
logo Googlebot2020-07-08 03:51:18UPUP
logo BuiltWith crawler2020-07-08 03:46:10UPUP
logo Naver Scrap2020-07-08 03:45:02UPUP
logo Google Read Aloud2020-07-08 03:43:11UPUP
logo archive.org_bot2020-07-08 02:39:39UPUP
logo Cronless2020-07-07 20:19:22UP
logo bitlybot2020-07-07 14:38:28UP
logo TextRazor crawler2020-07-07 08:00:07UP
logo Photon2020-07-07 04:28:59UPUP
logo masscan2020-07-07 02:41:28UP
logo w3dt.net crawler2020-07-06 14:22:42UPUP
logo Yandex.Webmaster2020-07-06 14:21:17UPUP
logo WebSniffer2020-07-06 14:16:54UPUP
logo OpenGraphCheck2020-07-06 14:15:44UPUP
logo alyze.info bot2020-07-06 14:14:31UPUP
logo REDbot2020-07-06 14:10:26UPUP
logo GAChecker2020-07-06 14:08:49UPUP
logo Prerender.cloud2020-07-06 14:08:22UP
logo DareBoost2020-07-06 14:07:53UP
logo sniptracker2020-07-06 14:07:10UPUP
logo OrgProbe2020-07-06 14:05:11UPUP
logo RobotsChecker2020-07-06 14:05:10UPUP
logo Prerender2020-07-06 14:04:51UP
logo Web Manifest Validator2020-07-06 14:04:44UPUP
logo Mergadobot2020-07-06 13:58:33UP
logo DuckDuckBot2020-07-06 06:23:58UPUP
logo letsencrypt bot2020-07-06 05:50:22UP
logo Taringa2020-07-05 18:51:20UP
logo CloudFlare crawler2020-07-05 06:21:18UPUP
logo Deskyobot2020-07-04 19:54:46UPUP
logo Google-Structured-Data-Testing-Tool 2020-07-03 15:32:14UPUP
logo Wappalyzer2020-07-01 06:54:15UPUP
logo Hexometer.com crawler2020-06-27 04:46:07UPUP
logo updown.io bot2020-06-26 06:57:56UPUP
logo PulnoBot2020-06-23 12:25:26UPUP
logo Google Apps Script2020-06-23 12:01:22UPUP
logo PowerMapper Crawler2020-06-23 07:38:26UP
logo WhatCMS.org bot2020-06-20 16:47:34UP
logo Check-Host2020-06-18 14:30:26UP
logo Browsershots2020-06-18 14:27:04UPUP
logo CastFeedValidator2020-06-18 14:23:35UPUP
logo ImageFetcher2020-06-18 14:22:55UPUP
logo WURFL ImageEngine2020-06-18 14:22:22UP
logo Filestack2020-06-18 14:21:47UP
logo Google AMP crawler2020-06-18 14:19:56UPUP
logo URLitor.com2020-06-18 14:19:22UPUP
logo Webcron.org crawler2020-06-18 13:31:38UP
logo NinjaBot2020-06-18 07:52:56UP
logo YandexBot2020-06-13 06:28:47UPUP
logo github-camo2020-06-11 23:03:05UPUP
logo DomainStatsBot2020-06-11 17:01:08UPUP
logo GoogleDocs2020-06-05 01:26:41UPUP
logo Mediumbot2020-06-04 21:41:36UP
logo XML Schema Validator2020-05-26 13:19:09UPUP
logo kazbtbot2020-05-26 13:16:22UPUP
logo PrintFriendly.com2020-05-26 13:06:42UP
logo XML Sitemaps Generator2020-05-25 04:33:18UP
logo SiteCheckerBot2020-05-17 21:12:12UPUP
logo StatOnlineRuBot2020-05-14 03:06:25UPUP
logo SimplyFast.info2020-05-13 19:26:53UPUP
logo 2GDPR bot2020-05-13 19:25:53UP
logo SSL Labs2020-05-13 17:34:34UPUP
logo Seobility2020-05-06 15:04:44UPUP
logo Bad-Neighborhood2020-05-03 11:11:12UP
logo BoardReader crawler2020-05-02 02:59:25UP
logo Closure Compiler Service2020-05-01 10:21:39UPUP
logo historious2020-04-29 10:30:57UPUP
logo SnapSearch2020-04-28 08:12:09UPUP
logo adstxtlab.com Crawler2020-04-25 05:09:00UP
logo EasyBib crawler2020-04-19 18:14:54UPUP
logo SetCronJob2020-04-09 18:48:04UPUP
logo Feedly2020-04-08 15:07:07UPUP
logo Protopage2020-04-07 08:28:44UP
logo link_thumbnailer2020-04-07 05:49:16UPUP
logo SpyOnWeb2020-03-31 16:12:54UP
logo Netpeak bot2020-03-31 10:20:12UPUP
logo ContentKing2020-03-30 03:16:23UPUP
logo Naver scrapbook2020-03-26 07:30:14UPUP
logo AdAuth2020-03-25 07:54:17UP
logo Robots_Tester2020-03-25 07:51:30UP
logo PrivacyScore.org crawler2020-03-24 09:22:31UPUP
logo Tools4noobs.com Spider2020-03-24 09:20:56UP
logo RSiteAuditor2020-03-24 09:10:20UPUP
logo TestURI crawler2020-03-23 11:35:44UP
logo hCardValidator2020-03-23 11:33:09UPUP
logo GetLinkInfo.com2020-03-23 11:23:59UP
logo Structured Data Markup Helper2020-03-21 13:24:03UPUP
logo GoogleSites2020-03-21 09:55:14UPUP
logo WGETbot2020-03-20 09:57:00UPUP
logo Unshorten.It2020-03-18 09:01:09UPUP
logo Google Cloud AI2020-03-17 10:31:25UPUP
logo LinqTo.me2020-03-16 07:51:41UP
logo linkgraph.io crawler2020-03-14 07:44:31UP
logo KeyCDN bot2020-03-11 08:39:39UPUP
logo Seo Servis2020-03-08 14:58:29UPUP
logo CrawlForMe2020-03-01 12:24:25UPUP
logo Pearltrees2020-03-01 10:30:50UPUP
logo Cronjob.de bot2020-03-01 10:30:50UPUP
logo SpotiboBot2020-02-29 17:40:21UP
logo Online Domain Tools2020-02-23 12:05:19UP
logo wmtips.com2020-02-23 12:03:32UPUP
logo XmlSitemapGenerator2020-02-23 11:58:13UP
logo Stephan Kopp bot2020-02-23 11:53:25UP
logo DNS-Tools Header-Analyzer2020-02-17 06:38:24UPUP
logo BrowserSpyBot2020-02-17 06:37:10UP
logo EasyCron2020-02-13 23:00:06UP
logo ROR Sitemap Generator2020-02-07 09:12:18UP
logo ipv6-test.com validator2020-02-07 07:19:33UP
logo Grabify.link2020-02-06 08:06:55UPUP
logo MB-SiteCrawler2020-02-06 07:57:15UPUP
logo Jeffrey Exif Viewer2020-02-06 07:51:53UPUP
logo iodc crawler2020-02-03 15:50:51UP
logo TomBot2020-01-30 06:18:57UPUP
logo NameFo2020-01-26 14:52:33UP
logo WormlyBot2019-10-23 09:31:12UP
logo YP crawler2019-10-22 10:36:56UP
logo Google AppsViewer2019-10-13 21:28:27UPUP
logo faviconkit2019-10-13 18:18:49UPUP
logo Riddlerbot2019-09-24 06:43:05UPUP
logo bingbot2019-09-13 19:23:03UPUP
logo DIGMATO.com crawler2019-09-09 13:33:29UPUP
logo ddline.cn crawler2019-08-11 09:43:19UP
logo GoSquared2019-08-11 08:52:52UP
logo ssl-tools.net2019-08-10 16:10:03UPUP

Maybe inactive robots (One year we have not heard about them)
logo hosterstats com2019-06-25 22:01:40UPUP
logo Genieo Web filter2019-05-20 16:42:02UPUP
logo SiteExplorer2019-05-18 12:31:46UPUP
logo ExtLinksBot2019-05-17 21:02:18UPUP
logo probethenet.com scanner2019-05-16 01:10:36UP
logo WWRanks.com crawler2019-05-09 03:50:36UP
logo RankingBot2019-04-29 03:03:03UPUP
logo Favicon2019-04-25 06:31:31UPDown
logo htmlyse crawler2019-04-08 10:33:45UP
logo Everyonedomainsbot2019-03-28 20:43:09UP
logo GoogleBot2019-03-27 09:35:44UPUP
logo northcutt.com crawler2019-03-26 11:34:16UPUP
logo HeartRailsBot2019-03-08 11:09:06UP
logo infohound.net crawler2019-01-04 08:04:06UPUP
logo cg-eye2018-12-27 10:08:49UP
logo Check-Domains.com bot2018-12-23 07:43:12UP
logo LongURL2018-12-08 07:23:09UP
logo Page Valet2018-12-05 10:10:34UP
logo FoundSeoTool2018-11-24 05:46:52UP
logo RankFlex.com Webspider2018-09-07 18:36:06UP
logo iThemes crawler2018-08-24 09:12:42UP
logo topster_de-crawler2018-06-13 14:26:48UP
logo webhint crawler2018-06-07 18:37:27UP
logo censys.io crawler2018-04-02 08:28:25UP
logo SurveyBot2018-03-24 01:23:47UP
logo TcpTools Bot2018-03-16 07:25:04UPUP
logo Office 365 viewer2017-10-13 08:44:31UPUP
logo hosterstats newgtld2017-10-07 10:02:45UPUP
logo netk6 crawler2017-08-09 06:47:30DownUP
logo Monsidobot2017-07-06 09:26:22DownUP
logo cuwhois2017-07-01 20:23:22UP
logo CSIRT.CZ crawler2017-06-29 08:38:50UPUP
logo 0xFF Domain Crawler2017-06-26 02:42:20UP
logo pdffillerbot2017-06-19 19:25:51UP
logo Mozilla Metadata Service2017-06-09 15:11:27UPUP
logo WhatsMyIP.org2017-05-29 17:25:35UPUP
logo GIDBot2017-05-29 17:22:54UP
logo Page Analyzer2017-05-29 16:12:36UP
logo DIY-SEOBot2017-05-24 12:21:39DownUP
logo Detectify bot2017-05-16 09:17:01UP
logo Whoiswebsitebot2017-04-08 05:53:34UP
logo MS Cognitive Services2017-03-28 08:17:07UP
logo metacdn2017-03-03 14:05:20UP
logo SBA Research bot2017-01-19 04:47:17UPUP
logo NeutrinoAPI2017-01-08 07:25:39UP
logo Google-HotelAdsVerifier2017-01-01 16:26:28UPUP
logo HootSuite crawler2016-09-17 04:15:21UP
logo Audisto Crawler2016-08-24 16:25:42UPUP
logo Whibse2016-08-24 15:21:05UP
logo WordPress.com bots2016-07-19 03:47:24UP
logo WP Engine2016-04-16 15:29:53UP
logo ResponseCodeTest2016-03-09 07:59:10UP
logo Siteimprove.com bot2016-01-11 03:42:51UP

Historical (Old robots, Inactive cravlers)
logo showsite crawler2020-05-28 04:07:22Down
logo Geek-Tools2020-01-15 03:26:17Down
logo Is it up2019-10-13 18:18:49DownDown
logo WebCookies2019-09-09 13:56:24DownDown
logo Adstxtaggregator.com2019-05-02 08:29:11UPUP
logo wf_crawler2018-12-05 10:10:34Down
logo richardstoolbox crawler2018-12-01 08:24:28DownDown
logo SiteXY Webmaster tools2018-11-13 08:13:39Down
logo SputnikBot2018-07-23 10:54:10UPUP
logo SEO Browser2018-07-14 08:45:19DownUP
logo seocharger-robot2018-04-19 08:53:36DownDown
logo wscheck.com2018-02-09 12:39:38Down
logo Web-sniffer.me/1.0.02018-02-09 11:05:43DownDown
logo HTTP-Header-Abfrage2017-10-23 06:44:30Down
logo Web-sniffer2017-08-07 14:14:12DownUP
logo ganarvisitas2017-07-01 19:50:33Down
logo Scrubby2017-06-19 14:11:21UPUP
logo ghostery bot2017-06-18 11:21:10UP
logo GC3pro2017-05-29 16:02:19Down
logo Untiny2017-03-03 12:50:50DownUP
logo GetProxi.es-bot2016-07-22 07:25:58DownDown
logo MetaCommentBot2016-06-09 11:12:02Down
logo seo-nastroj.cz2015-11-25 11:40:46Down
logo SEOCentro bot2015-11-03 10:25:02UP
logo make.myUrl2015-10-07 13:01:33Down
logo Whoismindbot2015-09-01 15:21:11UPUP
logo Yahoo!2010-01-05 19:09:55UPUP
Among our clients
View more...
 SAP  
 MailChimp  
 Dailymotion SA  
 Akamai Technologies, Inc.  
 Oracle  
 PayPal Holdings, Inc.