Resources > UA list: crawlers

Udger database includes detailed information about every single user agent and operating system

Select list type: Browsers, email clients ... | Operating systems | Devices| Crawlers (Robots)


Category: All - Search engines - Marketing - Screenshots - Monitors - Feeds - Link checkers - Scrapers - Tools - Speed test - Vulnerability/Virus


Crawlers (Robots)Last seenbot URLProducer URL
logo ClaudeBot2024-03-19 11:40:29
logo Monibot2024-03-19 11:34:34
logo Fake bot2024-03-19 11:22:28
logo Headless Chrome2024-03-19 10:51:02
logo nbertaupete952024-03-19 09:50:40
logo Scrapy2024-03-19 08:46:40UPUP
logo keys-so-bot2024-03-19 08:17:42
logo Mozilla2024-03-19 08:14:46
logo Magellan2024-03-18 15:59:27
logo HawaiiBot2024-03-18 15:52:52
logo ALittle Client2024-03-18 15:10:34
logo Hello World2024-03-18 12:10:02
logo wpbot2024-03-17 11:16:16
logo Headless Edge2024-03-17 01:39:01
logo DF Bot2024-03-16 00:06:21
logo Test Certificate Info2024-03-15 06:33:36
logo magicsearchdev2024-03-14 14:44:23
logo Orbbot2024-03-13 22:25:53
logo Nicecrawler2024-03-12 02:14:10UP
logo DnBCrawler2024-03-11 18:26:21
logo VsuSearchSpider2024-03-10 15:03:55
logo SSSSBot2024-03-03 09:59:54Down
logo crawler_eb_germany2024-03-01 19:10:16
logo FAST-WebCrawler2024-03-01 01:42:25
logo Wordup-12024-02-22 11:54:47
logo Test2024-02-19 16:35:34
logo SeoulBot2024-02-16 17:23:12
logo Spider2024-02-12 04:28:51
logo MetaInspector2024-02-11 14:22:39UPUP
logo Splash2024-02-07 14:31:42UPUP
logo Diffbot2024-02-02 16:13:30UPUP
logo SEBot-WA2024-02-01 20:15:53
logo nvdorz2024-01-28 04:56:59
logo FriendlyCrawler2024-01-25 10:56:44
logo Autoconfig Test from USTC2024-01-23 19:21:19
logo SCAN2024-01-22 23:09:55
logo PhxBot2024-01-22 22:21:18
logo HappyWing2024-01-21 18:36:13
logo CXK_Bot2024-01-19 22:02:03
logo ISSCyberRiskCrawler2024-01-15 11:26:10
logo BDBot2024-01-14 05:11:52
logo SummalyBot2024-01-08 09:37:25
logo Dmbot2024-01-08 09:04:42
logo img2dataset2023-12-13 11:52:28UPUP
logo centurybot92023-11-24 14:24:49
logo The Knowledge AI2023-11-20 11:09:06
logo gregcrawler2023-11-16 23:31:54
logo thesis-research-bot2023-11-08 23:32:51
logo fidget-spinner-bot2023-11-07 01:24:58
logo my-tiny-bot2023-11-06 04:11:49
logo AntBot2023-11-05 16:00:56
logo friendly-spider2023-11-02 16:17:42
logo test-bot2023-10-30 22:19:01
logo newspaper2023-10-29 05:51:50
logo RssReaderBot2023-10-27 22:21:01
logo RainBot2023-10-23 08:44:40
logo lolbot2023-10-23 01:08:44
logo SEMrushBot2023-10-10 17:21:24UPUP
logo TinyTestBot2023-10-06 17:05:57
logo ds-robot2023-10-03 14:51:34
logo jaddjabot2023-09-28 19:08:53
logo Implcit:AdsTxt-Scrapr2023-09-28 13:58:34
logo ChimeUnfurlAgent2023-09-24 07:53:13
logo TinyTestBoty2023-09-21 06:58:00
logo Trokambot2023-09-15 07:16:47
logo CrawlyProjectCrawler2023-08-27 15:19:53
logo MoonBot2023-08-25 09:43:50
logo PyWebCopyBot2023-08-25 05:28:16UP
logo LunaBot2023-08-24 01:51:31
logo GoogleRot2023-08-19 05:26:41
logo ScamadviserExternalHit2023-08-18 19:05:17
logo lunabot2023-08-18 07:39:37
logo Nutch Spider2023-08-18 04:16:39
logo badbot2023-08-14 09:33:35
logo researchxoftheweb2023-07-30 15:28:30
logo test2023-07-29 03:34:31
logo goodbot2023-07-25 11:48:36
logo QBOT2023-07-08 04:54:07
logo BSbot2023-07-03 05:22:29
logo StactBot2023-06-27 13:30:52
logo AccompanyBot2023-06-27 06:53:40
logo MJBot2023-06-17 14:29:57
logo ResearchBot2023-06-11 14:53:40
logo PoliteCrawl2023-05-29 12:26:46
logo Elisabot2023-05-26 16:01:35
logo Storm-crawler2023-05-13 10:53:09
logo GlobalGoalsDirectoryBot2023-04-26 06:43:04
logo PrivateBot2023-04-23 00:16:17
logo PhantomBot2023-04-10 01:28:30
logo SuperBot2023-04-02 05:06:23
logo Bimbot2023-03-26 04:56:01
logo sempi.tech bot2023-03-23 08:49:29UP
logo dcrawl2023-03-20 01:05:29UPUP
logo Keerthi Bot2023-03-20 00:10:24

Maybe inactive robots (One year we have not heard about them)
logo AteniumBot2023-03-02 14:05:15
logo MauiBot2023-01-18 21:21:43
logo AdsTxtCrawler2023-01-12 17:12:15
logo TMM Crawler2023-01-09 21:01:17
logo AdsTxtCrawlerTP2023-01-02 06:14:34
logo Bot2022-12-07 10:08:29
logo linabot2022-12-05 05:05:35
logo google resource2022-12-04 08:33:50
logo DownloaderChrome2022-12-04 05:26:27
logo websusebot2022-11-19 02:30:20
logo Centro Ads.txt Crawler2022-11-10 13:07:00
logo seoscanners.net bot2022-10-26 18:04:08
logo _1z_bot2022-10-26 06:11:14
logo MetadataScraper2022-10-22 09:43:04
logo WhatStuffWhereBot2022-10-12 09:00:22
logo TrendsmapResolver2022-10-10 14:09:35
logo Zephuli-Bot2022-10-07 15:33:30
logo ViberBot2022-09-25 15:04:00
logo loli_spider2022-09-22 07:48:34
logo seolyt2022-09-19 05:29:07
logo TprAdsTxtCrawler2022-09-01 11:33:03
logo polite-crawler2022-08-30 01:03:44
logo web-crawler2022-08-24 06:24:14
logo RW_Crawler2022-08-17 15:40:04
logo My Nutch Spider2022-07-28 19:11:23
logo ThinkChaos2022-07-16 13:33:33
logo sc_bot2022-05-01 19:27:36
logo browserClientA2022-03-28 08:58:37
logo DAutoBot2022-03-24 11:08:31
logo PHPCrawl2022-03-18 07:31:22UP
logo Unknown bot2022-03-17 16:25:18
logo jurgendata2022-02-24 10:44:12Down
logo Crawlbot2022-01-11 11:17:47
logo p40Bot2022-01-11 07:42:22
logo PopScreen Bot2021-12-31 04:43:54
logo YFF352021-12-27 08:38:37
logo Grover2021-12-24 20:28:14
logo clark-crawler2021-12-01 21:25:51
logo FlfBaldrBot2021-12-01 14:21:32
logo SemanticJuice crawler2021-11-30 19:33:28UPUP
logo polaris botnet2021-11-24 16:20:15
logo loadtest2021-11-22 16:43:45
logo tes-cocrawler2021-09-24 00:16:09
logo CF-UC User Agent2021-09-23 06:44:10
logo CrawlBot2021-08-18 03:48:00
logo TestBot2021-08-12 03:17:05
logo TinyBotTestUA2021-07-27 16:41:28
logo trafilatura2021-07-25 09:03:07UPUP
logo aranhabot2021-07-01 01:53:25
logo bot2021-06-27 15:15:16
logo KauaiSlowBot2021-06-17 04:33:58
logo LanaiBot2021-05-18 05:18:18
logo GoodBot2021-05-14 05:26:27
logo chimebot2021-05-12 22:45:16
logo Snap URL Preview Service2021-04-22 01:15:03
logo autoemailspider2021-04-20 08:50:07
logo fake_bot2021-04-07 02:48:18
logo FlorKz.2021-04-02 16:36:36
logo 1M7bot2021-02-25 17:15:24
logo Aream Crawler2021-01-29 23:51:39
logo AAABot2021-01-29 04:36:11
logo AnyBot2021-01-15 19:02:00
logo KauaiBot2021-01-14 23:39:01
logo MyBot2020-12-12 23:17:21
logo gschaid bot2020-10-18 12:43:50Down
logo XTC BOTNET2020-09-08 16:16:38
logo Rely Bot2020-08-27 10:43:37
logo Info bot2020-08-24 20:12:21
logo NiueBot2020-08-12 12:20:03
logo BananaBot2020-07-18 17:01:13
logo Web Crawler2020-05-31 19:42:48
logo BorneoBot2020-05-27 19:23:30
logo dark_NeXus_Qbot2020-05-27 00:01:04
logo YunSecurityBot2020-04-16 23:14:45
logo aylienbot2020-03-01 10:42:29DownUP
logo bbot2020-01-11 06:11:09
logo brands-bot-logo2020-01-08 07:31:44
logo brands-bot2020-01-06 23:57:31
logo WebBot2019-11-25 14:49:31
logo WebCrawler2019-11-25 14:49:30
logo Fuze_Bot2019-11-12 16:04:52
logo AdBot2019-11-10 14:25:36
logo OgScrper2019-10-31 16:32:37
logo AdsrvrBot2019-10-28 08:30:19
logo UrlTest2019-10-24 04:14:53
logo CLcrawler2019-10-12 04:10:48
logo WorksOgCrawler2019-09-11 10:47:55
logo arachnode.net2019-08-11 09:00:54UPUP
logo TSpider2019-08-07 07:58:51
logo DaveCrawler2019-07-07 11:25:07
logo DaaS.sh crawler2019-05-07 17:52:28UP
logo Mixnode2018-08-20 01:06:54UP
logo 0082018-08-14 06:14:32DownUP
logo voltron2018-04-01 12:07:39DownUP
logo MailDBBot2017-04-06 20:08:19DownUP
logo scrapy-redis2017-03-06 12:58:15UP
logo AlexionResearchBot2011-08-19 02:17:35

Historical (Old robots, Inactive cravlers)
logo Frontera2019-05-23 13:42:37UPUP
logo Contacts Crawler2018-06-28 00:35:28UP
logo Quick-Crawler2018-06-08 07:47:17UP
logo Saleslift2017-12-23 05:57:41Down
logo Profound scrapyproject2017-01-05 14:39:19UPUP
logo Crawlera2016-02-08 16:08:25UP
logo import.io2016-02-03 18:34:09UP
logo Spinn3r2015-12-16 04:26:23DownDown
logo KimonoLabs2015-06-22 04:28:40UP
Among our clients
View more...
 salesforce.com, inc.  
 MailChimp  
 Dailymotion SA  
 Akamai Technologies, Inc.  
 Oracle  
 PayPal Holdings, Inc.