Resources > UA list: crawlers

Udger database includes detailed information about every single user agent and operating system

Select list type: Browsers, email clients ... | Operating systems | Devices| Crawlers (Robots)

Category: All - Search engines - Marketing - Screenshots - Monitors - Feeds - Link checkers - Scrapers - Tools - Speed test - Vulnerability/Virus - AI

Crawlers (Robots)	Last seen	bot URL	Producer URL
anthropic bot	2026-02-03 10:22:27		UP
OpenAI bot	2026-02-03 09:09:38	UP	UP
PerplexityBot	2026-02-03 09:01:00	UP	UP
Gemini bot	2026-02-03 06:14:47	UP	UP
TerraCotta	2026-02-03 04:27:36	UP	UP
Google-NotebookLM	2026-02-02 16:36:29	UP	UP
QuillBot	2026-02-01 19:59:04	UP	UP
WRTNBot	2026-01-27 07:35:00	Down	UP
MistralAI crawler	2026-01-22 13:49:13	UP	UP
amazon-kendra	2026-01-21 17:33:33	UP	UP
Aithlas bot	2025-12-17 07:41:56		UP
iAsk.ai	2025-12-16 07:14:28	UP	UP
Algolia bot	2025-11-30 05:11:46		UP
Ai2Bot	2025-11-12 04:38:28	UP	UP
NetShelter ContentScan	2025-10-31 13:55:46		UP
LinerBot	2025-10-13 06:11:14	UP	UP
amazon-QBusiness	2025-09-14 16:42:36	UP	UP
PanguBot	2025-08-11 04:48:45	UP	UP
ZanistaBot	2025-06-21 02:20:16	UP	UP
Maybe inactive robots (One year we have not heard about them)
Spawning bot	2024-09-29 05:26:01		UP
ChatGLM Spider	2024-09-26 04:40:46	UP	UP
LLM-jp-Crawler	2024-08-22 04:21:26	UP	UP
Google Cloud AI	2020-10-19 17:23:41	UP	UP
Historical (Old robots, Inactive cravlers)
Crawling at Home	2021-11-03 05:24:00	Down	UP