Resources > Crawlers list > Robots-txt.ai bot

logo Robots-txt.ai bot

ID: 228165
ProducerRobots-txt.ai External link
Bot URLhttps://robots-txt.ai/crawler External link
StatusActive Active

Known variants

Robots.txt crawler

UseragentstringRobots.txt crawler - helps protect publisher content and lower bot traffic - https://robots-txt.ai/crawler
Category-- Uncategorised --
First seen2026-08-19 20:05:02
Last seen2026-09-21 21:51:12
IP addresses7
Walk from
213.188.90.118?US
136.227.167.144? US
136.227.188.43? US
134.199.94.96? US
134.199.82.178? US
213.159.13.42? US
136.227.164.243? US
 

Robots-txt Robots.txt Checker/1.0

UseragentstringRobots-txt Robots.txt Checker/1.0
Category-- Uncategorised --
First seen2026-08-22 20:42:24
Last seen2026-08-22 20:42:25
IP addresses1
Walk from
52.20.197.4ec2-52-20-197-4.compute-1.amazonaws.comUS
 
Among our clients
View more...
 salesforce.com, inc.  
 MailChimp  
 Dailymotion SA  
 Allegro  
 Oracle  
 PayPal Holdings, Inc.  

Recently Added

Products

Resources

About Us