Resources > Crawlers list > LLMS-Txt-Scanner

logo LLMS-Txt-Scanner

ID: 217521
ProducerUnknown Author
Other robots of the same producer1M7bot, aa-prod-public-product, AAABot, AASA-Bot, ABEvalBot, AccompanyBot, AcquisitionScout, AdaptaBot, adaptive-bot, AdBot, ads-txt-crawler, AdsBot, AdsrvrBot, AdsTxtBot, AdsTxtCrawler, AdsTxtCrawlerTP, advanced_crawler, Adventurer, aeverly, AI-Security-Scanner, AlexionResearchBot, AlgorandSearchBot, alienfarm, ALittle Client, AliyunSecBot, altsvc-probe, Amuninni-WixFinder, Analyzer, AntBot, AnyBot, Apache-Smuggler, app-ads-crawler, Aranet-SearchBot, aranhabot, Aream Crawler, AstroSafeDomainCategoryEvidence, AteniumBot, AteveSearchSourceUrlDiscovery, atlas, AtlasBot, Authorized-BugBounty-Env-Scanner, Autoconfig Test from USTC, autoemailspider, Aviator-WebFetch, awi-research-crawler, BacklinkBot, BacklinkCrawler, BacklinkSpider, BackScanX, badbot, BananaBot, bang2012, bbot, BDBot, Bender, BesaCrawler, Bimbot, BitFire-IconUpscaler, BlazeBot, BlohaBot, BlueBot-scout-url, BorneoBot, Bot, Bot/Crawler, bot;, BotPoke, br-crawler, brands-bot, brands-bot-logo, brightdata-datacomp-calibration, brightdata-serpapi-mcp, browserClientA, BSbot, BusinessNameBot, BuyersCloudBot, CAISearchBot, CandidateCrawler, capsfusion-downloader, CareerPageBot, CCMetaBot, cc_bot, Centro Ads.txt Crawler, centurybot, CerebroBot, CF-UC User Agent, chatui-agent, chimebot, ChimeUnfurlAgent, ClaimChartBot, clark-crawler, ClawBot, CLcrawler, Clearscopebot, CMS-Checker, CMS-Security-Auditor, CommonCrawlNutchBot, company-domain-enricher, company-financial-miner, CompanyCrawler, CompassBot, ComplianceChecker, ConceptBurstBot, ContextFlow-Research, ConvomeshBot, CoreBot, CrawlBot, Crawler, Crawler test, Crawler-IP-Fetcher, crawler-purpose-audit, crawler-verification-research, CrawlerID, crawler_eb_germany, CrawlmBot, CrawlyProjectCrawler, CT-WP-Scanner, CTI-Engram-UrlFollow, Ctrl0Worker, CUBES-OC2-ContactDiscovery, CuratedSearchBot, curb, Custom-AsyncHttpClient, CustomerIranOpenSourceResearch, CustomScanner, CXK_Bot, dark_NeXus_Qbot, DARPResearchBot, DatacompMetadataIngestion, DataFoundationResearch, DAutoBot, DaveCrawler, Desync-Exploit, Detectobot, devlo-celeste-domain-waterfall, DF Bot, Dispatch, DMARC-Research-Bot, Dmbot, DnBCrawler, DNSCrawler, docs-proofreader, domain-info, domain-profiler, DomainAtlas, DomainAuditor, domainbox-domain-snapshot, DomainChecker, DomainCrawler, DomainFilter, DomainIntelBot, DomainScoutBot, Dormouse, DownloaderChrome, DropCatcher, ds-robot, edansbot, EdgeChangePasswordResearchCrawler, EFML-crawler, Elisabot, EmailFinderBot, EmailURLExtractor, EmailWolf, employer-address-finder, env-checker, EnvScanner, ErisBot, ExaticsBot, exp-oi2-scrape3, ExpiBot, expo-research-bot, Fake bot, fake_bot, FAST-WebCrawler, fastcrawl, fedcrawl, FeedBot, FeedValidator, Fellow Links Parser Robot, fidget-spinner-bot, FileSearchTools, FindAPIs-HealthChecker, FLAI-WebScraper, FlfBaldrBot, FlorKz., Friendly testing bot, friendly-spider, FriendlyCrawler, FriendlyRssReader, Friendly_Crawler, fsd-crawler, Fuze_Bot, g2crawler, gaeskeun_bot, GDPRScreeningBot, GeneralWebsiteSpider, Gensparkbot, GlobalGoalsDirectoryBot, GlobalWebSearch, Golfe, GoodBot, GoodReports-Compliance-Audit, google resource, Googlebot-Style, GoogleRot, GoParserBot, GoRedirectChecker, Gort, GoScanner, Govini-DQ-Checker, gregcrawler, Grover, gschaid bot, Guest, HanaleiBot, HappyWing, HAProxy-Smuggler, HawaiiBot, Headless Chrome, Headless Edge, HealthCheckBot, heimdall, Hello World, HighVolumeNewsSourceDiscovery, hopgoblin, HostHeaderScanner, HostQuality, howBot, HSD-Public-Business-Contact-Research, HubSeedsBot, HUNT-Bot, hypertok-recap-datacomp1b, IBM Crawler, icmp-tcp-oracle-probe, ImageBot, ImageMind, IMATA-Bot, IMJ-CompanyPage-Scraper, Implcit:AdsTxt-Scrapr, Info bot, IntelligenceCollector, ip9max, ipcrawl-exposure-scanner, jaddjabot, Jetfuel-VLM-Raw-Fetch, jinming-test-hf-dataset-to-hdfs, JRCBot, JSACrawlerBot, JSCrawler, jurgendata, K/b-scanner, kagibot, KauaiBot, Keerthi Bot, keoki-ucp-probe, keys-so-bot, keyword-monitor, LanaiBot, LeadFeed, LetrixLabs, libredtail-http, linabot, link-geo-country, LinkOps-Liveness, LinkScanProcessor, linus-torvalds-loves-you, Linux Gnu (cow), LinuxGetSsl, LiteLLM-Scanner, little-loops-link-checker, live-forum-pipeline, LiveScanner, llmstxt-survey, LMCSearchBot, loadtest, LocalDomainCrawler, LocalSearch, localsearch-homepage-enricher, LogMonitorBot, lolbot, loli_spider, LookoutBot, LumeWebScan, lunabot, Magellan, magicsearchdev, MailBot, MarkdownWorker, marketing-crawler, MassScanner, MauiBot, MB-AML-Research-Pipeline, MeridianBot, MetadataScraper, metaphor-data-pipeline, MiniWebCrawler, mj-wordpress-crawler, MJBot, ModelContextProtocol, Monibot, MoonBot, More, Mozilla, msh-pdf-origin-canary, My Nutch Spider, my-tiny-bot, MyBB-Archive-Bot, MyBot, MyCheckerBot, MyCrawler, MyEducationalCrawler, n8n-serp-crawler, NapBot, NavCrawl, Navy Federal, nbertaupete95, NDDHunter, NetAPI, NeuroPipeline-OfferVerifier, newspaper, newswtbot, Nexgate Ruby Client, Nginx-Smuggler, Nicecrawler, NinjaAIBot, NiueBot, Noliae Crawler, NsearchBot, Nutch Spider, NVBot, nvdorz, Nx-zDHub, OgScrper, OIDC-Fetcher, onex-datacomp-lance, OnlyHTML, OpenInternet-Discovery, OpenTheBoxBot, OpstimalBot, Orbbot, OSeans7Bot, osint research, OSSPVTracking, p40Bot, paper-reproduce-check-asset-url, parisbot, parquet-filter-downloader, passkey-domain-check, PDFBot, PersodiaResearchBot, PhantomBot, PhxBot, pimeyes-downloader-api, PinjaOSINTBot, pixmo-cap-convert, PlamoExplorer, PoeSearchBot, polaris botnet, polite-crawler, PoliteCrawl, Polon, Pomothy-Bot, PopScreen Bot, pqc-http-scan, PrecisionBot, PrivacyPolicyBot, PrivateBot, ProductLookoutBot, ProfileGrid-5995-mass-async, PromptingBot, Prospex, ProxyPool-Scanner, ProxyTest, PS_Daily, PUBGProductResearchBot, QBOT, quick_checker, r00ts3c-owned-you, RainBot, Rankabot, reasonix-web-fetch, recon-engine, ReconBot, RecordedFuture Global Inventory Crawler, Relevant, Rely Bot, research crawler, research dataset crawler, research-crawler, ResearchBot, researchxoftheweb, RevicBot, ReyilBot, ris-bot, Robot, robots-ai-permissions, robots-bulk-crawler, robots-scanner, robots-survey, robots-txt-audit, robots-txt-research, robots.py, RobotsArchiveBot, RobotsTxtCheck, RobotsTxtScraper, RootCrawl-Crawler, rss-discovery-crawler, rss-feed-index-bot, RssReaderBot, runtime-resources-research, RW_Crawler, S-Scanner, S33DBot, SaluteBot, ScamadviserExternalHit, SCAN, ScannerBot, ScanProxy, scrap, scraper, sc_bot, sealdexbot, Seamus, search-agent, search-engine-indexer, searchdb, SearchEngineBot, Searcherweb, Searcherxweb, SearchExpress, SEBot-WA, SecurityAudit, SecurityHeaderAnalyzer, SecurityResearchBot, security_txt_analysis, sempi.tech bot, sendgrid-scanner, senthor-research, SeoCherryBot, seoscanners.net bot, SeoulBot, SequentryBrandRadarMVP, SerpCrawler, servicefilter, sexy-spider, sftp-scanner, ShellBot, SignalLabCrawler, SimpleAsyncPageCrawler, site-checker, SitemapBot, SitemapCrawler, SitemapCrawler.de crawler, siteradar, SkillGymCorpusBot, SkynetBot, SkyworkSpider, SlaccaleBot, SmartBotCrawler, smol-tiny-tarantula, Snap URL Preview Service, Social Listening API, SocialMediaScrapperv2, SOcrawl, some agent, SourcingEngineBot, SpamBot, Spider, Spider_Bot, SQ-Scanner, SQLi-Scanner, SSSSBot, StactBot, Storm-crawler, StrawberryBot, studyBot, Supabase crawler, SupaGuard, SuperBot, SuperFastScraper, SuperMegaHyperColossalMassiveGiganticAstronomicalM, supply-crawler, SurPhish-Reproduction-Crawler, SynapicCrawl, SyncSearch, t2i-data-curation, TakeoverScanner, TDM-Checker, TDMScanner, TechDetect, TechNewsBot, TechScanBot, terrabot-owned-you, tes-cocrawler, Test, Test Certificate Info, test-bot, TestBot, TestCrawler, TestSearchSpider, TestUserAgent, The Knowledge AI, TheInternetSearch, Theophrastus, thesis-research-bot, ThinkChaos, ThresselScanner, Tier1Bot, Timing-Analysis, TinyBotTestUA, TinyTestBot, TinyTestBoty, TLM-Audit-Scanner, TLSScanner, TMM Crawler, ToolissBot, toriset-llm-scraper, TprAdsTxtCrawler, Tranco Crawler, tranco-crawler, TrendsmapResolver, Triplechecker, Trokambot, TrustedSearchBot, TSpider, UAScrapper, UCP-Discovery, uk-nhs-data, ultimate_sitemap_parser, Unite-ResolveUrl, UniversalSeedIntelligenceAgent, Unknown bot, UppinaMonitor, UrlTest, UrukyFaviconProxy, v0bot, vague, VDP-Scanner, VexelSearchBot, ViberBot, VICIdial-Recon-Classifier, VisionSet, VsuSearchSpider, vuln_scanner, WAF-Bypass-Test, WanScannerBot, WarmBadge-Trawler, warmup-scraper, WarpBot, Web Crawler, web-crawler, WebBot, webcollage, WebCrawler, webflow-staging-diff, WebRankSpider, WebScraperBot, website-candidate-fetch-enricher, WebsiteFeeder, websusebot, web_fetcher, WF search, WhatStuffWhereBot, white-analyz-bug, wise, WooCommerce-Detector, Wordup-1, Wordupindexinfo1, WorksOgCrawler, WorldBot, wpbot, WPCheckBot, WPDataResearchBot, WPDetector, WPScanner, xfa1, XTC BOTNET, YFF35, your-search-bot, Ytools, YunSecurityBot, Yuuperbot, ZAKREGEX-Scanner, zebra-v2-bot, Zephuli-Bot, zero-tier-classifier, ZeroBot, zzhbot, _1z_bot, _zbot
Bot URL
StatusActive Active

Known variants

LLMS-Txt-Scanner/1.0

UseragentstringLLMS-Txt-Scanner/1.0 (Research)
CategoryWeb scraper
First seen2026-02-01 05:42:20
Last seen2026-08-09 06:25:19
IP addresses12
Walk from
20.109.86.22820.109.86.228US
132.196.80.130132.196.80.130GB
20.102.46.19620.102.46.196US
172.214.45.199172.214.45.199US
20.161.77.16220.161.77.162US
128.24.162.209128.24.162.209GB
52.154.132.18052.154.132.180US
20.25.34.3520.25.34.35US
64.236.131.24164.236.131.241 GB
172.183.135.241172.183.135.241US

 
.. and more .. (All IP Addresses available download to users with subscription. ... Learn More)
 
Among our clients
View more...
 salesforce.com, inc.  
 MailChimp  
 Dailymotion SA  
 Allegro  
 Oracle  
 PayPal Holdings, Inc.