Resources > Crawlers list > loc.gov crawler

logo loc.gov crawler

ID: 16510
ProducerThe Library of Congress USA External link
Bot URLhttps://www.loc.gov/programs/web-archiving/for-site-owners/ External link
StatusActive Active

Known variants

loc.gov crawler

UseragentstringMozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/78.0.3904.97 Safari/537.36 (+https://www.loc.gov/programs/web-archiving/for-site-owners/)
Category-- Uncategorised --
First seen2021-04-25 19:34:07
Last seen2021-04-25 19:34:07
IP addresses1
Walk from
54.236.226.189ec2-54-236-226-189.compute-1.amazonaws.comUS
 

special_archiver/3.3.0

UseragentstringMozilla/5.0 (compatible; special_archiver/3.3.0; +http://www.loc.gov/webarchiving/notice_to_webmasters.html)
Category-- Uncategorised --
First seen2018-08-14 10:41:49
Last seen2018-10-16 20:40:15
IP addresses1
Walk from
207.241.231.53wbgrp-crawl213.us.archive.orgUS
 

special_archiver/3.3.0

UseragentstringMozilla/5.0 (compatible; special_archiver/3.3.0 +http://www.loc.gov/webarchiving/notice_to_webmasters.html)
Category-- Uncategorised --
First seen2015-09-05 20:14:26
Last seen2015-11-05 03:21:28
IP addresses2
Walk from
207.241.237.152wbgrp-crawl025.us.archive.orgUS
207.241.237.95wbgrp-crawl028.us.archive.orgUS
 

heritrix/3.2.0

UseragentstringMozilla/5.0 (compatible; heritrix/3.2.0 +http://webarchiveqr.loc.gov/about/loc-notification-webmasters.html)
Category-- Uncategorised --
First seen2015-03-10 20:24:16
Last seen2015-03-10 20:24:16
IP addresses1
Walk from
140.147.249.70lx8.loc.gov US
 

special_archiver/3.2.0

UseragentstringMozilla/5.0 (compatible; special_archiver/3.2.0 +http://www.loc.gov/webarchiving/notice_to_webmasters.html)
Category-- Uncategorised --
First seen2013-09-11 08:03:39
Last seen2013-11-09 14:12:32
IP addresses1
Walk from
207.241.237.166wbgrp-crawl011.us.archive.orgUS
 
Among our clients
View more...
 salesforce.com, inc.  
 MailChimp  
 Dailymotion SA  
 Allegro  
 Oracle  
 PayPal Holdings, Inc.  

Recently Added

Products

Resources

About Us