Resources > Crawlers list > Scrapy

logo Scrapy

ID: 23751
ProducerZyte External link
Other robots of the same producerContacts Crawler, Crawlera, Frontera, Quick-Crawler, Splash
Bot URLhttps://scrapy.org/ External link
StatusActive Active

Known variants

Scrapy/1.7.3

UseragentstringScrapy/1.7.3 (+https://scrapy.org)
CategoryWeb scraper
First seen2019-08-16 06:35:10
Last seen2024-03-16 08:02:20
IP addresses8
Walk from
38.242.214.232vmi888549.contaboserver.net?
38.242.214.236vmi888550.contaboserver.net?
154.19.200.18154.19.200.18 US
27.224.137.10627.224.137.106 CN
124.160.236.43124.160.236.43 CN
78.47.48.49?DE
128.91.91.211hpcc017.wharton.upenn.edu US
128.91.88.38hpcc032.wharton.upenn.edu US
 

Scrapy/2.5.1

UseragentstringScrapy/2.5.1 (+https://scrapy.org)
CategoryWeb scraper
First seen2022-01-02 06:43:48
Last seen2024-03-12 11:53:52
IP addresses25
Walk from
45.154.163.17045.154.163.170 RU
77.83.80.20777.83.80.207 ?
45.153.52.7745.153.52.77 RU
195.19.169.199195.19.169.199 RU
45.130.144.23345.130.144.233 RU
45.146.168.13445.146.168.134 RU
194.226.123.223194.226.123.223 RU
94.158.189.15994.158.189.159 ?
45.139.55.1945.139.55.19 RU
45.139.126.18345.139.126.183 RU

 
.. and more .. (All IP Addresses available download to users with subscription. ... Learn More)
 

Scrapy/2.11.0

UseragentstringScrapy/2.11.0 (+https://scrapy.org)
CategoryWeb scraper
First seen2024-01-04 09:09:43
Last seen2024-02-27 22:18:50
IP addresses4
Walk from
77.181.127.82dynamic-077-181-127-082.77.181.pool.telefonica.de DE
222.185.36.65222.185.36.65 ?
223.73.191.20223.73.191.20 CN
112.49.240.214112.49.240.214 CN
 

Scrapy/2.9.0

UseragentstringScrapy/2.9.0 (+https://scrapy.org)
CategoryWeb scraper
First seen2023-07-08 12:50:36
Last seen2024-02-22 05:43:33
IP addresses5
Walk from
3.73.63.44ec2-3-73-63-44.eu-central-1.compute.amazonaws.comDE
52.128.13.17452.128.13.174 US
154.13.203.88154.13.203.88 US
54.152.101.126ec2-54-152-101-126.compute-1.amazonaws.comUS
150.109.18.41150.109.18.41SG
 

Scrapy/2.8.0

UseragentstringScrapy/2.8.0 (+https://scrapy.org)
CategoryWeb scraper
First seen2023-02-05 22:33:02
Last seen2023-12-19 04:42:32
IP addresses19
Walk from
34.42.253.119119.253.42.34.bc.googleusercontent.comUS
34.123.92.193193.92.123.34.bc.googleusercontent.comUS
34.71.173.104104.173.71.34.bc.googleusercontent.comUS
34.135.125.7676.125.135.34.bc.googleusercontent.comUS
34.135.155.6868.155.135.34.bc.googleusercontent.comUS
35.202.239.2424.239.202.35.bc.googleusercontent.comUS
34.170.11.231231.11.170.34.bc.googleusercontent.comUS
34.29.13.147147.13.29.34.bc.googleusercontent.comUS
34.170.34.250250.34.170.34.bc.googleusercontent.comUS
34.69.163.244244.163.69.34.bc.googleusercontent.comUS

 
.. and more .. (All IP Addresses available download to users with subscription. ... Learn More)
 

Scrapy/2.6.3

UseragentstringScrapy/2.6.3 (+https://scrapy.org)
CategoryWeb scraper
First seen2023-08-03 18:44:39
Last seen2023-12-13 10:26:53
IP addresses1
Walk from
34.29.54.138138.54.29.34.bc.googleusercontent.comUS
 

Scrapy/1.7.4

UseragentstringScrapy/1.7.4 (+https://scrapy.org)
CategoryWeb scraper
First seen2021-03-28 12:45:23
Last seen2023-09-15 03:34:15
IP addresses5
Walk from
116.203.206.201static.201.206.203.116.clients.your-server.deDE
88.99.170.66static.66.170.99.88.clients.your-server.deDE
95.216.173.132static.132.173.216.95.clients.your-server.deFI
95.216.173.189static.189.173.216.95.clients.your-server.deFI
65.21.253.28static.28.253.21.65.clients.your-server.deFI
 

Scrapy/2.10.0

UseragentstringScrapy/2.10.0 (+https://scrapy.org)
CategoryWeb scraper
First seen2023-08-27 17:31:21
Last seen2023-08-27 17:31:21
IP addresses1
Walk from
176.42.28.173176.42.28.173 TR
 

Scrapy/2.6.1

UseragentstringScrapy/2.6.1 (+https://scrapy.org)
CategoryWeb scraper
First seen2022-03-26 07:45:49
Last seen2023-04-11 07:34:22
IP addresses8
Walk from
104.154.180.126126.180.154.104.bc.googleusercontent.comUS
37.35.43.5837.35.43.58 IT
154.194.8.121154.194.8.121 FR
95.216.46.199static.199.46.216.95.clients.your-server.deFI
23.247.47.19823.247.47.198 US
173.212.239.43173.212.239.43DE
45.9.88.12345.9.88.123 RU
52.72.8.185ec2-52-72-8-185.compute-1.amazonaws.comUS
 

Scrapy/2.6.2

UseragentstringScrapy/2.6.2 (+https://scrapy.org)
CategoryWeb scraper
First seen2023-01-22 19:28:46
Last seen2023-03-26 23:16:58
IP addresses2
Walk from
46.2.14.184? TR
61.135.204.19461.135.204.194 CN
 

Scrapy/2.7.1

UseragentstringScrapy/2.7.1 (+https://scrapy.org)
CategoryWeb scraper
First seen2023-01-29 23:30:42
Last seen2023-02-02 13:18:10
IP addresses2
Walk from
90.154.10.13090.154.10.130 RU
79.110.55.1079.110.55.10?
 

Scrapy/2.4.1

UseragentstringScrapy/2.4.1 (+https://scrapy.org)
CategoryWeb scraper
First seen2021-03-25 04:33:03
Last seen2022-10-20 07:16:34
IP addresses4
Walk from
183.253.10.84183.253.10.84 CN
52.72.8.185ec2-52-72-8-185.compute-1.amazonaws.comUS
142.255.86.85cpe-142-255-86-85.nyc.res.rr.com US
175.102.16.34175.102.16.34 CN
 

Scrapy/2.5.1

UseragentstringScrapy/2.5.1 (+https://scrapy.org),gzip(gfe)
CategoryWeb scraper
First seen2022-10-17 10:38:17
Last seen2022-10-17 10:38:17
IP addresses1
Walk from
66.249.81.109google-proxy-66-249-81-109.google.comUS
 

Scrapy/1.1.3

UseragentstringScrapy/1.1.3 (+http://scrapy.org)
CategoryWeb scraper
First seen2018-03-10 17:09:30
Last seen2022-08-29 20:04:55
IP addresses8
Walk from
88.99.0.174static.174.0.99.88.clients.your-server.deDE
148.251.144.116static.116.144.251.148.clients.your-server.deDE
46.4.114.158static.158.114.4.46.clients.your-server.deDE
116.202.83.62?DE
88.99.164.194?DE
138.201.34.140?DE
195.201.85.124?DE
138.201.36.40?DE
 

Scrapy/1.5.1

UseragentstringScrapy/1.5.1 (+https://scrapy.org)
CategoryWeb scraper
First seen2018-09-05 13:34:29
Last seen2021-12-09 04:54:59
IP addresses21
Walk from
13.56.111.204ec2-13-56-111-204.us-west-1.compute.amazonaws.comUS
54.172.91.236ec2-54-172-91-236.compute-1.amazonaws.comUS
121.156.47.204? KR
195.3.144.210colo-200.altnet.lv LV
189.120.120.204bd7878cc.virtua.com.br BR
195.3.144.185colo-185.altnet.lv LV
83.137.2.249? GB
78.145.209.112host-78-145-209-112.as13285.net GB
208.110.66.34?US
188.92.74.189?LV

 
.. and more .. (All IP Addresses available download to users with subscription. ... Learn More)
 

Scrapy/2.5.0

UseragentstringScrapy/2.5.0 (+https://scrapy.org)
CategoryWeb scraper
First seen2021-08-10 20:54:06
Last seen2021-11-24 23:39:36
IP addresses3
Walk from
185.58.206.67ih2083552.ihor-dedic.ruRU
137.184.96.247137.184.96.247US
35.223.102.4040.102.223.35.bc.googleusercontent.comUS
 

Scrapy/1.5.0

UseragentstringScrapy/1.5.0 (+https://scrapy.org)
CategoryWeb scraper
Respects robots.txtNo
First seen2018-01-15 20:38:07
Last seen2021-07-29 22:13:37
IP addresses27
Walk from
100.25.46.189ec2-100-25-46-189.compute-1.amazonaws.comUS
54.152.77.31ec2-54-152-77-31.compute-1.amazonaws.comUS
52.86.112.146ec2-52-86-112-146.compute-1.amazonaws.comUS
13.56.179.125ec2-13-56-179-125.us-west-1.compute.amazonaws.comUS
104.144.97.164?US
52.193.88.113ec2-52-193-88-113.ap-northeast-1.compute.amazonaws.comJP
34.232.52.147ec2-34-232-52-147.compute-1.amazonaws.comUS
121.156.47.204? KR
35.193.175.92?US
54.236.48.97ec2-54-236-48-97.compute-1.amazonaws.comUS

 
.. and more .. (All IP Addresses available download to users with subscription. ... Learn More)
 

Scrapy/2.2.1

UseragentstringScrapy/2.2.1 (+https://scrapy.org)
CategoryWeb scraper
First seen2021-01-28 17:17:23
Last seen2021-01-28 17:17:23
IP addresses2
Walk from
34.76.170.167167.170.76.34.bc.googleusercontent.comUS
34.76.224.176176.224.76.34.bc.googleusercontent.comUS
 

Scrapy/2.3.0

UseragentstringScrapy/2.3.0 (+https://scrapy.org)
CategoryWeb scraper
First seen2020-11-07 05:39:00
Last seen2021-01-04 07:55:09
IP addresses2
Walk from
2a01:4f9:c010:eaec::12a01:4f9:c010:eaec::1DE
51.15.119.7474-119-15-51.instances.scw.cloudFR
 

Scrapy/1.8.0

UseragentstringScrapy/1.8.0 (+https://scrapy.org)
CategoryWeb scraper
First seen2019-11-11 05:12:58
Last seen2021-01-03 04:29:13
IP addresses7
Walk from
107.178.236.1515.236.178.107.gae.googleusercontent.comUS
35.202.86.4646.86.202.35.bc.googleusercontent.comUS
47.89.225.105?US
160.114.55.188rgai3.inf.u-szeged.hu HU
123.25.21.157mail.soft365.com.vn VN
14.238.15.74static.vnpt.vn VN
47.90.241.15747.90.241.157US
 

Scrapy/1.6.0

UseragentstringScrapy/1.6.0 (+https://scrapy.org)
CategoryWeb scraper
First seen2019-02-04 21:58:11
Last seen2020-10-19 13:26:38
IP addresses62
Walk from
138.201.35.152static.152.35.201.138.clients.your-server.deDE
138.201.36.40static.40.36.201.138.clients.your-server.deDE
80.211.112.51host51-112-211-80.serverdedicati.aruba.itIT
39.100.157.53?CN
39.100.156.45?CN
93.137.135.11993-137-135-119.adsl.net.t-com.hr HR
47.92.141.136?CN
39.100.121.198?CN
47.92.96.33?CN
39.100.156.237?CN

 
.. and more .. (All IP Addresses available download to users with subscription. ... Learn More)
 

Scrapy/2.2.0

UseragentstringScrapy/2.2.0 (+https://scrapy.org)
CategoryWeb scraper
First seen2020-07-06 11:04:52
Last seen2020-07-06 11:12:46
IP addresses1
Walk from
185.153.151.34185.153.151.34LU
 

Scrapy/2.1.0

UseragentstringScrapy/2.1.0 (+https://scrapy.org)
CategoryWeb scraper
First seen2020-07-01 14:16:52
Last seen2020-07-01 14:52:14
IP addresses1
Walk from
82.102.19.196ncopus196.niliny.pwBE
 

Scrapy/1.7.2

UseragentstringScrapy/1.7.2 (+https://scrapy.org)
CategoryWeb scraper
First seen2019-10-17 16:38:47
Last seen2020-06-25 04:38:18
IP addresses2
Walk from
176.74.192.99hbase7.domaincrawler.comSE
176.74.199.2?SE
 

Scrapy/1.4.0

UseragentstringScrapy/1.4.0 (+http://scrapy.org)
CategoryWeb scraper
Respects robots.txtNo
First seen2017-07-28 16:43:00
Last seen2020-01-13 13:16:28
IP addresses51
Walk from
36.226.224.3736-226-224-37.dynamic-ip.hinet.net TW
36.227.109.23136-227-109-231.dynamic-ip.hinet.net TW
36.224.51.736-224-51-7.dynamic-ip.hinet.net TW
36.226.231.2036-226-231-20.dynamic-ip.hinet.net TW
36.228.229.7936-228-229-79.dynamic-ip.hinet.net TW
36.228.229.9436-228-229-94.dynamic-ip.hinet.net TW
36.226.224.5436-226-224-54.dynamic-ip.hinet.net TW
36.228.225.17136-228-225-171.dynamic-ip.hinet.net TW
36.228.224.13136-228-224-131.dynamic-ip.hinet.net TW
36.226.230.20836-226-230-208.dynamic-ip.hinet.net TW

 
.. and more .. (All IP Addresses available download to users with subscription. ... Learn More)
 

Scrapy/1.0.5

UseragentstringScrapy/1.0.5 (+http://scrapy.org)
CategoryWeb scraper
Respects robots.txtNo
First seen2016-02-12 15:26:00
Last seen2019-12-30 08:01:46
IP addresses12
Walk from
136.243.11.22?DE
78.46.88.202?DE
136.243.76.224?DE
58.246.174.26? CN
52.198.176.249ec2-52-198-176-249.ap-northeast-1.compute.amazonaws.comJP
52.196.217.192ec2-52-196-217-192.ap-northeast-1.compute.amazonaws.comJP
52.201.250.99ec2-52-201-250-99.compute-1.amazonaws.comUS
52.68.41.151ec2-52-68-41-151.ap-northeast-1.compute.amazonaws.comJP
164.132.160.129ns3035942.ip-164-132-160.euFR
93.186.148.9493.186.148.94-ifnl.net GB

 
.. and more .. (All IP Addresses available download to users with subscription. ... Learn More)
 

Scrapy/1.7.1

UseragentstringScrapy/1.7.1 (+https://scrapy.org)
CategoryWeb scraper
First seen2019-08-06 08:42:08
Last seen2019-08-27 11:30:05
IP addresses2
Walk from
147.92.33.234147.92.33.234 HK
8.44.182.23? US
 

Scrapy/1.5.2

UseragentstringScrapy/1.5.2 (+https://scrapy.org)
CategoryWeb scraper
First seen2019-02-03 19:06:16
Last seen2019-07-07 06:18:17
IP addresses3
Walk from
130.203.136.75? US
185.47.220.131? CZ
188.175.125.148188.175.125.148 CZ
 

Scrapy/1.0.3

UseragentstringScrapy/1.0.3 (+http://scrapy.org)
CategoryWeb scraper
Respects robots.txtNo
First seen2015-12-14 08:00:00
Last seen2018-05-11 07:16:04
IP addresses21
Walk from
54.160.194.98ec2-54-160-194-98.compute-1.amazonaws.comUS
61.164.42.140? CN
139.59.29.77?IN
52.88.187.57ec2-52-88-187-57.us-west-2.compute.amazonaws.comUS
108.61.242.65108.61.242.65.vultr.comUS
46.166.188.231?NL
52.40.148.43ec2-52-40-148-43.us-west-2.compute.amazonaws.comUS
54.69.99.109ec2-54-69-99-109.us-west-2.compute.amazonaws.comUS
108.83.93.215108-83-93-215.lightspeed.wepbfl.sbcglobal.net US
108.61.205.28108.61.205.28.vultr.comUS

 
.. and more .. (All IP Addresses available download to users with subscription. ... Learn More)
 

Scrapy/1.3.0

UseragentstringScrapy/1.3.0 (+http://scrapy.org)
CategoryWeb scraper
First seen2017-01-30 01:37:54
Last seen2018-04-27 06:46:07
IP addresses5
Walk from
13.57.36.202ec2-13-57-36-202.us-west-1.compute.amazonaws.comUS
104.218.141.98? US
23.106.83.199?US
23.81.67.216?US
136.243.66.162?DE
 

Scrapy/1.1.2

UseragentstringScrapy/1.1.2 (+http://scrapy.org)
CategoryWeb scraper
First seen2016-09-10 01:45:54
Last seen2018-03-19 11:01:04
IP addresses8
Walk from
13.113.121.129ec2-13-113-121-129.ap-northeast-1.compute.amazonaws.comJP
52.197.188.46ec2-52-197-188-46.ap-northeast-1.compute.amazonaws.comJP
52.68.219.226ec2-52-68-219-226.ap-northeast-1.compute.amazonaws.comJP
13.113.12.64ec2-13-113-12-64.ap-northeast-1.compute.amazonaws.comJP
52.199.78.154ec2-52-199-78-154.ap-northeast-1.compute.amazonaws.comJP
52.199.104.67ec2-52-199-104-67.ap-northeast-1.compute.amazonaws.comJP
52.43.49.188ec2-52-43-49-188.us-west-2.compute.amazonaws.comUS
52.25.226.122ec2-52-25-226-122.us-west-2.compute.amazonaws.comUS
 

Scrapy/1.3.2

UseragentstringScrapy/1.3.2 (+http://scrapy.org)
CategoryWeb scraper
First seen2017-12-20 10:34:45
Last seen2017-12-20 10:34:45
IP addresses1
Walk from
111.200.195.186111.200.195.186 CN
 

Scrapy/1.3.3

UseragentstringScrapy/1.3.3 (+http://scrapy.org)
CategoryWeb scraper
First seen2017-02-08 11:05:24
Last seen2017-11-07 09:12:10
IP addresses8
Walk from
13.114.207.110ec2-13-114-207-110.ap-northeast-1.compute.amazonaws.comJP
52.196.202.55ec2-52-196-202-55.ap-northeast-1.compute.amazonaws.comJP
34.208.16.193ec2-34-208-16-193.us-west-2.compute.amazonaws.comUS
107.23.10.123ec2-107-23-10-123.compute-1.amazonaws.comUS
54.167.210.113ec2-54-167-210-113.compute-1.amazonaws.comUS
176.27.243.42b01bf32a.bb.sky.com GB
124.72.95.10? CN
45.33.43.32li993-32.members.linode.comUS
 

Scrapy/0.16.5

UseragentstringScrapy/0.16.5 (+http://scrapy.org)
CategoryWeb scraper
First seen2016-08-18 20:17:35
Last seen2017-08-16 09:26:14
IP addresses2
Walk from
98.100.180.20rrcs-98-100-180-20.central.biz.rr.com US
67.53.77.196rrcs-67-53-77-196.west.biz.rr.com US
 

Scrapy/0.24.5

UseragentstringScrapy/0.24.5 (+http://scrapy.org)
CategoryWeb scraper
First seen2014-12-19 09:53:17
Last seen2017-04-14 00:30:33
IP addresses5
Walk from
52.197.129.56ec2-52-197-129-56.ap-northeast-1.compute.amazonaws.comJP
52.69.69.138ec2-52-69-69-138.ap-northeast-1.compute.amazonaws.comJP
52.192.241.191ec2-52-192-241-191.ap-northeast-1.compute.amazonaws.comJP
107.23.56.124ec2-107-23-56-124.compute-1.amazonaws.comUS
52.69.82.104ec2-52-69-82-104.ap-northeast-1.compute.amazonaws.comJP
 

Scrapy/1.2.2

UseragentstringScrapy/1.2.2 (+http://scrapy.org)
CategoryWeb scraper
First seen2017-01-12 23:59:17
Last seen2017-03-26 23:59:17
IP addresses1
Walk from
52.200.30.172ec2-52-200-30-172.compute-1.amazonaws.comUS
 

Scrapy/1.0.5.post4

UseragentstringScrapy/1.0.5.post4+g4b324a8 (+http://scrapy.org)
CategoryWeb scraper
First seen2017-01-07 23:32:39
Last seen2017-01-07 23:32:39
IP addresses1
Walk from
98.126.65.56?US
 

Scrapy/1.2.1

UseragentstringScrapy/1.2.1 (+http://scrapy.org)
CategoryWeb scraper
First seen2016-11-09 23:25:03
Last seen2016-12-13 16:37:29
IP addresses3
Walk from
5.230.153.214?GB
178.18.151.162? CZ
54.213.151.77ec2-54-213-151-77.us-west-2.compute.amazonaws.comUS
 

Scrapy/1.1.1

UseragentstringScrapy/1.1.1 (+http://scrapy.org)
CategoryWeb scraper
Respects robots.txtNo
First seen2016-09-11 07:43:33
Last seen2016-12-04 14:34:58
IP addresses3
Walk from
138.201.52.218?DE
134.96.225.227colossus02.cs.uni-saarland.de DE
192.240.14.1host01.fna.fujitsu.comUS
 

Scrapy/1.1.0

UseragentstringScrapy/1.1.0 (+http://scrapy.org)
CategoryWeb scraper
Respects robots.txtNo
First seen2016-06-12 20:00:00
Last seen2016-09-01 16:03:46
IP addresses4
Walk from
138.201.57.140?DE
148.251.133.219?DE
192.240.14.1host01.fna.fujitsu.comUS
92.249.223.12192-249-223-121.pool.digikabel.hu HU
 

Scrapy/1.0.4

UseragentstringScrapy/1.0.4 (+http://scrapy.org)
CategoryWeb scraper
First seen2016-01-27 06:10:00
Last seen2016-02-19 11:08:30
IP addresses2
Walk from
103.14.245.38103.14.245.38 SG
219.92.50.109gbk-50-109.tm.net.my MY
 

Scrapy/1.0.1

UseragentstringScrapy/1.0.1 (+http://scrapy.org)
CategoryWeb scraper
First seen2015-02-07 05:07:09
Last seen2016-02-10 14:16:41
IP addresses3
Walk from
194.150.168.95kaputte.li DE
46.105.110.61ns3435966.ip-46-105-110.euFR
158.69.26.33ns517020.ip-158-69-26.netCA
 

Scrapy/0.24.0

UseragentstringScrapy/0.24.0 (+http://scrapy.org)
CategoryWeb scraper
First seen2014-08-08 09:15:00
Last seen2015-12-05 09:29:21
IP addresses2
Walk from
114.212.190.201cs-cloud.nju.edu.cn CN
54.72.183.135ec2-54-72-183-135.eu-west-1.compute.amazonaws.comUS
 

Scrapy/0.24.6

UseragentstringScrapy/0.24.6 (+http://scrapy.org)
CategoryWeb scraper
First seen2015-08-19 21:13:23
Last seen2015-08-19 21:13:23
IP addresses1
Walk from
188.40.45.131?DE
 

Scrapy/0.25.1

UseragentstringScrapy/0.25.1 (+http://scrapy.org)
CategoryWeb scraper
First seen2015-04-08 18:52:07
Last seen2015-04-08 18:52:08
IP addresses1
Walk from
69.124.113.52ool-457c7134.dyn.optonline.net US
 

Scrapy/0.24.4

UseragentstringScrapy/0.24.4 (+http://scrapy.org)
CategoryWeb scraper
First seen2015-12-01 07:19:29
Last seen2014-12-10 23:03:17
IP addresses2
Walk from
97.74.4.58ip-97-74-4-58.ip.secureserver.netUS
54.72.183.135ec2-54-72-183-135.eu-west-1.compute.amazonaws.comIE
 

Scrapy/0.22.0

UseragentstringScrapy/0.22.2 (+http://scrapy.org)
CategoryWeb scraper
First seen2014-02-18 10:40:06
Last seen2014-06-18 20:56:00
IP addresses2
Walk from
201.204.48.114201.204.48.114 CR
64.53.192.27d53-64-27-192.nap.wideopenwest.com US
 
Among our clients
View more...
 salesforce.com, inc.  
 MailChimp  
 Dailymotion SA  
 Akamai Technologies, Inc.  
 Oracle  
 PayPal Holdings, Inc.