| alecxe/scrapy-fake-useragent |
654 |
|
50 |
5 |
over 2 years ago |
12 |
August 15, 2020 |
5 |
mit |
Python |
| Random User-Agent middleware based on fake-useragent |
| AccordBox/awesome-scrapy |
450 |
|
0 |
0 |
over 3 years ago |
0 |
|
2 |
|
|
| A curated list of awesome packages, articles, and other cool resources from the Scrapy community. |
| cnu/scrapy-random-useragent |
170 |
|
0 |
0 |
over 6 years ago |
2 |
June 11, 2016 |
9 |
mit |
Python |
| Scrapy Middleware to set a random User-Agent for every Request. |
| Luqman-Ud-Din/random_user_agent |
79 |
|
1 |
30 |
almost 3 years ago |
9 |
December 26, 2018 |
3 |
mit |
Python |
| A package to get list of user agents based on filters such as operating system, software name etc.. |
| siyangbing/baiduwenku |
62 |
|
0 |
0 |
almost 5 years ago |
0 |
|
6 |
|
Python |
| 百度文库！进来吧，这是就是你想要的百度文库爬虫，必能运行！word生成可编辑的word文档，ppt和pdf生成不可编辑的ppt（排版完美）.可以实现百度文库自动化爬取，支持ppt,pdf,doc。 |
| backto17/SinaHouseCrawler |
42 |
|
0 |
0 |
about 9 years ago |
0 |
|
1 |
|
Python |
| 基于scrapy,scrapy-redis实现的一个分布式网络爬虫,爬取了新浪房产的楼盘信息及户型图片,实现了常用的爬虫功能需求. |
| ihandmine/anti-useragent |
34 |
|
0 |
8 |
over 3 years ago |
13 |
September 20, 2022 |
6 |
mpl-2.0 |
Python |
| fake pc or app browser useragent, anti useragent, and other awesome tools |
| svetlyak40wt/scrapy-useragents |
33 |
|
0 |
0 |
over 13 years ago |
0 |
|
0 |
|
Python |
| A middleware to use random user agent in Scrapy crawler. |
| WiliTest/Anonymous-scrapping-Scrapy-Tor-Privoxy-UserAgent |
20 |
|
0 |
0 |
almost 7 years ago |
0 |
|
0 |
|
Python |
| Example of how to use scrapy without getting caught (ex: changing IP) (+ info to set it on windows 10) |
| scrapedia/scrapy-useragents |
18 |
|
4 |
0 |
almost 7 years ago |
1 |
September 21, 2017 |
0 |
|
Python |
| A downloader middleware to change user-agent of scrapy |
