| scrapy/scrapy |
49,918 |
|
4,185 |
445 |
about 2 years ago |
96 |
September 18, 2023 |
692 |
bsd-3-clause |
Python |
| Scrapy, a fast high-level web crawling & scraping framework for Python. |
| scrapinghub/portia |
8,982 |
|
9 |
2 |
over 2 years ago |
26 |
May 25, 2015 |
127 |
bsd-3-clause |
Python |
| Visual scraping for Scrapy |
| BruceDone/awesome-crawler |
5,859 |
|
0 |
0 |
over 2 years ago |
0 |
|
27 |
mit |
|
| A collection of awesome web crawler,spider in different languages |
| scrapy/scrapely |
1,668 |
|
101 |
2 |
over 6 years ago |
13 |
November 28, 2019 |
29 |
|
HTML |
| A pure-python HTML screen-scraping library |
| istresearch/scrapy-cluster |
1,137 |
|
18 |
2 |
over 2 years ago |
15 |
December 23, 2020 |
17 |
mit |
Python |
| This Scrapy project uses Redis and Kafka to create a distributed on demand scraping cluster. |
| holgerd77/django-dynamic-scraper |
1,069 |
|
26 |
0 |
over 4 years ago |
65 |
June 25, 2021 |
38 |
bsd-3-clause |
Python |
| Creating Scrapy scrapers via the Django admin interface |
| okfn-brasil/querido-diario |
944 |
|
0 |
0 |
about 2 years ago |
0 |
|
182 |
mit |
Python |
| 📰 Diários oficiais brasileiros acessíveis a todos | 📰 Brazilian government gazettes, accessible to everyone. |
| vifreefly/kimuraframework |
874 |
|
4 |
2 |
about 4 years ago |
10 |
January 30, 2019 |
34 |
mit |
Ruby |
| Kimurai is a modern web scraping framework written in Ruby which works out of box with Headless Chromium/Firefox, PhantomJS, or simple HTTP requests and allows to scrape and interact with JavaScript rendered websites |
| scrapinghub/scrapyrt |
793 |
|
5 |
0 |
over 2 years ago |
7 |
September 20, 2023 |
31 |
bsd-3-clause |
Python |
| HTTP API for Scrapy spiders |
| MorvanZhou/easy-scraping-tutorial |
618 |
|
0 |
0 |
over 4 years ago |
0 |
|
7 |
mit |
Jupyter Notebook |
| Simple but useful Python web scraping tutorial code. |
