| istresearch/scrapy-cluster |
1,137 |
|
18 |
2 |
over 2 years ago |
15 |
December 23, 2020 |
17 |
mit |
Python |
| This Scrapy project uses Redis and Kafka to create a distributed on demand scraping cluster. |
| damklis/DataEngineeringProject |
644 |
|
0 |
0 |
over 3 years ago |
0 |
|
4 |
mit |
Python |
| Example end to end data engineering project. |
| jikan-me/jikan-rest |
391 |
|
0 |
0 |
about 2 years ago |
0 |
|
33 |
mit |
PHP |
| The REST API for Jikan |
| sdl60660/letterboxd_recommendations |
190 |
|
0 |
0 |
about 2 years ago |
0 |
|
7 |
gpl-3.0 |
Python |
| Scraping publicly-accessible Letterboxd data and creating a movie recommendation model with it that can generate recommendations when provided with a Letterboxd username |
| zkqiang/awesome-python-primer |
78 |
|
0 |
0 |
over 3 years ago |
0 |
|
0 |
mit |
Python |
| 自学入门 Python 优质中文资源索引，包含 书籍 / 文档 / 视频，适用于 爬虫 / Web / 数据分析 / 机器学习 方向 |
| Zartenc/collyzar |
65 |
|
0 |
0 |
about 5 years ago |
2 |
January 31, 2021 |
0 |
|
Go |
| Distributed redis-based web crawler framework for colly |
| hartleybrody/scraper-boilerplate |
54 |
|
0 |
0 |
almost 5 years ago |
0 |
|
0 |
|
Python |
| Insutanto/scrapy-distributed |
40 |
|
0 |
0 |
almost 3 years ago |
8 |
February 20, 2021 |
10 |
|
Python |
| A series of distributed components for Scrapy. Including RabbitMQ-based components, Kafka-based components, and RedisBloom-based components for Scrapy. |
| tenlee2012/scrapy-kafka-redis |
35 |
|
0 |
0 |
over 5 years ago |
5 |
July 24, 2018 |
0 |
apache-2.0 |
Python |
| Distributed crawling/scraping, Kafka And Redis based components for Scrapy |
| ReedD/crawler |
32 |
|
0 |
0 |
about 6 years ago |
0 |
|
0 |
|
JavaScript |
| Chromium / Puppeteer site crawler |
