| spotify/luigi |
17,046 |
|
338 |
76 |
about 2 years ago |
80 |
October 05, 2023 |
124 |
apache-2.0 |
Python |
| Luigi is a Python module that helps you build complex pipelines of batch jobs. It handles dependency resolution, workflow management, visualization etc. It also comes with Hadoop support built in. |
| ColZer/DigAndBuried |
645 |
|
0 |
0 |
over 9 years ago |
0 |
|
4 |
|
GCC Machine Description |
| 挖坑与填坑 |
| apache/tez |
446 |
|
0 |
0 |
about 2 years ago |
0 |
|
67 |
apache-2.0 |
Java |
| Apache Tez |
| ShifuML/shifu |
235 |
|
1 |
2 |
about 3 years ago |
9 |
April 03, 2019 |
237 |
apache-2.0 |
Java |
| An end-to-end machine learning and data mining framework on Hadoop |
| intel/graphbuilder |
90 |
|
0 |
0 |
over 11 years ago |
0 |
|
1 |
apache-2.0 |
Java |
| The GraphBuilder library provides functions to construct large scale graphs. It is implemented on Apache Hadoop. |
| smart-data-lake/smart-data-lake |
87 |
|
0 |
8 |
about 2 years ago |
26 |
October 25, 2023 |
64 |
gpl-3.0 |
Scala |
| Smart Automation Tool for building modern Data Lakes and Data Pipelines |
| bloomreach/briefly |
85 |
|
0 |
0 |
over 7 years ago |
0 |
|
2 |
apache-2.0 |
Python |
| Briefly - A Python Meta-programming Library for Job Flow Control |
| spencertipping/ni |
81 |
|
0 |
0 |
almost 3 years ago |
0 |
|
6 |
mit |
Perl |
| Say "ni" to data of any size |
| GoogleCloudPlatform/Data-Pipeline |
79 |
|
0 |
0 |
about 12 years ago |
0 |
|
2 |
apache-2.0 |
Python |
| Data pipeline is a tool to run Data loading pipelines. It is an open sourced app engine app that users can extend to suit their own needs. Out of the box it will load files from a source, transform them and then output them (output might be writing to a file or loading them into a data analysis tool). It is designed to be modular and support various sources, transformation technologies and output types. The transformations can be chained together to form complex pipelines. |
| occidere/TIL |
51 |
|
0 |
0 |
over 3 years ago |
0 |
|
173 |
gpl-3.0 |
DIGITAL Command Language |
| Today I Learned |
