Skip to content Skip to sidebar Skip to footer
Showing posts with the label Luigi

Passing Python Objects Between Tasks In Luigi?

I was coding my first project in Python 3.6 using Spotify's Luigi to arrange some Natural Langu… Read more Passing Python Objects Between Tasks In Luigi?

How To Continously Update Target File Using Luigi?

I have recently started playing around with Luigi, and I would like to find out how to use it to co… Read more How To Continously Update Target File Using Luigi?

Luigi Pipeline Beginning In S3

My initial files are in AWS S3. Could someone point me how I need to setup this in a Luigi Task? I … Read more Luigi Pipeline Beginning In S3

Luigi: How To Pass Different Arguments To Leaf Tasks?

This is my second attempt at understanding how to pass arguments to dependencies in Luigi. The firs… Read more Luigi: How To Pass Different Arguments To Leaf Tasks?

Recurrent Machine Learning Etl Using Luigi

Today, running the machine learning job I've written is done by hand. I download the needed inp… Read more Recurrent Machine Learning Etl Using Luigi

When A New File Arrives In S3, Trigger Luigi Task

I have a bucket with new objects getting added at random intervals with keys based on their time of… Read more When A New File Arrives In S3, Trigger Luigi Task