Processing huge tables with pandas and multiprocessing
A practical pattern for splitting a giant table into chunks and using every CPU core without running out of memory.
The blog
Data engineering blog: Notes from the pipeline.
Daily writing on data engineering: what worked, what broke and what I’d do differently.
Small beginnings: why I'm writing this blog
From intern to full-time data engineer in nine months, and why I'm going to document the journey one post at a time.