Data Engineering

Tools

• Pandera - A light-weight, flexible, and expressive data validation library for dataframes.
• polars - Lightning-fast DataFrame library for Rust and Python.
• atlas - A modern tool for managing database schemas.

Resources

• Data Engineer Roadmap
• Data Mesh Architecture
• Gently Down the Stream - A gentle introduction to Apache Kafka.
• Data Engineering Vault

Glossary

• Data Engineering Terms Explained
• Database Glossary
• Modern Data Stack Glossary

Management

• Building Modern Data Teams
• Data Team Handbook