Data Engineering

Tools

• Pandera - A light-weight, flexible, and expressive data validation library for dataframes.
• polars - Lightning-fast DataFrame library for Rust and Python.
  • polars-st - Spatial extension for Polars DataFrames
• atlas - A modern tool for managing database schemas.
• kafka-ui - Open-Source Web UI for managing Apache Kafka clusters

Resources

• Data Engineer Roadmap
• Data Mesh Architecture
• Gently Down the Stream - A gentle introduction to Apache Kafka.
• Data Engineering Vault

Glossary

• Data Engineering Terms Explained
• Database Glossary
• Modern Data Stack Glossary

Management

• Building Modern Data Teams
• Enterprise Data Team