Data Engineering Vault
Crafted, Connected, Compounded
Welcome to my public Second Brain, a knowledge map for my notes, regarding data engineeringcurated, connected, and compounded over time.
Key Concept
I used to take a lot of scattered notes while, but I often struggled with connecting concepts across different topics. This Second Brain is my attempt to solve that: by turning disconnected notes into a structured knowledge map where concepts are linked, contextual, and continuously reinforced over time.
Data Engineering Foundations
Data Modeling, OLAP vs OLTP, Distributed System Concepts, Normalization, Slowly Changing Dimension - SCD, Star vs Snowflake Schema
Modern Data Infrastructure
Data Warehouse, Data Lake, Data Lakehouse, Data Fabric, Data Mart, Data Mesh
Data Transformation & Processing
SQL, ELT (Extract, Load, Transform), Apache Airflow, Apache Spark
Specialized Data Technologies
Data Contracts, Data Product, Change Data Capture (CDC), Snapshotting, Slowly Changing Dimension, Time Travel, ACID Transactions, Schema Evolution, Schema Drift, Software-Defined Asset, Data Integration CLI tools, Cube, VertiPaq, Idempotency