Databricks Data Engineer: Lakehouse Pipelines & PySpark

Job Description

• Design, build, and maintain end-to-end data pipelines for ingestion, transformation, and delivery of large‑scale data.
• Develop and optimize data processing logic using PySpark on Databricks (Apache Spark).
• Implement ETL/ELT pipelines integrating data from multiple structured and semi‑structured sources.
• Contribute to the design and implementation of lakehouse architectures (Delta Lake, Medallion architecture).
• Ensure data quality, reliability, performance, and observability across pipelines.
• Optimize Spark jobs through partitioning, caching, and performance tuning techniques.
• Collaborate with data architects, analysts, and business stakeholders to translate requirements into scalable data solutions.
• Implement best practices in CI/CD, version control, and pipeline automation.
• Support the evolution of modern data platforms and analytics capabilities.
• Work with o...