Senior Data Engineer - PySpark & SQL

Role: Senior Data Engineer - PySpark & SQL

Relevant Experience Required: 5 Years

Job Description

Data Transformation

• Query, clean, and transform datasets using SQL and Python on the Databricks platform.
• Ensure data quality and consistency across systems in accordance with government Instruction Manual (IM8) standards.

Analytics & Dashboard Development

• Build, enhance, and maintain dashboards for data monitoring , performance reporting , and user behaviour analytics .
• Partner with business stakeholders to understand data requirements and translate them into impactful visual insights.

Documentation & Knowledge Management

• Develop clear and comprehensive user guides for dashboards, charts, and data workflows in Databricks
• Maintain technical documentation (e.g., on Confluence) for analyti...