Data Platform Engineer

We are seeking a skilled and proactive Data Engineer to join Vocus Data Platform team and contribute to the development, execution and maintenance of our end-to-end data pipelines.

In this role, you will collaborate with stakeholders and senior team members to implement scalable architectures across AWS and Databricks. You will focus on building robust ETL/ELT process, optimising existing pipelines and ensuring reliable data delivery across the organisation.

Key Responsibilities

• Designing, building and supporting enterprise-scale data platforms and data integration solutions in Databricks
• Advanced SQL development skills and strong proficiency in Python, PySpark, and modern data engineering frameworks
• Designing and implementing reusable, metadata-driven ingestion frameworks, as well as establishing engineering standards, reusable patterns, and platform accelerators
• Working with cloud-based data platforms including AWS and Databricks, with hands-on experience in technologies such as Redshift, Databricks, Delta Lake, S3, Glue, Lambda, and MWAA
• Dimensional modelling, data warehousing concepts, and supporting analytical data structures (such as Medallion architecture)
• Designing and implementing data quality frameworks, automated validation controls, and data observability practice
• Develop and maintain reliable batch, near real-time, and real-time ingestion pipelines leveraging metadata-driven frameworks in Databricks
• Manage workflow orchestration using AWS Airflow (MWAA), implement efficient serverless AWS Lambda and Job pipelines in Databricks.

Skills & Experience

• 5+ years of hands-on Data Engineering (Databricks Preferred)
• Strong core experience with AWS Services
• Solid hands-on experience in building job pipelines, streaming, and managing cluster configurations within the Databricks ecosystem
• Strong production-grade Python and PySpark skills
• Advanced SQL for complex querying, data manipulation, and expertise in modern data warehouse/lakehouse data modelling techniques
• Good knowledge and practical experience with CI/CD pipelines and infrastructure-as-code using Terraform.