Middle Data Engineer (#5706)

N-iX is looking for a Middle Data Engineer to help build and operate a new AWS-based data lakehouse for our customer in the e-commerce domain. The ideal candidate has hands-on experience with AWS data services, modern data pipelines, and lakehouse architectures.

Our Client is a global full-service e-commerce and subscription billing platform on a mission to simplify software sales everywhere. For nearly two decades, we’ve helped SaaS, digital goods, and subscription-based businesses grow by managing payments, global tax compliance, fraud prevention, and recurring revenue at scale. Our flexible, cloud-based platform combined with consultative services helps clients accelerate growth, reach new markets, and build long-term customer relationships.

Data is at the heart of everything the client does—powering insights, driving innovation, and shaping business decisions. The client is building a next-generation data platform and is looking for a Middle Data Engineer to contribute to its development.

Role Overview:

As a Middle Data Engineer, you will contribute to building and operating a scalable and cost-efficient AWS data lakehouse. You will work closely with data engineers, DevOps engineers, architects, and product leads to implement reliable data pipelines, data models, and data products that support business decision-making.

Your Responsibilities:

• Develop and maintain scalable data pipelines using S3, Apache Iceberg, Glue Data Catalog, EMR, Airflow (MWAA), Python, and PySpark.
• Work with data across medallion layers, from source ingestion to reusable business entities, dimensional models, and analytics-ready datasets.
• Integrate data from various source systems, including databases, APIs, business applications, and file-based sources.
• Implement data transformations and contribute to building end-to-end data products, including scheduled data deliveries and serverless workloads using services such as AWS Lambda and S3.
• Apply automated testing and data quality practices across data pipelines and contribute to metadata management, cataloguing, lineage, and documentation using DataHub and AWS services.
• Prepare and make curated data available in Snowflake for analytics, reporting, and SQL-based use cases.
• Collaborate with architects, DevOps engineers, product owners, engineering managers, and Power BI engineers on solution implementation, CI/CD, and Terraform-based infrastructure.
• Monitor data pipelines and production workflows, investigate issues, improve performance and reliability, and maintain technical documentation and runbooks.

What We’re Looking For:

• 3+ years of experience in data engineering, ideally with AWS, lakehouse, or hybrid data platforms.
• Strong Python, PySpark, and SQL skills, with experience developing production data pipelines and data transformations.
• Practical experience with Airflow and AWS data services, including S3, Apache Iceberg, Glue Data Catalog, Glue Data Quality, Athena, and EMR.
• Good understanding of data modelling and harmonisation, including dimensional modelling, SCD Type 1 and 2, schema evolution, and working with data from multiple source systems.
• Experience integrating data from databases, APIs, business applications, and file-based sources.
• Understanding of data governance concepts, including metadata management, cataloguing, lineage, and documentation. Experience with DataHub is a plus.
• Familiarity with Snowflake, BI platforms, semantic layers, and SQL- or dbt-based transformation approaches.
• Experience with Git, GitLab CI/CD, automated testing, and Infrastructure as Code. Terraform experience is a plus.
• Good problem-solving and troubleshooting skills, with a proactive approach to learning and taking ownership of assigned tasks.
• Good communication and documentation skills.
• Upper-Intermediate English or higher to collaborate effectively with English-speaking stakeholders on the client side.