Big Data Engineer (#15923770)

N-iX is looking for Big Data Engineer to join our Technology Office team.

Technology Office designs and executes industry-relevant inventions that allow organisations to realise exceptional business value from technology.  We are looking for an energetic and passionate Big Data Engineer who can bring their experience and knowledge to the company. If you want to be a part of the Technology Office department, you would like to participate in consulting: pre-sales and discoveries, to start new projects, you would like to investigate new features of top technologies, so welcome to the team.
 

Responsibilities:

• Create and maintain optimal data pipeline architectures

• Collaborate with the team to decide on which tools and strategies to use within specific data preparation scenarios

• Work with stakeholders to assist with data-related technical issues and support their data infrastructure needs
  Monitor and anticipate trends in data engineering, and propose changes in alignment with organizational goals and needs

• Share knowledge with other teams on various data engineering or project related topics

Required experience and skills:

• At least 5 or more years of experience working in the data engineering field

• Experience in building and optimization of high volume batch and real-time ETL data pipelines

• Experience with Azure (Azure Data Lake Gen 2, Azure Data Factory, Azure Stream Analytics, Azure Databricks)

• Extensive experience with Apache Flink (Flink SQL, PyFlink)

• Experience with Apache Kafka

• Experience with workflow orchestration tools (AirFlow)

• Strong knowledge of Python language and SQL

• English – Upper-Intermediate

Would be a plus:

• Experience with ML

• Experience with Java, Scala

• Experience with Docker

• Experience with Debezium CDC

• Experience with Confluent Schema Registry

• Experience with Hadoop ecosystem

• Experience with setting up from scratch Big Data processing systems

• Experience in AGILE methodologies

• Cloud certifications

• Knowledge of automation services (Terraform, Azure DevOps, AWS CodeBuild, Jenkins)

• Knowledge of visualization technologies (Microsoft PowerBI)

• Data modelling skills