Senior Data Engineer (part-time) (#1131)

N-iX is looking for Data Engineer to join our team.

Responsibilities:

• Design, develop, and maintain real-time and batch data pipelines 
• Collaborate with cross-functional teams, including data scientists and software engineers, to decide on which tools and strategies to use within specific data preparation scenarios
• Implement data quality and validation processes to ensure accuracy and reliability of data used in decision-making.
• Optimise data infrastructure for performance and efficiency, considering scalability and future growth.
• Work with stakeholders to assist with data-related technical issues and support their data infrastructure needs.

Requirements:

• Experience in building real-time data pipelines from scratch with stateful computations  (Spark streaming/Storm/DataFlow/Kafka Streams/Amazon Kinesis)
• Experience with Big Data tool stack (Hadoop/Hive/Kafka)
• Experience with Apache Flink APIs (Flink SQL, PyFlink); 
• Experience with workflow orchestration tools (AirFlow);
• Strong knowledge of Python language and SQL.

Would be a plus:

• Experience with ML;
• Experience with Java/Scala;
• Experience with Docker;
• Experience with Debezium CDC;
• Experience with Confluent Schema Registry