Senior Data Engineer (#4260)

N-iX is expanding in Latin America! As a global software development company, we unite over 2,400 professionals across Europe and the Americas, delivering innovative solutions for industry-leading enterprises and Fortune 500 companies. Recognized as a Great Place to Work in Colombia, we invite you to join our growing team, enjoy flexible work options, collaborate on international projects, and thrive in a vibrant, inclusive tech community.

Key Responsibilities:

• Design, build, and maintain robust data pipelines within Databricks.
• Collaborate closely with international teams, including data scientists and architects, to develop scalable data solutions.
• Debug complex issues in data pipelines and proactively enhance system performance and reliability.
• Set up Databricks environments on cloud platforms (Azure/AWS).
• Automate processes using CI/CD practices and infrastructure tools such as Terraform.
• Create and maintain detailed documentation, including workflows and operational checklists.
• Develop integration and unit tests to ensure data quality and reliability.
• Migrate legacy data systems to Databricks, ensuring minimal disruption.
• Participate actively in defining data governance and management strategies.

Qualifications:

• 5+ years of proven experience as a Data Engineer.
• Advanced proficiency in Python for developing production-grade data pipelines.
• Extensive hands-on experience with Databricks platform.
• Strong knowledge of Apache Spark for big data processing.
• Familiarity with cloud environments, specifically Azure or AWS.
• Proficiency with SQL and experience managing relational databases (MS SQL preferred).
• Practical experience with Airflow or similar data orchestration tools.
• Strong understanding of CI/CD pipelines and experience with tools like GitLab.
• Solid skills in debugging complex data pipeline issues.
• Proficiency in structured documentation practices.
• B2 level or higher proficiency in English.
• Strong collaboration skills, ability to adapt, and eagerness to learn in an international team environment.

Would be a plus:

• Experience with Docker and Kubernetes.
• Familiarity with Elasticsearch or other vector databases.
• Understanding of DBT (data build tool).
• Ability to travel abroad twice a year for on-site workshops.