Junior Big Data Engineer (#5363)

We are seeking a motivated Junior Big Data Engineer to join our team. In this role, you will contribute to the development and maintenance of data pipelines, data models, and data processing solutions within modern cloud and big data environments. You will work closely with senior engineers to build scalable data solutions and support data-driven business initiatives.

The ideal candidate has hands-on experience with Python, PySpark, and SQL, a solid understanding of data engineering fundamentals, and an interest in working with large-scale data platforms and AI-powered solutions.

Key Responsibilities:

• Support the design, development, and maintenance of data pipelines and ETL/ELT processes.
• Develop and optimize data transformation workflows using Python and PySpark.
• Assist in integrating data from multiple sources while ensuring data quality, consistency, and reliability.
• Collaborate with data engineers, data scientists, and AI/ML engineers to prepare and manage datasets for analytics and machine learning applications.
• Participate in data modeling, schema design, and data warehousing activities.
• Monitor data pipelines, investigate issues, and contribute to performance improvements.
• Follow data governance, security, and privacy best practices.
• Document technical solutions, processes, and data flows.
• Stay current with emerging technologies and industry trends, incorporating innovative solutions into data engineering practices, and effectively document and communicate technical solutions and processes.

Tools and skills you will use in this role:

• Palantir Foundry
• Python
• PySpark
• SQL
• TypeScript

Requirements:

• 1+ year of experience in data engineering
• Strong proficiency in Python and SQL
• Hands-on experience with PySpark
• Familiarity with big data technologies (e.g., Apache Hadoop, Spark, Kafka, BigQuery, etc.)
• Basic experience with cloud services (e.g., AWS Glue, Azure Data Factory, Google Cloud Dataflow)
• Understanding of ETL/ELT pipelines and data integration techniques
• Experience with database systems (e.g., PostgreSQL, MySQL, NoSQL, etc.)
• Effective problem-solving and analytical skills, coupled with excellent communication and collaboration abilities
• Strong communication and teamwork abilities
• Strong mathematical, statistical, and algorithmic skills
• English - Upper-Intermediate or higher

Nice to have:

• Experience with AI/ML
• Previous experience working with JavaScript and TypeScript