We are seeking a proactive Senior Data Engineer to join our vibrant team. As a Senior Data Engineer, you will play a critical role in designing, developing, and maintaining sophisticated data pipelines, Ontology Objects, and Foundry Functions within Palantir Foundry. The ideal candidate will possess a robust background in cloud technologies, data architecture, and a passion for solving complex data challenges.
Technical stack: Palantir Foundry, Python, PySpark, SQL, TypeScript.
Responsibilities:
Collaborate with cross-functional teams to understand data requirements, and design, implement, and maintain scalable data pipelines in Palantir Foundry, ensuring end-to-end data integrity and optimizing workflows.
Gather and translate data requirements into robust and efficient solutions, leveraging your expertise in cloud-based data engineering. Create data models, schemas, and flow diagrams to guide the development process.
Develop, implement, optimize, and maintain efficient and reliable data pipelines and ETL/ELT processes to collect, process, and integrate data to ensure timely and accurate data delivery to various business applications, while implementing data governance and security best practices to safeguard sensitive information.
Monitor data pipeline performance, identify bottlenecks, and implement improvements to optimize data processing speed and reduce latency.
Assist in optimizing data pipelines to improve machine learning workflows.
Troubleshoot and resolve issues related to data pipelines, ensuring continuous data availability and reliability to support data-driven decision-making processes.
Stay current with emerging technologies and industry trends, incorporating innovative solutions into data engineering practices, and effectively document and communicate technical solutions and processes.
Requirements:
Nice to have:
We offer*:
*not applicable for freelancers
We’ll drop you a note when there are new jobs that match your search.