Senior Data Engineer (#4752)

We are a leading European AI company developing large language models and generative platforms for enterprise and government clients.
Our products combine high-performance technologies, data security, and transparency, fully aligned with European regulatory and ethical standards.

As a Data Engineer, you will design, build, and maintain high-performance backend services and data pipelines, enabling our teams to deliver scalable, reliable, and production-ready systems.

Responsibilities:

1. ETL Data Pipelines (Batch & Streaming)

• Develop and operate ETL pipelines to extract, transform, and load data from multiple sources
• Support both batch workloads (large-scale periodic data processing) and streaming workloads (real-time or near-real-time data flows)
• Optimize performance, scalability, and reliability of data processing pipelines
• Collaborate with data engineers and analysts to ensure high-quality, clean, and accessible datasets

2. ML Data Pipelines using Temporal

• Design, implement, and maintain robust ML data pipelines for training, validation, and inference of machine learning models
• Use Temporal or similar tools for workflow orchestration, ensuring reliability, retries, and state management across complex ML workflows
• Collaborate closely with ML engineers and researchers to automate and scale model pipelines
• Ensure pipelines are reproducible, maintainable, and observable

3. Backend Services & Infrastructure

• Design, build, and maintain backend services in Go or Python 
• Work with PostgreSQL and object storage (S3) to store and manage structured and unstructured data
• Deploy and manage services using Kubernetes (K8s) and Helm
• Implement best practices in CI/CD using GitHub Actions
• Apply system design and data modeling principles, handling concurrency and performance optimization

Requirements:

• 2-3+ years of commercial experience in Go or/and proficient in Python (most of which in a production setup with real customers) 
• Strong knowledge of ETL pipeline development (batch and streaming workloads)
• Experience with Temporal or other asynchronous workflow orchestration tools
• Experience with PostgreSQL and object storage (S3)
• Familiarity with Kubernetes (K8s) and Helm
• Understanding of concurrency patterns and performance optimization in Go
• Experience building and operating ML data pipelines is highly desirable
• Strong collaboration skills and attention to detail

Nice to Have:

• Experience designing APIs/SDKs
• Experience with complex migrations or data model changes
• Knowledge of TDD, DDD, or other development best practices
• Familiarity with resiliency patterns (retries, circuit breakers)
• Experience integrating backend systems with ML models
• Experience with OpenFGA or similar tools
• Familiarity with stakeholder management and brief, concise communication

Technology Stack:

• Core Backend: Python (FastAPI), Go
• Data Storage: PostgreSQL, S3 / object storage
• Workflow Orchestration: Temporal (for ML pipelines)
• ETL: Batch and streaming pipelines
• DevOps & Infrastructure: Kubernetes (K8s), Helm, GitHub Actions
• Internal Tools: TypeScript / Nx for CLI automation (if applicable)