About the customer

Our Customer is a rapidly growing B2B SaaS analytics company utilizing advanced machine learning and AI technologies to deliver actionable insights to both enterprise and SME customers.

Their cloud-based platform integrates datasets, executes complex ML workflows, and provides intuitive analytics dashboards that drive strategic growth decisions.

About the team

You will be working closely with their Machine Learning, Frontend Engineers, as well as their CTO and Product Owner.

They are a cross-functional team that focuses on efficient communication.

Assignment description

Role Overview

As a Senior Data Engineer, you’ll work with and enhance their end-to-end ML infrastructure on Google Cloud Platform (GCP). Collaborating closely with ML Engineers, Frontend Engineer, and Cloud Architect, you’ll build scalable, secure, and highly available backend systems powering their analytics platform. You’ll also have opportunities to leverage and implement GenAI solutions, influence system architecture, and directly impact core product capabilities. It is a small team so you will have a key role in the organization.

Key Responsibilities

• Data Pipeline Architecture: Design, build, and optimize scalable data pipelines using GCP services (BigQuery, Dataflow, Dataform) to manage large-scale structured and unstructured data.
• Backend ML Services: Develop and maintain backend services for ML model training, deployment, inference, and integration of generative AI models.
• API Development: Create secure, high-performance APIs to seamlessly serve data-driven insights to frontend applications.
• Serverless & Containerization: Utilize Cloud Functions, Cloud Run, and Kubernetes (GKE) to enable rapid, scalable, and cost-efficient deployment processes.
• MLOps Practices: Champion continuous integration and deployment (CI/CD), implement rigorous testing frameworks, and oversee data and model versioning and monitoring to ensure reliability and high quality.
• Technical Leadership: Guide architecture decisions, perform thorough design reviews, and drive the adoption of innovative GenAI solutions.
• Cross-Functional Collaboration: Work closely with ML Engineers, Frontend Engineers, and CTO to align requirements and deliver cohesive product features.

Must haves

• Backend Experience: 5+ years designing and building production-grade backend systems using Python or similar languages.
• Strong Architecture Skills: Proven expertise in microservices, distributed systems, and serverless architectures, emphasizing scalability, security, and reliability.
• GCP Expertise: Deep experience with GCP services including BigQuery, Dataflow, Dataform, Cloud Functions, and Cloud Run, along with Terraform or similar infrastructure-as-code tools.
• Data Pipeline Expertise: Proven ability designing robust ETL/ELT data pipelines; familiarity with orchestration tools like Airflow or Kestra is a plus.
• API Development: Experienced in crafting well-documented, performant APIs.
• DevOps Knowledge: Experienced with Docker, Kubernetes, automated testing, and CI/CD best practices.
• Communication & Collaboration: Excellent communication skills, capable of collaborating effectively across diverse teams.
• Ownership: Autonomous and driven, capable of managing complete project lifecycles in a dynamic environment.

PIG PLUSS

• Understanding of ML workflows, model lifecycle management, and experience with frameworks such as TensorFlow, PyTorch, Scikit-learn, and generative AI technologies (e.g., LLM models).
• Familiarity with frontend integration (React, TypeScript).
• Expertise in advanced monitoring and observability tools (Stackdriver, Grafana, Prometheus).

Other requirements

• Collaborative Mindset: You thrive in a team environment, value diverse perspectives, and actively contribute to shared goals.
• Strong Communicator: You have excellent verbal and written communication skills, making it easy for you to convey ideas clearly and effectively.
• Adaptable & Agile: You’re comfortable in fast-paced environments and can quickly adjust to changing priorities and challenges.
• Proactive & Self-Driven: You take initiative, work independently when needed, and don’t wait for instructions to make things happen.
• Solution-Oriented Thinker: You approach challenges with a problem-solving mindset, always looking for innovative and effective solutions.