Accountabilities:

Design, develop, and maintain batch and streaming data ingestion, transformation, and quality pipelines using Google Cloud services such as BigQuery, Dataflow, Dataproc, Pub/Sub, and Cloud Storage.
Build and organize scalable Data Lake and Lakehouse architectures using BigQuery and BigLake, implementing appropriate data layers, partitioning strategies, and performance and cost optimizations.
Define and maintain data contracts, including schemas, data freshness requirements, and service-level agreements (SLAs), while monitoring data quality, consistency, and lineage.
Prepare and optimize datasets for AI applications through data cleaning, anonymization, chunking, and embedding generation to support Vertex AI RAG Engine, Vertex AI Search, and vector databases.
Ensure compliance with Brazil's General Data Protection Law (LGPD) when processing sensitive healthcare information, implementing masking, anonymization, Sensitive Data Protection, policy tags, and data residency controls before information reaches AI environments.
Design secure cross-project data access architectures using Cloud IAM, Dataplex, BigLake, and Analytics Hub, minimizing unnecessary data duplication while enforcing least-privilege access and auditability.
Enable internal systems and data sources to be consumed as tools by AI agents through MCP servers and MCP Toolbox, collaborating with AI teams on integrations with Vertex AI Agent Engine and Agent Development Kit (ADK).
Develop and maintain MLOps and LLMOps pipelines using Vertex AI Pipelines, incorporating model and response evaluations, security guardrails through Model Armor, and monitoring for data drift and response quality.
Provision and manage data and AI infrastructure as code using Terraform and CI/CD practices, ensuring consistent, automated, and scalable deployments.
Implement FinOps practices by monitoring and allocating infrastructure, project, and model costs through BigQuery billing exports and resource labels.
Establish end-to-end observability using OpenTelemetry, Cloud Logging, Cloud Monitoring, and Cloud Trace to monitor system health, performance, and reliability.
Contribute to resilience and disaster recovery strategies, ensuring the availability and continuity of critical data and AI services.
Collaborate with data scientists, AI engineering teams, business stakeholders, and external partners to communicate technical decisions, document architectures, and establish clear ownership of pipelines, data sources, and tools.

Requirements

Strong proficiency in Python and SQL, with the ability to develop efficient, scalable, and maintainable data engineering solutions.
Solid hands-on experience with Google Cloud Platform (GCP) data engineering services, particularly BigQuery, Cloud Storage, Dataflow or Dataproc, and Pub/Sub.
Experience designing, developing, and orchestrating data pipelines using Cloud Composer/Airflow, Cloud Workflows, or similar orchestration tools.
Knowledge of GCP data governance and security services, including Cloud IAM, Dataplex, policy tags, and Cloud KMS.
Practical experience with Terraform and CI/CD tools such as Cloud Build, GitHub Actions, or equivalent platforms.
Experience working with GCP environments involving multiple projects, organizations, and resource folders.
Strong understanding of distributed and event-driven architectures, including messaging systems, retries, idempotency, and data consistency.
Practical experience preparing and managing data for at least one production-level LLM application, such as Retrieval-Augmented Generation (RAG) or AI agents with tool-calling capabilities.
Knowledge of LGPD requirements for sensitive data processing, including data anonymization, masking, and privacy protection techniques.
Strong communication skills, with the ability to collaborate effectively with data scientists, AI engineers, business teams, and external stakeholders.
High standards of security, responsibility, and ethical judgment when handling sensitive information, particularly patient data accessed by autonomous AI agents.

How Jobgether works

We use an AI-powered matching process to ensure your application is reviewed quickly, objectively, and fairly against the role's core requirements. Our system identifies the top-fitting candidates, and this shortlist is then shared directly with the hiring company. The final decision and next steps (interviews, assessments) are managed by their internal team.

We appreciate your interest and wish you the best!

Why Apply Through Jobgether?

Data Privacy Notice: By submitting your application, you acknowledge that Jobgether will process your personal data to evaluate your candidacy and share relevant information with the hiring employer. This processing is based on legitimate interest and pre-contractual measures under applicable data protection laws (including GDPR). You may exercise your rights (access, rectification, erasure, objection) at any time.

#LI-CL1