Accountabilities

Develop and optimize data infrastructure: Enhance, maintain, and scale existing batch and streaming pipelines, Delta Lake tables, and data workflows using Databricks.

Build business-focused data solutions: Collaborate with analysts, advertising strategists, and sales teams to transform business requirements into reliable datasets, reporting solutions, and actionable insights.

Manage data models and metrics: Develop and maintain dbt models, automated tests, semantic layers, and standardized business metrics to ensure consistency and accuracy across teams.

Optimize performance and costs: Improve Spark and Databricks workloads through query optimization, partitioning, cluster configuration, and efficient resource allocation.

Manage cloud infrastructure: Maintain data platform infrastructure using Terraform and AWS, ensuring scalable, secure, and reliable operations.

Strengthen data quality and governance: Implement monitoring, data lineage, SLAs, alerting, access controls, and data contracts while ensuring compliance with privacy and regulatory requirements.

Lead engineering excellence: Establish best practices for version control, CI/CD, testing, documentation, and code reviews while mentoring colleagues and contributing to technical roadmap decisions.

Integrate diverse data sources: Consolidate and reconcile information from advertising platforms, CRM systems, external partners, and internal applications, including migrations from legacy reporting tools.

Advance AI-native engineering: Develop tools, workflows, and AI agents that provide LLMs with relevant context and accelerate development, debugging, and system maintenance.

Drive continuous improvement: Take ownership of foundational data pipelines and models, reduce processing times and infrastructure spending, and increase self-service access to reliable business information.

Requirements

At least 5 years of professional data engineering experience, building and maintaining production-grade data platforms at enterprise scale.

Advanced hands-on expertise in Apache Spark, including PySpark or Scala, with strong knowledge of partitioning, joins, shuffle behavior, caching, skew management, and performance optimization.

Deep knowledge of Databricks, including Delta Lake, Unity Catalog, Jobs and Workflows, Databricks Asset Bundles, Structured Streaming, and SQL Warehouses.

Strong proficiency in SQL and Python or Scala for data pipeline development, automation, and query optimization.

Practical experience with dbt, including data modeling, incremental strategies, testing, documentation, and CI/CD integration.

Hands-on experience with Terraform and AWS, particularly S3, IAM, and cloud infrastructure fundamentals.

Experience with orchestration technologies such as Airflow or Databricks Workflows, Git, CI/CD pipelines, and production monitoring.

Strong understanding of data architecture and modeling principles, including dimensional modeling, star schemas, medallion architecture, and semantic layers.

Demonstrated ability to improve existing pipelines and data models while maintaining reliability, scalability, maintainability, and backward compatibility.

Experience using AI coding assistants such as Claude, Cursor, or GitHub Copilot for substantive engineering tasks, with the ability to review, test, and take full ownership of AI-generated code.

Strong analytical and problem-solving skills, with the ability to translate complex business requirements into effective technical solutions.

Excellent communication and collaboration skills, particularly when working with non-technical stakeholders and cross-functional teams.

Familiarity with data governance, privacy, and compliance standards, including GDPR, CCPA, and SOX.

Experience with business intelligence platforms such as Domo, Tableau, or Looker is an advantage.

Previous experience in advertising technology, media data, campaign performance, audience analytics, attribution, or revenue reporting is highly desirable.

A bachelor's degree in Computer Science, Engineering, or a related discipline, or equivalent professional experience.

How Jobgether works

We use an AI-powered matching process to ensure your application is reviewed quickly, objectively, and fairly against the role's core requirements. Our system identifies the top-fitting candidates, and this shortlist is then shared directly with the hiring company. The final decision and next steps (interviews, assessments) are managed by their internal team.

We appreciate your interest and wish you the best!

Why Apply Through Jobgether?

Data Privacy Notice: By submitting your application, you acknowledge that Jobgether will process your personal data to evaluate your candidacy and share relevant information with the hiring employer. This processing is based on legitimate interest and pre-contractual measures under applicable data protection laws (including GDPR). You may exercise your rights (access, rectification, erasure, objection) at any time.

#LI-CL1