Accountabilities

Build and maintain scalable data infrastructure: Enhance, extend, and operate batch and streaming pipelines, Delta Lake tables, and data workflows on Databricks to support growing data volumes and evolving business requirements.

Develop business-focused data products: Collaborate with business analysts, advertising strategists, and sales teams to translate their needs into reliable datasets, campaign performance reports, audience insights, and revenue analytics.

Own data modeling and metrics: Build and maintain dbt models, automated tests, curated Silver and Gold datasets, and semantic layers to ensure consistent and trusted business metrics across the organization.

Optimize performance and infrastructure costs: Improve Spark and Databricks workloads through query optimization, partitioning, cluster sizing, shuffle management, and efficient resource allocation.

Manage cloud infrastructure: Maintain and automate Databricks and AWS infrastructure using Terraform, ensuring secure, scalable, and reliable data platform operations.

Integrate multiple data sources: Connect, consolidate, and reconcile data from advertising platforms, CRM systems, external partners, and internal applications while supporting migrations from legacy reporting solutions.

Strengthen data quality and reliability: Implement monitoring, data lineage, SLAs, alerting, automated testing, and data contracts. Lead root-cause analysis and resolve issues affecting data availability and accuracy.

Establish engineering best practices: Promote effective version control, CI/CD, code reviews, testing, and documentation while mentoring engineers and contributing to the team's technical roadmap.

Ensure data governance and compliance: Apply appropriate privacy controls, security standards, and access management practices to protect sensitive information and meet GDPR, CCPA, and SOX requirements.

Advance AI-native engineering capabilities: Build tools, workflows, and AI agents that provide LLMs with additional context, helping accelerate development, debugging, code generation, and system maintenance.

Drive continuous improvement: Take ownership of foundational pipelines and data models, reduce processing times and infrastructure costs, and enable stakeholders to access trusted information independently.

Requirements

At least 5 years of professional data engineering experience, with a proven track record of building and operating production-grade data platforms at enterprise scale.

Advanced hands-on expertise in Apache Spark, including PySpark and/or Scala, with strong knowledge of partitioning, joins, shuffle behavior, caching, skew handling, and performance optimization.

Extensive experience with Databricks, including Delta Lake, Unity Catalog, Jobs and Workflows, Databricks Asset Bundles, Structured Streaming, and SQL Warehouses.

Strong knowledge of Databricks performance and cost optimization, including cluster and warehouse sizing, autoscaling, Photon, and serverless computing.

Excellent proficiency in SQL and Python or Scala for pipeline development, automation, data transformation, and query optimization.

Hands-on production experience with dbt, including model design, incremental strategies, testing, documentation, and CI/CD-driven workflows.

Practical experience with Terraform and AWS, particularly S3, IAM, and cloud networking fundamentals relevant to data platforms.

Experience with orchestration frameworks such as Airflow or Databricks Workflows, alongside Git, CI/CD, and production monitoring tools.

Strong understanding of data architecture and modeling principles, including dimensional modeling, star schemas, medallion architecture, and semantic layers.

Demonstrated ability to inherit, refactor, and improve existing data models and pipelines while maintaining reliability, scalability, maintainability, and backward compatibility.

Experience using AI/LLM coding assistants such as Claude, Cursor, or GitHub Copilot, with the ability to critically review, test, and take ownership of generated code.

How Jobgether works

We use an AI-powered matching process to ensure your application is reviewed quickly, objectively, and fairly against the role's core requirements. Our system identifies the top-fitting candidates, and this shortlist is then shared directly with the hiring company. The final decision and next steps (interviews, assessments) are managed by their internal team.

We appreciate your interest and wish you the best!

Why Apply Through Jobgether?

Data Privacy Notice: By submitting your application, you acknowledge that Jobgether will process your personal data to evaluate your candidacy and share relevant information with the hiring employer. This processing is based on legitimate interest and pre-contractual measures under applicable data protection laws (including GDPR). You may exercise your rights (access, rectification, erasure, objection) at any time.

#LI-CL1