Data Engineer (Databricks + Fabric)
Accountabilities:
- Design, develop, deploy, and maintain production-grade data platforms for clients, taking hands-on responsibility for implementation and ongoing operations.
- Build and optimize scalable ETL/ELT pipelines using Databricks and Microsoft Fabric to support complex data integration and transformation requirements.
- Design and implement modern lakehouse architectures based on the Medallion framework, organizing data into Bronze, Silver, and Gold layers using Delta Lake.
- Develop and maintain data processing solutions using PySpark, Python, SQL, Databricks Workflows, and Microsoft Fabric components.
- Implement and manage data governance through Unity Catalog, including permissions, access controls, data lineage, and data quality practices.
- Work across the Microsoft Fabric ecosystem, leveraging Lakehouse, OneLake, Data Pipelines, Notebooks, Dataflows Gen2, and Warehouse capabilities.
- Establish and automate deployment processes using CI/CD pipelines and Infrastructure as Code (IaC) practices.
- Ensure data platforms are scalable, secure, reliable, and optimized for performance and maintainability.
- Gather, analyze, and translate business and technical requirements in collaboration with clients and internal teams.
- Identify opportunities to improve existing data architectures, infrastructure, and engineering workflows.
- Proactively recognize potential client needs and recommend technical enhancements that may lead to additional projects and engagements.
- Collaborate with cross-functional teams to ensure successful project delivery and alignment with business objectives.
Requirements
- Proven hands-on professional experience implementing and managing Databricks solutions in production environments. Proofs of concept or theoretical knowledge alone are not sufficient.
- Strong experience with PySpark, Delta Lake, Databricks Workflows, and Unity Catalog, including catalog management, permissions, and data lineage.
- Demonstrated experience designing and implementing Medallion architectures (Bronze, Silver, and Gold) in production data environments.
- Proven hands-on experience with Microsoft Fabric across its full platform, including Lakehouse, OneLake, Data Pipelines, Notebooks, Dataflows Gen2, and Warehouse. Experience limited to proofs of concept is not sufficient.
- Advanced proficiency in SQL and Python, with the ability to develop efficient data transformation and processing solutions.
- Strong understanding of modern data engineering principles, lakehouse architectures, data governance, and production data platform management.
- Experience building and maintaining automated deployment workflows using CI/CD and Infrastructure as Code.
- Current Microsoft DP-900 or DP-700 certification, with a commitment to obtain the remaining certification within the first 60 days of employment. Certification status should be verifiable through Microsoft Learn.
- Professional English proficiency at B2 level or above, with the ability to communicate effectively with international clients and technical teams.
- Fluent Spanish, with the ability to participate confidently in professional discussions and client interactions.
- Strong analytical and problem-solving skills, combined with a practical, hands-on approach to technical implementation.
- Excellent communication and collaboration skills, with the ability to gather requirements, understand client needs, and translate them into effective technical solutions.
- Proactive mindset, attention to detail, and the ability to identify opportunities for continuous improvement.
Benefits
- Fully remote work: Work remotely from Brazil, with a location-independent working environment.
- International exposure: Collaborate with clients and technical teams across Latin America and potentially other international markets.
- Advanced technology stack: Gain hands-on experience with Databricks, Microsoft Fabric, Delta Lake, PySpark, and modern cloud data engineering technologies.
- Technical ownership: Take direct responsibility for building, deploying, and operating production data platforms.
How Jobgether works
We use an AI-powered matching process to ensure your application is reviewed quickly, objectively, and fairly against the role's core requirements. Our system identifies the top-fitting candidates, and this shortlist is then shared directly with the hiring company. The final decision and next steps (interviews, assessments) are managed by their internal team.
We appreciate your interest and wish you the best!
Why Apply Through Jobgether?
Data Privacy Notice: By submitting your application, you acknowledge that Jobgether will process your personal data to evaluate your candidacy and share relevant information with the hiring employer. This processing is based on legitimate interest and pre-contractual measures under applicable data protection laws (including GDPR). You may exercise your rights (access, rectification, erasure, objection) at any time.
#LI-CL1