Lead Data Engineer
Accountabilities:
Design, develop, and maintain scalable data pipelines running on AWS infrastructure.
Build reliable data processing solutions using Python, Pandas, and PySpark.
Configure and manage data workflows using AWS Managed Workflows for Apache Airflow (MWAA) or Apache Airflow.
Develop and optimize queries with AWS Athena to support efficient data analysis.
Automate infrastructure provisioning and deployment processes using Terraform and GitHub Actions.
Implement testing, monitoring, troubleshooting, and data-quality controls to ensure pipeline reliability.
Participate in code reviews and promote engineering best practices across the team.
Collaborate with engineering, product, and business stakeholders in an Agile environment.
Provide technical guidance and leadership while supporting the development team's delivery and performance.
Requirements
At least 5 years of relevant professional experience.
At least 1 year of experience leading and managing development teams.
Strong Python programming skills applied to data engineering.
Practical experience using Pandas for data manipulation and analysis.
Advanced working knowledge of PySpark for large-scale distributed data processing.
Experience with AWS MWAA or Apache Airflow for workflow orchestration.
Familiarity with AWS Athena for querying and analyzing data stored in AWS.
Hands-on experience with Terraform for infrastructure automation and management.
Working knowledge of GitHub Actions and automated CI/CD pipelines.
Good understanding of DevOps principles and their application to data engineering.
Strong data engineering, troubleshooting, and complex problem-solving skills.
Upper-intermediate English proficiency or higher (B1 stated as the minimum in the source).
Nice to have
Experience using GitHub Copilot to support development, code reviews, testing, and release activities.
Knowledge of CI/CD methodologies and tools that improve software delivery.
Experience working in Agile or Scrum teams.
Background in financial services or investment management.
Experience developing Generative AI applications and integrating GenAI practices into the software development lifecycle (SDLC).
Benefits
Opportunity to contribute to international projects for leading global brands.
Collaboration with skilled and diverse teams across different locations.
Healthcare benefits.
Employee financial programs.
Paid time off and sick leave.
Upskilling, reskilling, and professional certification opportunities.
Unlimited access to LinkedIn Learning, including more than 22,000 courses.
Global career development opportunities.
Volunteer and community involvement initiatives.
Employee Groups and opportunities to connect with colleagues.
An award-winning workplace culture recognized by Glassdoor, Newsweek, and LinkedIn.
How Jobgether works
We use an AI-powered matching process to ensure your application is reviewed quickly, objectively, and fairly against the role's core requirements. Our system identifies the top-fitting candidates, and this shortlist is then shared directly with the hiring company. The final decision and next steps (interviews, assessments) are managed by their internal team.
We appreciate your interest and wish you the best!
Why Apply Through Jobgether?
Data Privacy Notice: By submitting your application, you acknowledge that Jobgether will process your personal data to evaluate your candidacy and share relevant information with the hiring employer. This processing is based on legitimate interest and pre-contractual measures under applicable data protection laws (including GDPR). You may exercise your rights (access, rectification, erasure, objection) at any time.
#LI-CL1