Lead Data Software Engineer (Databricks)
- Published on
About the Role
We are looking for a Lead Data Software Engineer with experience and knowledge in Databricks and an open-minded personality. You will join our friendly environment and become a core contributor to our team of experts. You will lead a team of engineers in creating and maintaining scalable data pipelines and architectures, as well as developing and optimizing data models.
Responsibilities
- Lead a small team of engineers in creating and maintaining scalable data pipelines and architectures.
- Develop and optimize data models and ETL processes utilizing Databricks alongside other technologies.
- Implement data quality checks and monitoring frameworks to ensure high data integrity.
- Stay abreast with emerging data engineering trends and technologies, advocating for the adoption of new tools when beneficial.
- Troubleshoot and resolve data-related issues efficiently.
- Conduct code reviews and uphold high standards of code quality.
- Foster a collaborative and inclusive environment for the team to grow their skills and expertise.
- Interact with clients, managing expectations and delivering high-quality solutions that meet their needs.
- Drive innovation by proposing new data strategies and enhancements.
- Work collaboratively within an Agile framework to meet project timelines and deliverables.
About the Candidate
To qualify for the role, you should possess:
- Hands-on experience with Databricks, including Delta Lake, workflows, Delta Live Tables, deployment, and versioning.
- Capability to lead a development team of 2-5 engineers.
- Solid understanding of data architectures and proficient in data modeling, building, and optimizing ETL pipelines.
- Experience in Spark with Scala or PySpark.
- Knowledge of the software development lifecycle and Agile frameworks.
- Proficiency in cloud-native technologies and familiarity with engineering best practices, including containers, unit tests, and code style checks.
- Engineering experience in cloud environments with AWS, Azure, or GCP.
- Focus on big data and performance optimization of data-intensive applications.
- A proactive approach with the ability to operate independently.
- Excellent client-facing skills with the ability to handle ambiguous situations effectively.
- Fluent English communication skills at a B2+ level.
About the Company
EPAM is a leading digital transformation services and product engineering company with over 52,650 employees across more than 55 countries. Since 1993, our multidisciplinary teams have been assisting clients and communities to make the future real. In 2018, we opened an office in Spain, quickly growing it to over 1,450 employees distributed between Málaga, Madrid, and remotely across the country.
Company Culture and Benefits
At EPAM, we value work-life balance, offering flexible work options, 24 working days of annual leave, and paid time off for public holidays. We foster a continuous learning culture where you can create a personalized Career Development Plan, benefit from internal training, mentorship, and sponsored certifications. We also provide clear career paths to grow into management or technical specialist roles. Join our strong professional community, connect with skilled experts, solve challenges, and make friends.