Senior Data Engineer
- Published on
About the Role
As a Senior Data Engineer at Brooksource, you will lead the design, development, and implementation of scalable data pipelines and ELT processes using tools such as Databricks, DLT, and Airflow. You will collaborate with stakeholders to understand data requirements and deliver top-notch solutions. Your role will also encompass optimizing and maintaining existing data pipelines to ensure data quality and reliability.
Responsibilities
- Lead the design, development, and implementation of scalable data pipelines and ELT processes using Databricks, DLT, dbt, Airflow, and other tools.
- Collaborate with stakeholders to understand data requirements and deliver high-quality data solutions.
- Optimize and maintain existing data pipelines to ensure data quality, reliability, and performance.
- Develop and enforce data engineering best practices, including coding standards, testing, and documentation.
- Mentor junior data engineers, fostering a culture of continuous learning and technical leadership.
- Monitor and troubleshoot data pipeline issues, ensuring timely resolution.
About the Candidate (expectations and nice to have skills)
The ideal candidate will have:
- 8+ years in data engineering/architecture focused on big data technologies.
- Extensive experience with Databricks and Apache Spark.
- Proficiency in SQL and programming languages like Python, Scala, or Java.
- Strong understanding of cloud platforms (Azure, AWS, Google Cloud) and data warehousing.
- Preferred: Experience with machine learning workflows, knowledge of data governance, and relevant certifications (Databricks, Azure, Google Cloud).
About the Company
Brooksource is committed to creating a high-quality work environment where every team member can thrive and achieve their full potential. We support continual learning and improvement alongside a collaborative team spirit.
Company Culture and Benefits
At Brooksource, you will find a culture that values continuous learning, mentorship, and collaboration. We offer competitive salary packages and benefits to ensure our employees feel valued, including opportunities for professional growth and development.