Senior Databricks Data Engineer

Bristlecone is actively seeking a highly skilled and experienced Senior Databricks Data Engineer to join our dynamic team. This is an exceptional opportunity for a seasoned professional with extensive expertise in developing and optimizing data solutions within the Databricks ecosystem. As a Senior Databricks Data Engineer, you will be instrumental in designing, building, and maintaining robust, scalable data pipelines that leverage cutting-edge technologies like PySpark and Delta Lake to drive critical business insights across our Pan-India operations. We are looking for an immediate joiner who can hit the ground running, bringing their leadership and technical prowess to complex data challenges.

In this pivotal role, you will lead the charge in transforming raw data into actionable intelligence, working closely with various stakeholders to deliver high-quality data products. If you are passionate about data, possess a deep understanding of cloud data architectures, and thrive in an environment that values innovation and collaboration, we encourage you to apply. This position offers the flexibility of remote work, allowing you to contribute from anywhere in India while being an integral part of our globally recognized team.

Key Responsibilities

• Design, develop, and implement scalable, optimized data pipelines and ETL/ELT processes using Databricks, PySpark, and Delta Lake.
• Architect and manage data solutions within cloud environments (AWS, Azure, GCP) ensuring data integrity, security, and performance.
• Collaborate with data scientists, analysts, and business stakeholders to understand data requirements and translate them into technical specifications.
• Optimize existing data workloads and data models for improved efficiency, cost-effectiveness, and reliability.
• Implement robust data governance, data quality, and data cataloging strategies.
• Lead troubleshooting efforts for production data issues and develop proactive monitoring and alerting systems.
• Mentor junior data engineers, promote best practices, and contribute to the overall technical growth of the team.
• Stay abreast of emerging data engineering technologies and methodologies, evaluating their potential application within our data landscape.

Requirements

• Minimum of 6 years and a maximum of 12 years of hands-on experience in data engineering, with a strong focus on Databricks.
• Proven expertise in developing and optimizing data pipelines using PySpark and Delta Lake.
• Solid understanding and practical experience with cloud data platforms such as AWS, Azure, or GCP.
• Proficiency in SQL for data manipulation, querying, and analysis.
• Strong grasp of data warehousing concepts, data modeling, and ETL/ELT principles.
• Experience with version control systems (e.g., Git) and CI/CD pipelines.
• Excellent problem-solving skills, with an ability to analyze complex data challenges and propose effective solutions.
• Immediate availability to join the team is a strict requirement.

What We Offer

• An opportunity to work on challenging and impactful data projects leveraging cutting-edge technologies.
• A collaborative, inclusive, and supportive work environment that fosters innovation.
• Significant opportunities for professional growth, learning, and skill development.
• The flexibility of a remote, Pan-India work model.
• Being part of a leading global consulting and technology services company with a focus on supply chain transformation.