Senior Azure Data Engineer

Are you an experienced and highly skilled Azure Data Engineer looking for your next challenge? Albireo Recruiters is seeking talented professionals for multiple full-time openings in Pune. This is an excellent opportunity to leverage your expertise in Azure Data Factory, Azure Databricks, and PySpark to design and build robust data solutions for a leading client.

As an Azure Data Engineer, you will play a crucial role in developing and optimizing scalable data pipelines, contributing to cutting-edge data initiatives, and shaping the future of data architecture.

Key Responsibilities

• Design, develop, and maintain high-performance, scalable data pipelines using Azure Data Factory (ADF) and Azure Databricks (ADB).
• Implement efficient ETL/ELT processes to ingest, transform, and load large datasets into Azure Data Lake Storage (ADLS) and Azure Synapse Analytics.
• Write complex and optimized PySpark and SQL queries for data processing, analysis, and transformation within the Azure ecosystem.
• Develop and manage robust data lake and lakehouse architectures, ensuring data quality, security, and governance.
• Collaborate closely with data scientists, analysts, and other engineering teams to understand data requirements and translate them into technical specifications.
• Perform comprehensive performance tuning and optimization of existing data solutions and infrastructure to enhance efficiency and reliability.
• Ensure adherence to best practices in data engineering, including version control, automated testing, and continuous integration/continuous deployment (CI/CD) pipelines for data solutions.

Requirements

• Bachelor's or Master's degree in Computer Science, Information Technology, Engineering, or a related quantitative field.
• A minimum of 4 to 10 years of professional hands-on experience as an Azure Data Engineer or in a similar data-focused role.
• Demonstrable expertise with Microsoft Azure data services, including Azure Data Factory (ADF), Azure Databricks (ADB), Azure Data Lake Storage (ADLS), and Azure Synapse Analytics.
• Strong proficiency in programming languages such as Python and Scala, particularly for data manipulation and processing.
• Advanced SQL writing skills for complex data querying, analysis, and database management.
• Solid understanding and practical experience with PySpark or Apache Spark for big data processing.
• Experience with ETL/ELT methodologies and developing end-to-end data pipelines from various sources.
• Familiarity with data lake and lakehouse architectural principles and best practices.

What We Offer

• An exciting opportunity to work on challenging and impactful data engineering projects within the Azure cloud environment.
• A collaborative and innovative work culture that fosters continuous learning and professional development.
• Exposure to a diverse range of industries and complex data challenges, enhancing your skill set as an Azure Data Engineer.
• A competitive compensation package commensurate with your experience and expertise.
• The chance to be part of a forward-thinking team committed to leveraging cutting-edge data technologies.