Associate Big Data Engineer

Persistent Systems Ltd, a leading global company specializing in software product development and digital transformation, is actively seeking a talented and experienced Data Engineer to join our innovative team in Pune. This is an exceptional opportunity for a professional with 2-5 years of experience who is passionate about building and optimizing robust data pipelines and architecting scalable data warehousing solutions. If you are eager to leverage your expertise in Shell Scripting, SQL, and Python to make a significant impact on complex data ecosystems, we invite you to explore this challenging and rewarding role.

Key Responsibilities

• Design, develop, and maintain robust, scalable, and efficient data pipelines using scripting languages and ETL tools to ingest, transform, and load large datasets.
• Architect, build, and optimize data warehousing solutions, ensuring high performance, reliability, and accessibility for business intelligence and analytical needs.
• Collaborate closely with data scientists, analysts, and other stakeholders to thoroughly understand data requirements and translate them into effective data solutions.
• Implement rigorous data quality checks, validation processes, and monitoring mechanisms to ensure the integrity, accuracy, and completeness of all data.
• Develop and maintain automation scripts utilizing Shell Scripting and Python for routine data processing, job orchestration, and system management tasks.
• Perform complex data transformations, aggregations, and manipulations using advanced SQL queries, stored procedures, and database functions.
• Contribute significantly to the overall architecture, design, and implementation of new data platforms, services, and features within our evolving data landscape.
• Ensure strict compliance with data governance, security policies, and industry best practices across all data initiatives and systems.

Requirements

• Bachelor's or Master's degree in Computer Science, Information Technology, Engineering, or a related quantitative field.
• A minimum of 2 to 5 years of hands-on professional experience as a Data Engineer or in a similar data-centric engineering role.
• Demonstrated proficiency in Python for data manipulation, scripting, API integration, and building data processing applications.
• Strong expertise in relational databases and advanced SQL for complex querying, data extraction, performance optimization, and database design.
• Proven experience with Shell Scripting for task automation, system administration, and managing data workflows in Linux/Unix environments.
• Solid understanding and practical experience with Data Warehousing concepts, including dimensional modeling, schema design, and ETL/ELT methodologies.
• Familiarity with cloud data platforms and services (e.g., AWS S3, Redshift, Glue; Azure Data Lake, Synapse; GCP BigQuery, Dataflow) is highly desirable.
• Experience with version control systems, particularly Git/GitHub, and agile development methodologies.
• Excellent problem-solving abilities, strong analytical skills, and the capacity to work effectively both independently and as a vital part of a collaborative team.

What We Offer

• A dynamic and stimulating work environment within Persistent Systems, a company at the forefront of technological innovation and digital transformation.
• Exceptional opportunities for continuous learning, professional development, and career advancement through challenging projects and cutting-edge technologies.
• Competitive compensation package, comprehensive benefits, and a focus on employee well-being.
• Access to mentorship programs and a culture that fosters knowledge sharing and collaboration among talented professionals.
• The chance to contribute to impactful projects that leverage vast datasets and drive significant business outcomes for global clients.
• Be part of a globally recognized organization known for its strong values, innovative spirit, and commitment to nurturing talent.