AI Data Engineer

Prudential is actively seeking an experienced and highly skilled Data Engineer to join our dynamic team in Newark, NJ. This is a unique opportunity for a talented professional with a strong background in AWS, Python, and PySpark to design, develop, and maintain cutting-edge scalable data platforms and robust ingestion pipelines. As a key member of our data engineering group, you will contribute significantly to building and optimizing our cloud-native data ecosystem, driving critical business insights and operational efficiency.

We are looking for a proactive and hands-on Data Engineer who thrives in a hybrid work environment, collaborating closely with cross-functional teams to deliver innovative data solutions.

Key Responsibilities

• Design, build, and maintain scalable data engineering solutions primarily on the AWS cloud platform.
• Develop robust and efficient data ingestion and processing pipelines using technologies like PySpark.
• Implement and optimize data lake and data warehouse solutions, ensuring high performance, reliability, and data quality.
• Perform hands-on development, rigorous testing, and seamless deployment of complex data solutions.
• Collaborate closely with architects, developers, QA engineers, and business stakeholders to understand requirements and deliver impactful data products.
• Troubleshoot complex data issues, identify bottlenecks, and optimize system performance across the data lifecycle.
• Actively participate in architecture reviews and code reviews to uphold high engineering standards and best practices.
• Evaluate, research, and implement new data technologies and lead proof-of-concept initiatives to drive continuous innovation.

Requirements

• Strong hands-on experience with AWS services, Python, and PySpark for data engineering initiatives.
• Expertise in designing, building, and supporting AWS Data Lakes and Data Warehouses utilizing services such as S3, Glue, Athena, EMR/Spark, Redshift, RDS, and DynamoDB.
• Proven experience with ETL/ELT development and building robust data integration pipelines.
• Solid understanding and practical experience with SQL and Shell Scripting for data manipulation and automation.
• Familiarity with cloud-native architectures, CI/CD implementation, and strategies for migrating data from on-premises systems to AWS.
• Knowledge and practical experience with Kafka and Kinesis for real-time data streaming are highly preferred.

What We Offer

• Opportunity to work on cutting-edge AWS cloud technologies and large-scale data challenges in a leading financial institution.
• A collaborative and supportive team culture that values innovation, continuous learning, and professional growth.
• The chance to contribute to high-impact projects that directly influence business intelligence and strategic decision-making.
• Competitive compensation package for your expertise and valuable contributions to our team.
• A hybrid work model, fostering both independent productivity and strong team collaboration in our Newark, NJ office.