Senior Data Engineer

Martinettechnologies is urgently seeking an experienced and highly skilled Senior Data Engineer to join our dynamic team in Hyderabad for a 1-year contract. If you possess a robust background in Python and distributed data processing, and thrive in a high-volume production data environment, this is an excellent opportunity to contribute to cutting-edge data architectures. As a Senior Data Engineer, you will play a pivotal role in designing, building, and maintaining our next-generation data platforms, ensuring scalability and reliability.

We are looking for immediate joiners or candidates serving notice periods who can contribute effectively with minimal onboarding. This role is crucial for supporting our high-volume production data environments and advancing our data capabilities.

Key Responsibilities

• Design, develop, and optimize robust ETL/ELT pipelines using advanced Python programming and distributed processing frameworks like Apache Spark or Flink.
• Implement and manage data warehousing solutions and data lake/lakehouse architectures, leveraging technologies such as Redshift, BigQuery, Databricks, and Snowflake.
• Orchestrate complex data workflows using tools like Airflow, Dagster, or Prefect to ensure efficient and reliable data processing.
• Work with real-time streaming platforms such as Kafka or Kinesis to process high-volume, low-latency data streams effectively.
• Manage and optimize various data storage solutions including PostgreSQL, DynamoDB, Cassandra, and other NoSQL databases.
• Utilize Infrastructure as Code (Terraform, CloudFormation) for provisioning and managing cloud resources on AWS and GCP.
• Implement monitoring and alerting solutions using Prometheus, Grafana, CloudWatch, and Datadog to ensure data pipeline health and performance.
• Champion data quality, data governance, and best practices in data modeling and architectural design across all data initiatives.

Requirements

• 8-9 years of extensive experience in Data Engineering, specifically supporting high-volume production data environments.
• Exceptional proficiency in advanced Python programming is mandatory for this role.
• Demonstrated expertise with distributed data processing frameworks such as Apache Spark or Apache Flink.
• Solid experience with workflow orchestration tools like Airflow, Dagster, or Prefect.
• Hands-on experience with cloud data services on AWS or GCP, including S3, Redshift, BigQuery, Databricks, and Snowflake.
• Proficiency in real-time streaming platforms like Kafka or Kinesis is essential.
• Strong command of advanced SQL and experience with various database technologies including PostgreSQL, DynamoDB, and Cassandra.
• Familiarity with containerization (Docker, Kubernetes) and Infrastructure as Code (Terraform, CloudFormation).
• Deep understanding of Data Lake/Lakehouse architectures and data formats such as Parquet, Avro, Delta Lake, or Iceberg.
• Proficiency with Git and CI/CD pipelines for continuous integration and deployment.

What We Offer

• Opportunity to work on challenging and impactful data engineering projects within a fast-paced and innovative environment.
• Be part of a dedicated team that values innovation, scalable data architectures, and robust solutions.
• A stimulating work environment where your contributions as a Senior Data Engineer will directly impact critical business functions.
• Engage with cutting-edge technologies and best practices in the data engineering domain.
• A 1-year contract opportunity in the vibrant city of Hyderabad, with a mandatory work-from-office setup.