Data Engineer TASC Outsourcing

About the Role: We are seeking a skilled Data Engineer to join our Hyderabad, Telangana team. In this role, you will design, build, and maintain scalable data pipelines and architectures that support analytics and data science initiatives. You will collaborate with cross-functional teams to ensure data availability, quality, and performance across on-premise and cloud environments. Responsibilities: Design, develop, and maintain scalable ETL pipelines to ingest, process, and transform large volumes of data Optimize and tune data workflows for performance, reliability, and cost efficiency Implement and manage workflows using Apache Airflow or equivalent orchestration tools Develop Spark jobs for batch and stream processing use cases Collaborate with data scientists, analysts, and stakeholders to understand data requirements and translate them into technical solutions Ensure data quality and integrity through automated testing, validation, and monitoring Deploy and maintain data solutions on cloud platforms (AWS, GCP), leveraging services such as S3, Redshift, BigQuery, and Dataflow Document data models, pipeline architectures, and operational processes Troubleshoot and resolve production incidents, ensuring high availability and reliability Required Qualifications: 4+ years of hands-on experience in a Data Engineering role Proficiency in Python for data extraction, transformation, and scripting Strong SQL skills and experience with relational databases Proven experience designing and maintaining ETL pipelines Hands-on experience with Apache Spark for batch and streaming workflows Experience with Apache Airflow or similar orchestration frameworks Solid understanding of data warehousing principles and architectures Experience deploying and managing data solutions on AWS or GCP Bachelor’s degree in Computer Science, Engineering, or a related field Excellent communication and collaboration skills Preferred Qualifications: Experience working with both AWS and GCP cloud ecosystems Professional certification in AWS, GCP, or big data technologies Familiarity with containerization and orchestration tools such as Docker and Kubernetes Experience with real-time streaming platforms (e.g., Kafka) Knowledge of data modeling best practices and BI tools (e.g., Looker, Tableau) Experience implementing CI/CD pipelines for data workloads