Data Engineer

Confidential Company

Data_Engineer (#Digital_Transformation)

Minimum Qualifications

• Master of Science in Data Analytics or Bachelor’s in Computer Science/Engineering

• Certified Data Engineer with IBM/Google Cloud/Azure

• AWS Certified Solutions Architect

• Databricks Certified Data Engineer Associate

Experience (Minimum 10+ Years)

• Data Architectures

• Big Data Analytics

• ETL Processes

• Data Governance and Security

Job Responsibilities

• Design and implement efficient database structures

• Develop scalable data pipelines from multiple sources such as rigs, sensors, and production facilities

• Implement ETL processes to ensure data quality and integrity

• Manage SQL for data retrieval and Python/R for data analysis

• Handle big data with Hadoop and Spark

• Use data warehousing solutions like Redshift, BigQuery, or Snowflake

• Build machine learning pipelines and workflow tools such as Airflow or Luigi

• Work with cloud services like AWS, GCP, Azure, and Oracle

• Manage containerization with Docker and Kubernetes

Key Skills and Tools

• Programming: Python (Pandas, NumPy), Java, Scala, SQL, NoSQL, Bash scripting

• Data Engineering Tools: Hadoop, Spark, Kafka, Airflow, Docker

• Data Visualization and BI Tools: Tableau, Power BI, Looker, QlikView, Google Data Studio

• Database Management: MySQL, PostgreSQL, MongoDB, SQL Server, Redis

• Cloud Computing: AWS (EC2, S3, RDS, Redshift), Google Cloud Platform (BigQuery, Compute Engine), Azure Data Factory