Azure Data Engineer

Evolvice GmbH

Overall Responsibilities:

• Design, develop and manage data pipelines on Microsoft Azure for internal and external data sources.
• Develop ingestion frameworks for batch and real-time data.
• Ensure data pipelines are optimized for reliability and scalability.
• Ensure the scalability, security, and performance of our cloud architecture while aligning with business goals.
• Work closely with solution architect, backend developers, data governance specialists

Technical Responsibilities :

• Collaborate with business stakeholders to translate functional requirements into scalable technical solutions.
• Hands-on experience with Microsoft Fabric platform in building end-to-end data solutions.
• Hands-on experience with building pipelines using MS Fabric.
• Build and optimize data lake zones (raw, curated, consumption) Proficiency in developing business and dataflow logic for data processing using tools like PySpark, T-SQL, Stored Procedures Experience implementing message queues using Azure Service Bus Integrate cloud-based solutions with on-premises systems.
• Implement infrastructure as code (IaC) solutions using Terraform for consistent, automated provisioning In-depth knowledge of Azure architecture components, including compute, storage, networking, and security.
• Optimize cloud storage solutions for performance and cost.
• Establish monitoring and logging solutions using Azure Monitor, Log Analytics, Application Insights, and third-party tools.
• Configure automated scaling solutions to handle workload fluctuations.
• Create and maintain comprehensive architecture documentation, including diagrams, processes, and technical specifications.