Azure Data Engineer

About the Role

As an Azure Data Engineer at Virtusa, you will be working closely with clients, architects, and cross-functional teams to design and implement end-to-end data engineering solutions on Microsoft Azure. Your primary focus will be on developing and maintaining data pipelines using Azure Data Factory (ADF), building scalable data processing frameworks using Azure Databricks (PySpark/Scala), and architecting and managing data lakes using Azure Data Lake Storage (ADLS Gen2). You will also be responsible for developing data warehousing and analytics solutions using Azure Synapse Analytics and working with HDInsight for big data workloads.

Key Responsibilities

• Design and implement end-to-end data engineering solutions on Microsoft Azure.
• Develop and maintain data pipelines using Azure Data Factory (ADF).
• Build scalable data processing frameworks using Azure Databricks (PySpark/Scala).
• Architect and manage data lakes using Azure Data Lake Storage (ADLS Gen2).
• Develop data warehousing and analytics solutions using Azure Synapse Analytics.
• Work with HDInsight for big data workloads (Hive, Spark, Kafka, etc.) where applicable.
• Optimize data workflows for performance, scalability, and cost-efficiency.

Skills & Qualifications

• Strong experience with Azure Data Factory (ADF).
• Azure Databricks (PySpark, Spark SQL).
• Azure Data Lake Storage (ADLS Gen2).
• Azure Synapse Analytics.
• HDInsight (Spark/Hadoop ecosystem).
• Proficiency in Python and/or Scala.
• Strong SQL and data modeling skills (star/snowflake schema).
• Experience with ETL/ELT design patterns.
• Azure & Data Expertise.
• Hands-on experience with Azure-native data services.
• Understanding of data lake, lakehouse, and data warehouse architectures.
• Experience implementing CI/CD pipelines (Azure DevOps).
• Familiarity with data security, RBAC, and encryption in Azure.
• Experience with monitoring tools (Azure Monitor, Log Analytics).

What You'll Learn

As an Azure Data Engineer, you will have the opportunity to work on complex data engineering projects and develop skills in designing and implementing end-to-end data solutions on Microsoft Azure. You will learn about the latest data engineering tools and technologies, including Azure Data Factory, Azure Databricks, and Azure Synapse Analytics. You will also gain experience in working with big data workloads and optimizing data workflows for performance, scalability, and cost-efficiency.

Resume Tip

When applying for this role, make sure to highlight your experience with Azure Data Factory (ADF) and Azure Databricks (PySpark, Spark SQL). Emphasize your ability to design and implement end-to-end data engineering solutions on Microsoft Azure and your experience with data modeling and ETL/ELT design patterns. Additionally, mention any certifications you have, such as Microsoft Certified Azure Data Engineer Associate.