Data Engineer

Job Summary

We are seeking a skilled and detail-oriented Data Engineer with deep expertise in Azure, SQL Server, and Databricks to design, build, and manage scalable data pipelines and enterprise data solutions. This role will be critical in supporting our analytics, reporting, and data science initiatives by delivering high-quality, reliable, and performant data systems.

Key Responsibilities

• Design, develop, and manage ETL/ELT pipelines using Azure Data Factory (ADF) and Databricks for batch and real-time data processing.
• Integrate data from various structured and unstructured sources including SQL Server, Azure SQL Database, Azure Data Lake Storage (ADLS), and external APIs.
• Build and maintain data models, data marts, and data warehouses using SQL Server and Azure Synapse Analytics.
• Write efficient and optimized SQL queries, stored procedures, views, and triggers in SQL Server.
• Use Databricks (Spark with Python/Scala) to process large datasets for transformation and analytics.
• Ensure data quality, integrity, security, and compliance across the pipeline using data validation, monitoring, and auditing techniques.
• Collaborate with data analysts, data scientists, and business stakeholders to define and deliver data solutions.
• Implement CI/CD pipelines using Azure DevOps.
• Monitor and optimize the performance of data pipelines and queries across Azure and SQL Server environments.

Required Skills & Qualifications

• Bachelor’s degree in Computer Science, Engineering, Information Systems, or related field.
• 5+ years of experience in data engineering or related roles.
• Proven experience with Azure Data Services: Data Factory, ADLS, Azure SQL, Azure Synapse.
• Experience with Databricks (Azure implementation), including experience with Spark (Python or Scala).
• Microsoft SQL Server: Writing advanced SQL, stored procedures, and performance tuning.
• Strong understanding of data warehousing concepts, ETL/ELT best practices, and data modelling (star/snowflake schema).
• Familiarity with data governance, RBAC, and data security practices in Azure.
• Experience with CI/CD tools like Azure DevOps and version control with Git.
• Excellent problem-solving skills and the ability to work collaboratively in a fast-paced environment.