Your responsibilities
We are seeking a highly experienced Senior Data Engineer with expertise in Azure-based cloud architecture to join our team. In this role, you will design, build, and optimize complex data pipelines and cloud infrastructure to support data-driven business decisions. You’ll be responsible for implementing robust, scalable solutions leveraging Azure Synapse Analytics, Databricks, ADF, SQL with DevOps. The ideal candidate will also possess experience in Power BI, data mining, data analysis, and migration of on-premises systems to the cloud. Familiarity with Microsoft Fabric is an added advantage.
Key Responsibilities
Cloud Architecture & Infrastructure:
Design and implement Azure-based cloud infrastructure, including data storage, processing, and analytics components.
Develop and optimize scalable data architectures to ensure high performance and availability.
Data Pipeline Development:
Build and manage ETL/ELT pipelines using Azure Synapse, Azure Data Factory (ADF), and Databricks.
Ensure the efficient flow of data from source to target systems, implementing robust data quality controls.
Data Transformation & Analysis:
Utilize SQL, Synapse, and Databricks for data transformation, data mining, and advanced data analysis.
Implement best practices for data governance, lineage, and security within Azure environments.
Migration Projects:
Lead the migration of on-premises data systems to Azure cloud infrastructure, ensuring minimal disruption and data integrity.
Optimize data migration strategies and methodologies for various applications and workloads.
DevOps & CI/CD Pipelines:
Manage DevOps processes, ensuring continuous integration and deployment (CI/CD) for data solutions.
Develop and maintain infrastructure as code (IaC) for deployment, testing, and monitoring.
Business Intelligence & Reporting:
Collaborate with business stakeholders to design and implement reporting solutions in Power BI, ensuring data is accessible and actionable.
Develop visualizations and dashboards to support data-driven decision-making.
Collaboration & Best Practices:
Work closely with data scientists, analysts, and other business stakeholders to understand requirements and provide optimized data solutions.
Drive best practices in data engineering, including coding standards, testing, version control, and documentation.
Your profile
- Candidate must have 3 - 6 years of experience
-
Education: Bachelor’s or master’s degree in computer science, Information Technology, Data Engineering, or related field.
Technical Expertise:
- Azure Cloud: Advanced proficiency in Azure services, including Synapse Analytics, Data Factory, Databricks, SQL, Blob Storage, and Data Lake.
- Data Engineering: Strong skills in SQL, ETL/ELT processes, data warehousing, and data modeling.
- DevOps: Experience with CI/CD pipeline setup, automation, and Azure DevOps.
- Data Analysis & BI: Proficient in data analysis and visualization using Power BI; experience in data mining techniques is desirable.
- Migration Experience: Proven track record of migrating on-premises systems to Azure cloud.
Additional Skills:
- Knowledge of Microsoft Fabric is a plus.
- Familiarity with Infrastructure as Code (IaC) tools like ARM templates or Terraform.
- Strong understanding of data governance, security, and compliance best practices.
- Soft Skills:
- Strong problem-solving and analytical skills.
- Excellent communication skills, with the ability to collaborate effectively across teams.
- Ability to manage multiple priorities and work independently in a dynamic environment.
- Work location: Thane (Mumbai)
Contact
Vinit Poojary - tkmits-in-recruitment@thyssenkrupp-materials.com
