Data Engineer – Contractor

This is a fully remote position, open to applicants in India.

📋 Description

• Design, develop, and sustain scalable data solutions using Databricks, Microsoft Fabric, SQL, Spark, and Delta Lake.

• Construct and enhance data ingestion, transformation, and orchestration pipelines for enterprise data platforms.

• Create automation frameworks for data validation, testing, monitoring, deployment, and operational support.

• Direct data engineering initiatives encompassing requirements, design, development, testing, deployment, and support.

• Develop and uphold automated QA and reconciliation frameworks for analytical and reporting solutions.

• Ensure data quality, completeness, consistency, and compliance with business rules across enterprise datasets.

• Design and maintain semantic models, lakehouses, warehouses, and data marts within Microsoft Fabric.

• Execute and verify Row-Level Security (RLS) and data governance standards.

• Collaborate with business stakeholders, architects, analytics teams, and platform engineers.

• Facilitate CI/CD, DevOps, Infrastructure-as-Code, and deployment automation efforts.

• Promote performance optimization and cost efficiency within Databricks and Fabric environments.

• Establish engineering best practices, coding standards, reusable frameworks, and governance processes.

⛳️ Requirements

• Bachelor's Degree.

• Over 5 years of experience in Data Engineering, Analytics Engineering, or Data Platform Development.

• Proficient hands-on knowledge in Databricks, Apache Spark (PySpark), SQL, Microsoft Fabric, Delta Lake, and Data Warehousing.

• Experience in designing and implementing data pipelines, ETL/ELT solutions, lakehouse architectures, and data integration frameworks.

• Demonstrated expertise with Databricks Workflows, Jobs, Automation, and Orchestration.

• Experience in developing automated testing and validation frameworks for data platforms.

• Practical experience in implementing CI/CD pipelines utilizing Azure DevOps, GitHub, or similar tools.

• Strong understanding of data quality, reconciliation, metadata management, and governance practices.

• Experience in performance tuning and optimization of Spark workloads.

• Familiarity with Microsoft Fabric components including Data Factory, Lakehouse, Warehouse, Semantic Models, and Power BI.

• Preferred: Knowledge of Infrastructure as Code (Terraform, Bicep, ARM Templates), DataOps/MLOps, Power BI semantic models and DAX, enterprise data governance and security frameworks, cloud platforms such as Azure, AWS, or GCP, and technical leadership/mentoring.

• Strong analytical and problem-solving abilities.

• Excellent communication and stakeholder management skills.

• Capability to lead initiatives independently and collaborate effectively in cross-functional teams.

• Strategic mindset to align technical solutions with business goals.

🏝️ Benefits

• Competitive salary and performance-based bonuses.

• Comprehensive health, dental, and vision insurance.

• Generous paid time off and flexible work arrangements.

• Opportunities for professional development and continuous learning.

• Collaborative and inclusive work environment.