Junior Data Engineer

This is a fully remote position, open to applicants in United States.

📋 Description

• Develop and uphold data ingestion pipelines and notebooks within Microsoft Fabric, incorporating Data Factory pipelines, Python/Spark notebooks, and Delta tables.

• Supervise daily scheduled pipelines and container jobs, address failures, and ensure that any partially failed jobs are reported accurately.

• Create and sustain data quality checks for data freshness, row counts, duplicate records, and reconciliation with source systems, while setting up appropriate alerts.

• Transition ad-hoc and desktop-scheduled scripts to a managed, version-controlled, and monitored infrastructure.

• Transform source tables into conformed Silver dimensions and Gold facts, complete with documented data grain and business definitions.

• Keep runbooks and the data catalog up to date.

• Diagnose pipeline issues in collaboration with senior engineering team members, identify root causes, and devise dependable solutions.

• Work together with data, analytics, and business teams to guarantee that data is accurate, accessible, and suitable for reporting and analysis.

⛳️ Requirements

• 1–3 years of experience in data engineering, analytics engineering, software development, or a relevant area; internships and significant academic or personal projects may be considered as experience.

• Proficient in SQL, including joins and window functions, with an understanding of how joins can inadvertently multiply or "fan out" rows.

• Practical knowledge of Python, including libraries such as pandas or PySpark.

• Regular experience using Git and version control systems.

• Exceptional attention to detail and a methodical approach to investigating existing pipelines to determine the reasons behind changes in data results or metrics.

• Strong dedication to data accuracy and quality, with the discernment to flag discrepancies rather than presenting figures that merely seem reasonable.

• Robust problem-solving abilities and a keen interest in learning data engineering practices, tools, and technologies.

• Familiarity with Microsoft Fabric, Azure Data Factory, Databricks, or Synapse.

• Understanding of Delta Lake and Parquet formats.

• Basic familiarity with Azure services such as App Service, Container Apps, Key Vault, and Entra ID.

• Experience dealing with ERP or manufacturing data, including systems like Oracle, IQMS/DELMIAworks, SAP, or Epicor.

• Proficient in Power BI.

🏝️ Benefits

• Competitive salary and performance-based bonuses.

• Opportunities for professional development and career advancement.

• Comprehensive health and wellness benefits.

• Flexible work arrangements and support for work-life balance.

• Access to the latest tools and technologies in data engineering.