Data Engineering Lead

This is a fully remote position, open to applicants in Ohio.

📋 Description

• Manage the architecture, development, and upkeep of the data platform.

• Ensure the quality, efficiency, integrity, and security of data.

• Act as the main subject matter expert for the applications and technologies in use.

• Collaborate with business analysts, architects, product owners, vendor partners, and IT teams.

• Review architectural designs and project requirements.

• Lead code and design reviews, analyze outcomes, and propose solutions.

• Identify technical risks and create contingency plans.

• Analyze operations and assess potential improvements.

• Conduct security audits to ensure compliance.

• Assist teams in navigating technical challenges and issues.

• Support governance, quality assurance, standards, and adherence to technology architecture.

• Spearhead technology changes and manage implementation and modification activities.

• Develop production-ready IT services for agile teams.

• Assess new and emerging technologies and oversee proof-of-concept initiatives.

• Create detailed designs and review developer implementations for quality and adherence to best practices.

• Draft, review, and approve technical documentation.

• Lead the development of technology strategy and roadmap.

• Collaborate with third-party vendors to verify technical aspects, evaluate feasibility, and develop metrics, dashboards, and status reports.

• Coordinate operational processes, including crisis management, support escalation, change management, release management, and capacity management.

• Analyze, plan, justify, schedule, and coordinate initiatives related to administration, operations, and finances in specialty areas.

⛳️ Requirements

• Bachelor's Degree in Computer and Information Science.

• Bachelor’s or Master’s degree in Computer Science, Information, or a related field.

• At least 12–15 years of experience in data engineering, focusing on ETL and cloud technologies.

• Proficient in Informatica ETL tools, SQL, and Azure Cloud platforms.

• Experience with Informatica IDMC and the suite of Informatica tools.

• Familiarity with MDM architecture and data flow.

• Experience migrating from existing data platforms to Databricks and Microsoft Fabric.

• Working knowledge of Azure data pipelining, scheduling, and monitoring.

• Ability to debug problematic pipelines.

• Hands-on experience with data pipelines.

• Strong background in big data technologies, including Spark and Databricks.

• Understanding of integration between cloud and on-premises resources.

• Proficient in database and data warehouse development, encompassing replication, staging, ETL, stored procedures, partitioning, change data capture, triggers, scheduling tools, cubes, and datamarts.

• Experience with backend programming languages such as Python.

• Proficient in data manipulation using analytics tools/platforms like Databricks and Azure Fabric with Spark engine.

• Expertise in at least one technology stack, such as Java, Python, or PySpark.

• Excellent debugging, troubleshooting, analytical, problem-solving, communication, and collaboration skills.

• Ability to work with architects and managers to develop metrics and KPIs.

• Experience in an Agile development environment is preferred.

• Experience in building data pipelines that support ML workflows is a plus.

• Preferred experience working with geographically distributed teams.

• Adaptable to new technologies and possess a continuous-learning mindset.

• Ability to estimate the financial impact of technology alternatives.

• Proven ability to collaborate effectively and earn respect as a leader.

🏝️ Benefits

• Work-from-home option for residents of Ohio.

• Day shift schedule.

• 40 hours of work scheduled each week.

• Opportunities for learning and professional growth.

• Supportive and professional work environment.

• Recognized as a Great Place to Work Certified™ organization.