Data Engineer

This is a fully remote position, open to applicants in United States.

📋 Description

• Design and create scalable data ingestion pipelines utilizing AWS services and Azure DevOps.

• Develop APIs and file upload systems for the ingestion of client data.

• Create data mapping and transformation workflows for importing data into SQL Server client databases.

• Contribute to the design and execution of a Databricks-based Lakehouse architecture.

• Facilitate a smooth data transfer from the SaaS platform to the analytical platform.

• Enhance ETL/ELT processes for improved performance and reliability.

• Collaborate with analytics teams to guarantee data availability and quality for reporting and machine learning applications.

• Offer operational support for current ingestion processes integrated within C# .NET code and user interface.

• Identify and resolve data pipeline issues across various environments.

• Partner with Software Engineers, Architects, and Product teams to ensure data solutions meet business requirements.

• Apply data governance, security, and compliance best practices in a multi-tenant setting.

• Document architecture, workflows, and operational procedures.

⛳️ Requirements

• Bachelor’s degree in Computer Science, Information Technology, or a relevant field is required.

• Over 5 years of experience in Data Engineering or a similar position.

• Extensive experience with AWS services for data engineering, such as S3, Lambda, Glue, Step Functions, and API Gateway.

• Proficiency in Python and/or C# .NET for data processing and integration tasks.

• Practical experience with SQL Server and concepts related to relational databases.

• Understanding of Databricks, Spark, and Lakehouse architectures.

• Familiarity with ETL/ELT design patterns and data modeling techniques.

• Experience in developing APIs and file-based ingestion solutions.

• Strong analytical and problem-solving abilities.

• Capability to work effectively in a cross-functional team environment.

• Outstanding communication and documentation skills.

• Experience with CI/CD pipelines for data workflows is preferred.

• Knowledge of data governance and security best practices in cloud environments is preferred.

• Familiarity with modern data integration tools and frameworks is preferred.

• Exposure to machine learning pipelines and real-time data processing is preferred.

🏝️ Benefits

• Bonus eligibility.

• Medical and dental coverage for employees, dependents, domestic partners, and spouses.

• Paid time off with flexible options.

• 10 paid company holidays where applicable.

• Vision insurance.

• Short- and long-term disability insurance.

• Basic life insurance.

• Generous family leave options, including adoption and foster care placements.

• Flexible Spending Account.

• Health Savings Account.

• Commuter Benefits.

• Dependent Care Savings Account.

• 401(k) with company match up to 4%.

• Employee Assistance Program (EAP).

• Education Assistance Program for industry/role-specific certifications.

• Wellness reimbursement program.

• Pet Insurance.

• Critical Illness Insurance.

• Voluntary Life & AD&D coverage.

• Flexible PTO not available in California or the UK.