Senior Data Engineer

This is a fully remote position, open to applicants in United States.

📋 Description

• Design and implement infrastructure and data automation pipelines to gather, process, and load data from diverse sources.

• Automate and integrate additional components into the data pipeline.

• Work closely with stakeholders and data science teams to create data products, including entity resolution and optimal selection, to effectively realize the product vision and strategy in line with organizational goals and priorities.

• Develop unit tests and stress-test components to monitor technical performance and address identified issues.

• Create data analysis tools that deliver insights and capture essential metrics.

• Investigate solutions and maintain comprehensive technical documentation.

• Adhere to best practices for data governance, quality assurance, cleansing, and other ETL-related tasks.

⛳️ Requirements

• Over 7 years of development experience in data engineering within a production environment (excludes internships and academic settings).

• Demonstrated experience with large datasets exceeding 100 million records or multiple terabytes.

• More than 2 years of development experience in highly scalable, distributed systems and cluster architectures utilizing AWS.

• At least 5 years of hands-on programming experience with Python.

• Over 5 years of professional experience in big data ecosystems; Spark experience is required, while PySpark is preferred.

• More than 3 years of experience with SQL, schema design, and dimensional data modeling.

• At least 2 years of professional experience using dataflow orchestration tools, such as Airflow.

• More than 2 years of experience with non-relational databases (e.g., DynamoDB, Elasticsearch, etc.).

• A bachelor’s degree in Computer Science, Information Systems, Mathematics, or a related field is mandatory.

🏝️ Benefits

• Comprehensive medical, dental, and vision coverage at 100%.

• Unlimited paid time off for employees.

• Bonus program.

• Equity plans available.

• 401(k) plan.