Data Engineer

This is a fully remote position, open to applicants in United States.

📋 Description

• Design, develop, and manage scalable data pipelines for both batch and real-time data processing.

• Construct and enhance ETL/ELT workflows to gather, transform, and load data from various sources.

• Create and sustain data models, data warehouses, and data lakes.

• Ensure the quality, consistency, accuracy, and reliability of data across pipelines.

• Improve data processing workflows for enhanced performance, scalability, and cost-effectiveness.

• Work with extensive and intricate datasets from multiple origins.

• Collaborate with Data Scientists and ML Engineers to prepare and provide high-quality datasets for machine learning applications.

• Develop data integrations with APIs, databases, cloud platforms, and third-party systems.

• Monitor data pipelines and resolve issues related to data processing and infrastructure.

• Implement frameworks for data validation, monitoring, and testing.

• Maintain comprehensive documentation for data pipelines, data models, and technical processes.

• Adhere to best practices for data security, governance, and access management.

• Contribute to the design and advancement of Hypersonix's data platform and architecture.

⛳️ Requirements

• 5–7 years of experience in Data Engineering or a related field.

• Proficient programming skills in Python or an equivalent programming language.

• Strong experience with SQL and relational database systems.

• Practical experience in constructing ETL/ELT pipelines and data workflows.

• Experience with data processing technologies like Spark/PySpark.

• Familiarity with cloud platforms such as AWS, Azure, or GCP.

• Solid understanding of data warehousing and data lake principles.

• Experience with workflow orchestration tools such as Airflow, Dagster, or similar technologies.

• Experience in working with distributed systems and large-scale datasets.

• Strong comprehension of data modeling, database design, and performance enhancement.

• Knowledge of Git and CI/CD methodologies.

• Good grasp of software engineering principles and coding best practices.

🏝️ Benefits

• Competitive salary and performance-based incentives.

• Comprehensive health, dental, and vision insurance.

• Opportunities for professional development and career advancement.

• Flexible work hours and the possibility of remote work.

• Collaborative and innovative work environment.