Data Engineer II

This is a fully remote position, open to applicants in India.

📋 Description

• Design and implement scalable, secure, and reliable data pipelines utilizing contemporary data platforms like Spark, Databricks, Airflow, and Snowflake.

• Create ETL/ELT processes for data ingestion from both structured and unstructured sources.

• Conduct Exploratory Data Analysis to identify trends, verify data integrity, and extract insights for the development of data products and informed business decisions.

• Work collaboratively with data scientists, analysts, and software engineers to craft data models that facilitate analytics and real-time insights.

• Write maintainable Python code accompanied by thorough unit and integration tests.

• Take ownership of complete feature delivery within an agile, team-oriented setting.

• Lead complex data and trend analyses in coordination with data science, product, engineering, and customer success teams.

• Evaluate and assess product operations projects; produce and distribute scorecards and reports.

• Recognize and pursue opportunities stemming from customer and business data.

• Contribute to the development of policies, processes, and tools that tackle product quality challenges.

⛳️ Requirements

• A Bachelor's or Master's degree in Computer Science, Engineering, or a related discipline.

• Over 4 years of practical experience in data engineering or backend software development roles.

• Strong grasp of Relational Databases (RDS, MySQL, PostgreSQL).

• Experience with Apache Kafka or RabbitMQ for constructing asynchronous, decoupled systems.

• Proficient in Python, SQL, and at least one data pipeline orchestration tool (such as Apache Airflow, Luigi, or Prefect).

• Significant experience with cloud-based data platforms (for instance, AWS Redshift, GCP BigQuery, Snowflake, Databricks).

• Thorough understanding of data modeling, data warehousing, and distributed systems.

• Familiarity with DevOps practices (CI/CD, infrastructure as code, and containerization using Docker/Kubernetes).

• Exposure to AI/ML-integrated solutions or a willingness to collaborate with data science teams.

• Awareness of data security and privacy regulations (e.g., GDPR, HIPAA).

• Understanding of prompt engineering and the interactions of LLM-based systems with data.

🏝️ Benefits

• Competitive salary and performance-based bonuses.

• Comprehensive health, dental, and vision insurance plans.

• Opportunities for professional development and continuous learning.

• Flexible work hours and options for remote work.

• Collaborative and inclusive work environment.