Data Engineer

This is a fully remote position, open to applicants in India.

📋 Description

• Design and develop resilient ETL pipelines that gather, modify, and store data from various sources into a consolidated data warehouse.

• Create and maintain dashboards and data visualizations using Streamlit to deliver real-time insights for product, marketing, and engineering teams.

• Incorporate Posthog event data into the analytics workflow and establish segmented user funnels for product enhancement.

• Develop and sustain LangChain workflows to facilitate conversational AI features that engage with structured data.

• Utilize deepeval to automate the evaluation of machine learning models, ensuring their performance meets business KPIs.

• Enhance SQL queries and database architectures for improved speed, reliability, and scalability across extensive datasets.

• Collaborate with data scientists to provision and manage accessible data lakes and data marts.

• Enforce data governance, security, and privacy measures to adhere to industry regulations.

• Monitor the health of pipelines, troubleshoot issues, and implement performance improvements.

• Document data architecture, pipeline processes, and best practices for both internal and external stakeholders.

⛳️ Requirements

• Minimum of 3 years of professional experience in developing data pipelines and analytics platforms within a production environment.

• Proficient programming skills in Python, with substantial experience in crafting reusable, testable code.

• Extensive knowledge of SQL and experience with contemporary database systems such as PostgreSQL, BigQuery, or Snowflake.

• Demonstrated success in integrating Posthog for behavioral data collection and analysis.

• Practical experience with deepeval or similar frameworks for model evaluation.

• Familiarity with LangChain for constructing LLM-driven data workflows and prompt engineering.

• Strong understanding of Streamlit or similar open-source toolkits for creating interactive dashboards.

• Capability to work collaboratively with cross-functional teams in a remote, agile setting.

• Excellent problem-solving skills, with the ability to address complex data challenges under tight deadlines.

• Dedication to code quality, thorough documentation, and ongoing learning in the rapidly evolving field of data engineering.

🏝️ Benefits

• Competitive salary with performance-based bonuses.

• Flexible working hours and remote work options.

• Opportunities for professional development and continuous learning.

• Comprehensive health benefits and wellness programs.

• Collaborative and inclusive company culture that values innovation.