Data Engineer

This is a fully remote position, open to applicants in United States.

📋 Description

• Design, enhance, and sustain dependable ETL/ELT pipelines for the ingestion, transformation, validation, and loading of transactional data into Microsoft SQL Server and PostgreSQL.

• Assist in data ingestion and preparation processes that support AI applications, which include document processing, metadata management, chunking, and embedding creation for RAG systems.

• Leverage modern AI tools to expedite code development, refine data transformation workflows, troubleshoot challenges, and boost data engineering productivity.

• Create, schedule, and oversee data processing tasks utilizing Airflow, Azure Data Factory, and SQL Server Agent.

• Examine pipeline errors and establish data quality verification processes.

• Compose and optimize SQL queries, engage in data modeling, and enhance the performance of database and analytical data structures.

• Work collaboratively with Senior Data Engineers, data scientists, product owners, and IT Operations to address production challenges and implement enhancements.

• Keep documentation updated and adhere to Git, Bitbucket, and Jira workflows.

⛳️ Requirements

• A minimum of 3 years of professional experience in data engineering, focusing on building, maintaining, and troubleshooting data pipelines and processing workflows within production settings.

• Practical experience with Python, especially for data processing and transformation tasks.

• Proficient SQL skills, encompassing joins, CTEs, window functions, aggregations, and query optimization.

• Familiarity with pandas is essential.

• Experience with Microsoft SQL Server or PostgreSQL is required.

• Proven experience in developing or managing data workflows using Airflow, Azure Data Factory (ADF), or SQL Server Agent.

• Knowledge of cloud environments, preferably Azure, is a plus.

• Understanding of data modeling, relational database architectures, and data quality principles.

• Experience with document ingestion, embeddings, vector search, or data pipelines supporting AI and RAG applications is advantageous.

• Familiarity with Git-based version control, collaborative development processes, Bitbucket, and Jira.

• Knowledge of CI/CD practices and Docker is a bonus.

• Proactive use of AI tools to hasten development, enhance code quality, automate repetitive tasks, and tackle technical challenges.

• Strong analytical and problem-solving abilities, with a keen attention to detail and a commitment to ownership of assigned responsibilities.

• Proficient in English communication skills, both written and verbal.

• Availability to overlap with U.S. Central business hours is necessary.

🏝️ Benefits

• USA Contractor / 100% Remote

• Competitive Pay (USD)

• Holidays Off & Paid Time Off

• Health insurance assistance program

• Opportunity to collaborate with high-growth US-based companies and innovative tech stacks.

• Access to agentic AI tools and training

• Coursera Enterprise

• Studio practice labs

• Comprehensive onboarding experience

• Open leadership communication

• Community recognition

• Psychological safety

• Ongoing multi-project pipeline