
Data Engineer
Posted Nov 13

Posted Nov 13
This is a fully remote position, open to applicants in India.
• Design, develop, and sustain scalable ETL and ELT pipelines for batch and scheduled data processing.
• Construct and enhance data transformations utilizing SQL, Python, and PySpark.
• Ingest and integrate data from a variety of sources including databases, APIs, files, SharePoint, and cloud storage.
• Create and manage curated, analytics-ready datasets for Power BI, Superset, and other reporting platforms.
• Implement data quality checks, reconciliation controls, monitoring, logging, and failure alerts.
• Diagnose pipeline failures, schema mismatches, performance bottlenecks, and data discrepancies.
• Optimize large-scale data processing for runtime, memory, storage, and cost efficiency.
• Automate recurring workflows and schedule production jobs using orchestration tools like Apache Airflow.
• Maintain technical documentation, data mappings, business rules, and operational runbooks.
• Collaborate with BI analysts, application teams, and business stakeholders to deliver reliable data solutions.
• 2–5 years of professional experience in data engineering, ETL development, or a related field.
• Proficient in SQL, including joins, common table expressions, window functions, data validation, and query optimization.
• Practical experience with Python and/or PySpark for data processing and automation tasks.
• Hands-on experience in building and supporting ETL or ELT pipelines in a production environment.
• Familiarity with large datasets, incremental loads, upserts, schema alterations, and partitioned data.
• Solid understanding of relational databases, dimensional modeling, data warehousing, and file formats such as Parquet, CSV, and JSON.
• Experience using Git or another version control system along with structured deployment practices.
• Strong skills in debugging, analytical thinking, documentation, and stakeholder communication.
• Preferred: Familiarity with AWS services like Amazon S3, Athena, EMR, and EMR Serverless.
• Preferred: Experience with Apache Spark, Delta Lake, Apache Airflow, Dremio, or Trino-compatible SQL engines.
• Preferred: Knowledge of Power BI, Apache Superset, or other reporting tools.
• Preferred: Experience with REST APIs, SharePoint integrations, and automated file delivery workflows.
• Preferred: Understanding of data security, access controls, privacy, and production monitoring practices.
• 5 Days working
• Remote Job
• Medical Insurance
Trax Technologies
Fiserv
Ontrac Solutions
General Dynamics Information Technology
Get handpicked remote jobs straight to your inbox weekly.