Senior Data Engineer

This is a fully remote position, open to applicants in New Jersey.

📋 Description

• Design, develop, and maintain scalable data pipelines utilizing Azure data services, Python, Scala, and Apache Spark.

• Create and refine ETL/ELT processes for data ingestion, transformation, and processing from various data sources.

• Construct and oversee data pipelines using Azure Data Factory, Azure Databricks, and Azure Data Lake Storage.

• Develop and enhance Spark applications to boost data processing performance, scalability, and reliability.

• Engage with Hadoop and related big data technologies as required.

• Monitor pipeline execution, address failures, and resolve data processing challenges to ensure dependable data delivery.

• Implement data validation, error management, and data quality assessments to guarantee accuracy and consistency.

• Collaborate with data architects, analysts, and cross-functional teams to comprehend requirements and provide reliable data solutions.

• Adhere to coding standards, testing methodologies, version control, and CI/CD practices.

• Assist with production deployments, incident resolution, performance optimization, and the ongoing maintenance of data engineering solutions.

⛳️ Requirements

• Bachelor’s degree in Computer Science, Information Technology, Computer Engineering, or a related discipline.

• Over 10 years of comprehensive Data Engineering experience.

• Extensive hands-on experience with Microsoft Azure data services, especially Azure Data Factory, Azure Databricks, and Azure Data Lake Storage.

• Proficiency in Python and Scala for data processing and application development.

• Significant experience with Apache Spark and distributed data processing.

• Practical experience with Hadoop and big data technologies.

• In-depth understanding of ETL/ELT processes, data transformations, and data pipeline architecture.

• Strong SQL capabilities and experience with performance tuning, debugging, troubleshooting, and production support.

• Familiarity with Control-M for batch scheduling, job monitoring, workflow orchestration, and dependency management.

• Knowledge of version control, CI/CD, and deployment practices.

• Exceptional analytical, problem-solving, and collaboration abilities.

🏝️ Benefits

• Medical, Dental, Vision

• Retirement Accounts

• Long and Short Term Disability

• Life Insurance

• HSA/FSA Accounts (USA)

• Discretionary time-off

• Strategic engagements involving enterprise-wide AI adoption, cloud-native data modernization, and measurable business outcomes

• Collaboration with AI, data, engineering, and consulting talent across geographies

• Continuous learning and hands-on exposure to emerging technologies

• Opportunities to build expertise in Generative AI, agentic AI, and next-generation data platforms

• Opportunities to grow strategic client partnerships

• Equal employment opportunities