Senior Data Engineer

This is a fully remote position, open to applicants in Florida.

📋 Description

• Design and execute ELT/ETL solutions for batch and streaming data ingestion, integration, refinement, and publishing patterns on the Lakehouse.

• Create reusable data processing frameworks and configuration-driven pipelines utilizing Python and PySpark.

• Develop and sustain scalable orchestration workflows for production data delivery, including mechanisms for retries, historical loads, and operational runbooks.

• Implement data quality assessments, validation frameworks, and monitoring aligned with defined contracts and SLAs.

• Adopt DataOps methodologies, encompassing Git-based development, CI/CD/CT, automated testing, and controlled environment promotion.

• Contribute to practices surrounding data lifecycle, including retention, archival, disaster recovery, and resilience.

• Assist in the modernization of platforms and the cloud migration of legacy data flows into Lakehouse patterns.

• Work alongside stakeholders to align technical designs with business processes, non-functional requirements, and consumption needs.

• Establish and document engineering standards, naming conventions, and practices; engage in Agile ceremonies and cross-team delivery.

• Provide technical guidance through mentoring, design and code reviews, and enhancements to reliability, performance, and cost efficiency.

• Take ownership of reliable datasets that support analytics and reporting across the governed Lakehouse.

⛳️ Requirements

• A Bachelor's degree in Computer Science, Information Systems, Engineering, or a related technical field, or equivalent work experience.

• A minimum of 8 years of overall IT experience.

• At least 5 years of practical experience in designing and developing enterprise-scale data engineering solutions.

• Proficient in developing scalable data pipelines and reusable frameworks with Python and PySpark.

• Familiarity with AWS Glue, dbt, Apache Spark, or similar technologies.

• Strong grasp of data warehousing concepts, dimensional modeling, and modern data lake/Lakehouse architectures.

• Experience with cloud platforms such as AWS, Microsoft Azure, or Google Cloud Platform; AWS is preferred.

• Solid SQL proficiency with relational databases; experience with NoSQL databases is an advantage.

• Familiarity with Apache Spark, Amazon EMR, or Hadoop-based platforms.

• Experience utilizing Git-based source control and Agile software development practices.

• Excellent analytical, problem-solving, and communication abilities.

• Preferred: Knowledge of AWS cloud-native data platforms, Apache Iceberg, CI/CD, DataOps, Terraform, Airflow/MWAA/Step Functions, RESTful APIs, Kafka/Kinesis/Spark Structured Streaming, data quality and observability, governance and metadata management, security and healthcare regulatory compliance (HIPAA/PHI), workload optimization, mentoring, design and code reviews.

🏝️ Benefits

• Comprehensive medical, dental, and vision coverage.

• Incentive and recognition programs.

• Life insurance options.

• 401k contributions.

• A competitive compensation and benefits package.