Senior Software Engineer, Data Platform

This is a fully remote position, open to applicants in United States.

📋 Description

• Design, develop, and sustain production-grade data services and pipelines utilizing Python and Java, deployed on the Google Cloud Platform.

• Construct and enhance Google Cloud Dataflow pipelines for both batch and streaming data, facilitating ingestion, standardization, curation, and analytics.

• Establish and maintain event-driven workflows for data processing using Google Cloud Pub/Sub.

• Design and oversee BigQuery datasets, including table schemas, partitioning strategies, clustering, and reporting views.

• Create and manage Cloud Composer (Apache Airflow) DAGs for effective workflow orchestration.

• Develop and sustain Cloud Run microservices and Cloud Functions.

• Engage in design and code reviews; mentor junior engineers and contribute to the establishment of shared coding standards.

• Collaborate with teams to ensure timely delivery and adherence to best engineering practices.

• Actively participate in the triage and resolution of escalated production issues.

• Adhere to all organizational security policies and procedures, including compliance with HIPAA and HITRUST regulations.

⛳️ Requirements

• A Bachelor's degree in Computer Science, Software Engineering, Data Engineering, or equivalent practical experience.

• Over 5 years of experience in data engineering or backend software engineering, specifically in building production data pipelines and platform services.

• Demonstrated hands-on expertise with Google Cloud Platform data services, including BigQuery, Cloud Storage, Cloud Pub/Sub, Cloud Dataflow, Cloud Composer, Cloud Run, Cloud Functions, Firestore, Cloud SQL, and Secret Manager.

• Strong proficiency in Java for the development of data pipelines.

• Proficiency in Python for authoring Airflow DAGs and automation scripting.

• Experience in designing and implementing both batch and streaming data pipelines.

• Proficiency in BigQuery data modeling and optimization of SQL queries.

• Familiarity with Apache Airflow / Cloud Composer.

• Experience with containerization technologies such as Docker.

• Proficiency in using GitLab CI/CD for automating pipelines.

• Exceptional communication skills.

🏝️ Benefits

• Options for remote work.

• Opportunities for professional development.