Senior Data Engineer

This is a fully remote position, open to applicants in United States.

📋 Description

• Oversee the creation of robust, observable, and measurable applications utilizing extreme programming methodologies and a user-focused approach.

• Engage in the complete application lifecycle alongside designers, product managers, and engineers.

• Apply critical thinking, experimentation, data analysis, and industry best practices to drive business results.

• Facilitate team discussions and ceremonies to cultivate a shared understanding.

• Provide and receive empathetic, actionable, and precise feedback.

• Practice emergent architecture and develop software that is user-friendly and easily modifiable.

• Define and guide product engineering and software standards.

• Conceptualize products from a user standpoint and develop prioritized, feasible solutions for testing.

• Investigate and remain updated on technological market trends and practices.

• Spearhead technical initiatives within the team and department.

• Develop, automate, and sustain batch and streaming ETL pipelines utilizing Apache Airflow, Apache Spark, Python, and Scala.

• Construct and oversee cloud-based data ecosystems on GCP, incorporating BigQuery, Bigtable, Dataproc, Pub/Sub, Cloud Storage, IAM, and VPC.

• Design and fine-tune SQL and NoSQL data models for data lakes and warehouses using BigQuery, MongoDB, and Snowflake.

• Write intricate SQL queries for data transformation, aggregation, and analytics optimization.

• Implement TDD and test automation across Airflow workflows, Spark jobs, and transformation logic.

• Utilize data mesh and data-as-a-product principles to facilitate reusable, domain-driven datasets.

• Implement real-time ingestion with Kafka Connect and process streaming data using Spark Streaming, Apache Flink, or similar technologies.

• Enhance data performance, scalability, and cost efficiency across GCP components.

• Ensure adherence to PCI and PII data standards, including GDPR, PCI DSS, SOX, and CCPA.

• Integrate OpenAI, Gemini, and Anthropic LLMs to improve data quality and analytical capabilities.

• Collaborate with stakeholders, data scientists, and full-stack engineers to produce reliable, documented, and reusable data products.

• Execute additional assigned responsibilities.

⛳️ Requirements

• Minimum of 4 years of experience in software development.

• Strong understanding of application design patterns, event-driven architecture, databases, schemas, and testing methodologies.

• Extensive knowledge and experience with continuous integration, continuous deployment, and test-driven development.

• Bachelor’s Degree or equivalent in MIS, Computer Science, or a related field (Preferred).

• Experience in troubleshooting large-scale applications and performance tuning (Preferred).

• Familiarity with major cloud platforms (GCP, AWS, or Azure) (Preferred).

• Experience with XP (Extreme Programming) methodologies (Preferred).

• Capability to fulfill the responsibilities outlined in the “What You’ll Do” section.

• Ability to adhere to dress code policies.

• Basic math and reading skills, clear handwriting, and fundamental computer operation skills.

• Capacity to maintain regular attendance and adhere to scheduling requirements.

• Willingness to learn and comply with company policies, procedures, standards, and guidelines.

• Ability to understand, respond to, and act on direction from leadership and company personnel.

• Strong team player who can interact effectively and appropriately with others.

• Ability to maintain composure and work efficiently in a fast-paced environment while managing multiple tasks within set timeframes.

• Capacity to successfully complete company training programs.

• Proficiency in using a personal computer for communication, report generation, and other tasks.

• Ability to plan, prioritize, and oversee activities across various business units.

• Capability to complete or supervise assigned projects in a timely manner.

🏝️ Benefits

• Flexible remote work arrangement.

• Reasonable accommodations may be provided to assist individuals with disabilities in performing essential functions, barring undue hardship.