Senior Software Engineer – Data Pipelines

This is a fully remote position, open to applicants in United States.

📋 Description

• Serve as a technical leader and Subject Matter Expert in the automation of data pipelines and orchestration of workflows.

• Design, develop, and oversee complex, reliable data solutions utilizing Airflow, dbt, and Google Cloud data products.

• Manage an extensive portfolio of dbt models through the use of macros and DRY principles.

• Champion test-driven development and support QA by implementing a robust continuous integration and delivery process.

• Monitor, diagnose, and enhance the performance of data pipelines and workflows.

• Architect scalable solutions and reusable patterns that align with business requirements.

• Provide comprehensive documentation for implementation, configuration, and deployment.

• Proactively identify issues and deploy innovative solutions.

• Engage in agile ceremonies, including daily standups and sprint planning sessions.

• Mentor fellow engineers and promote a culture of technical excellence.

• Keep abreast of industry trends and technologies to enhance data engineering practices.

• Ensure adherence to data security, governance, compliance, and privacy regulations.

• Lead intricate data projects and technical initiatives for Polaris, Wpromote’s data and marketing intelligence platform.

• Collaborate with senior management and cross-functional teams.

• Assist in improving the delivery speed and quality of data ELT.

⛳️ Requirements

• At least 7+ years of experience in software development, with significant expertise in Python, SQL, and data pipeline management.

• Proficiency in constructing and automating ETL/ELT pipelines using Airflow and DAG-based workflow management tools.

• Experience with Airbyte for data ingestion is advantageous but not mandatory.

• Extensive experience in managing dbt models utilizing macros and dbt tests.

• Mastery of Python or a similar scripting language, along with API integrations and software architecture.

• In-depth experience with PostgreSQL and BigQuery, including performance and cost optimization of queries.

• Strong understanding of business intelligence tools like Looker or equivalent solutions.

• Familiarity with Google Cloud Platform, Kubernetes, and infrastructure as code methodologies using Terraform.

• Proficient in advanced data formats such as Parquet, Avro, Hive, and JSONL, along with data integration techniques.

• Experience with monitoring and logging tools like Prometheus and Grafana is a plus.

• Knowledge of version control systems and CI/CD tools, such as GitHub Actions.

• Strong grasp of agile methodologies, continuous integration, and test-driven development principles.

• Exceptional problem-solving abilities and technical leadership skills.

• Capability to influence and guide cross-functional teams and projects.

• Must possess authorization to work in the United States; this position does not qualify for immigration sponsorship.

🏝️ Benefits

• Unlimited PTO.

• Extended holiday break during winter.

• 100% paid parental leave.

• 401(k) matching.

• Comprehensive medical, dental, vision, life, and pet insurance.

• Sponsored life insurance.

• Short-term disability insurance alongside additional voluntary insurance options.

• Annual Class Pass credits.

• Flexibility for remote work.

• Office hubs located in Los Angeles and New York, offering learning and development opportunities, events, workspace, and in-person collaboration.