Senior Data Engineer

This is a fully remote position, open to applicants in United States.

📋 Description

• Convert business requirements into infrastructure that captures customer data.

• Acquire datasets that align with business objectives and develop algorithms to transform data into actionable insights.

• Construct, test, and sustain database pipeline architectures.

• Develop methods for data validation and tools for data analysis.

• Design, host, and maintain enterprise APIs and data solutions.

• Oversee, test, and validate the products being supported.

• Cleanse, organize, transform, maintain, safeguard, and update data structures and integrity using big-data techniques.

• Set design standards and assurance processes for the development of software, systems, and applications.

• Review business and product requirements concerning data operations and suggest system and storage modifications.

• Collaborate with business partners and cross-functional teams.

• Provide advanced analytics training to both technical and non-technical partners.

• Direct project objectives, prioritization, quality of work, workload, and resource allocation.

• Mentor junior team members and assist with recruiting and hiring efforts.

• Choose and implement advanced analytical methodologies.

• Communicate insights, recommendations, impacts, reports, updates, and presentations effectively.

• Lead the requirements gathering, design, and development of complex applications and programs.

• Create replicable predictive and prescriptive analytics solutions.

• Report to a Manager or higher; no direct reports.

⛳️ Requirements

• Must be at least 18 years old.

• Must have legal authorization to work in the United States.

• Required education: bachelor's degree or equivalent in a relevant field.

• At least 4 years of professional experience is needed.

• Proven experience in predictive modeling, data mining, and data analysis.

• Demonstrated experience in developing and testing ETL jobs/pipelines, configuring orchestration, automated CI/CD, writing automation scripts, and supporting production pipelines.

• Proficiency in Python.

• Familiarity with defining and capturing metadata and rules related to ETL processes.

• Experience in constructing batch and streaming pipelines.

• Ability to write analytical SQL queries and optimize query performance.

• Capability to integrate and maintain data from various sources.

• Proficient in JavaScript, React, Nucleus, Retina, KPI Shield, and Alert Goose.

• Ability to generate tags for site data.

• Experience using Python and Google BigQuery to stitch and enhance raw data from multiple sources.

• Familiarity with PySpark, AirFlow, and DataProc.

• Ability to optimize pipeline runtime and lower slot/storage consumption costs.

• Skill in prioritizing requests and managing a product roadmap.

• Experience in coaching junior engineers.

• Excellent verbal and written communication skills.

🏝️ Benefits

• Overnight travel typically required 5% to 20% of the time.