Data Engineer, Databricks

This is a fully remote position, open to applicants in United States.

📋 Description

• Design, develop, and sustain data pipelines, infrastructure, and systems utilizing AWS, Python, Databricks, Spark, and PostgreSQL.

• Collaborate with cross-functional teams to convert business requirements into technical solutions.

• Offer technical guidance and mentorship to junior data engineers.

• Establish and implement data engineering standards and best practices.

• Monitor and resolve issues within data pipelines to ensure data quality, integrity, and availability.

• Investigate industry trends and assess new data technologies.

• Construct and maintain data lakes to facilitate analytics and business intelligence.

• Oversee the software development life cycle and DevOps activities.

• Collaborate with data scientists and analysts to provide effective data solutions.

• Create data validation and analysis tools.

⛳️ Requirements

• Bachelor’s degree in computer science, information technology, or a related discipline.

• Five or more years of experience in data engineering or a similar role.

• Expert-level proficiency in Python, SQL, Databricks, and Spark.

• Experience in constructing cloud-based data pipelines within AWS.

• Understanding of data governance, security, privacy, and retention practices.

• Strong communication, collaboration, and problem-solving abilities.

• Experience in agile environments utilizing tools like Jira.

• Proficient with Git, Docker, and CI tools such as Jenkins.

• Enthusiasm for data and a commitment to continuous improvement.

• Capability to work independently and take initiative.

• Experience in optimizing ETL workflows in Databricks.

• Familiarity with data lineage and data quality practices.

• Experience in supporting data science and machine learning initiatives.

• Previous experience in the energy or utility sector.

• Master’s degree in a relevant STEM field.

🏝️ Benefits

• Excellent insurance options, including medical, dental, and vision plans;

• Company-paid life insurance;

• Company-paid long- and short-term disability insurance;

• Medical and dependent-care flexible spending plans;

• Paid parental leave.

• A flexible time off (FTO) policy that allows for paid time away from work, subject to manager approval, while ensuring business needs, workload commitments, and appropriate coverage are upheld.

• A 401(k)/RRSP plan featuring a 3% employer match.