Senior Data Platform Engineer

This is a fully remote position, open to applicants in United States.

📋 Description

• Develop and enhance data pipelines for both batch and streaming processes.

• Extract, analyze, and model a wide array of structured and unstructured datasets.

• Create software that is designed for easy testing and maintenance.

• Assist in establishing data strategies and our overarching vision.

• Stay updated on emerging technologies and trends in the Data Engineering domain, integrating modern tools and best practices at Craft.

• Work on scalable data processing systems that enable the addition and expansion of pipelines.

• Utilize machine learning techniques, including anomaly detection, clustering, regression classification, and summarization, to derive insights from our datasets.

• Utilize AI-driven development tools (e.g., Cursor) to expedite development, refactoring, and code generation.

⛳️ Requirements

• Over 4 years of experience in Data Engineering.

• More than 4 years of experience with Python.

• Proven experience in developing, maintaining, and ensuring the reliability, scalability, fault tolerance, and observability of data pipelines in a production setting.

• Fundamental understanding of data engineering methodologies: ETL/ELT, batch and streaming, DWH, Data Lakes, and distributed processing.

• Strong knowledge of the Software Development Life Cycle (SDLC) and sound software engineering practices.

• Familiarity with an infrastructure-as-code approach.

• A demonstrated curiosity through asking questions, exploring new technologies, and a continuous desire for growth.

• Excellent problem-solving skills and the ability to communicate ideas effectively.

• A self-starter who is independent and takes initiative.

• Familiar with at least some technologies in our current tech stack: Python, PySpark, Pandas, SQL (PostgreSQL), ElasticSearch, Airflow, Docker.

• Experience with Databricks, AWS (S3, Batch, Athena, RDS, DynamoDB, Glue, ECS, Amazon Neptune).

• Experience with CircleCI, GitHub, and Terraform.

• Knowledge of AI-assisted coding and experience with tools like Cursor, Co-Pilot, or Codex.

🏝️ Benefits

• Competitive salary starting at $170,000 USD per year.

• Equity in a well-funded, rapidly growing startup.

• Unlimited vacation time to take as needed.

• 99% coverage for health, dental, and vision insurance for employees and their dependents.

• 401K through Empower with flexible investment options.