Data Engineer

This is a fully remote position, open to applicants in California.

📋 Description

• Oversee the development, upkeep, and enhancement of both batch and streaming ETL/ELT processes.

• Design, refine, test, containerize, and manage workflows for data loading and processing using event-driven architecture and workflow orchestration.

• Create and sustain workflows within IDEs utilizing AWS, Snowflake, Python, SQL, dbt, and shell scripts.

• Evaluate, redesign, and expand data models that support data warehousing and self-service business intelligence.

• Establish automated data pipelines from staged or streamed data through the organization of data warehouses, reporting, alerts, or API access.

• Conceptualize and execute data pipeline automation that integrates Salesforce, Slack, and AWS services within serverless, cloud-server, and containerized frameworks.

• Engage in meetings, conduct research, gather requirements, and perform data analysis.

• Collaborate with project stakeholders and key contributors.

• Cultivate and maintain an expert understanding of CaliforniaColleges.edu data, systems, and business processes.

• Persistently enhance data pipelines and architecture by adhering to industry trends and best practices.

• Partner with the Cloud Infrastructure Team to supervise CCGI’s cloud-based data infrastructure, data warehousing, engineering, orchestration, and user-focused data products.

• Report directly to the Director of Cloud Infrastructure.

• Fulfill additional responsibilities as assigned.

⛳️ Requirements

• A relevant BA/BS in Computer Science, Statistics, Informatics, Information Systems, or another quantitative discipline.

• At least three (3) years of experience as a Data Engineer or in a similar capacity.

• Must currently reside full-time in California.

• Capability to develop processes that support data transformation, data structures, metadata, dependency, and workload management.

• Advanced expertise in cloud data warehousing and analytical tools such as Snowflake, Redshift, Spark, and Kafka.

• Proficient in SQL, including SQL Server, PostgreSQL, MySQL, relational databases, query crafting, and optimization.

• Advanced skills with ETL tools like Jupyter Notebooks, dbt, and Talend.

• Strong proficiency in scripting languages such as Python, Scala, and R.

• Advanced capability in building and optimizing data pipelines and datasets.

• Proven track record of manipulating, processing, and deriving value from large structured and unstructured datasets.

• Demonstrated ability to produce thorough technical documentation.

• Experience with K-12 data and/or educational data is preferred.

• Advanced certifications such as SnowPro Core, AWS Cloud Practitioner Certification, or AWS Solutions Architect - Associate are advantageous.

• Strong analytical and decision-making abilities.

• Capacity to progress projects independently, prioritize tasks, and manage multiple projects concurrently.

• Experience in supporting and collaborating with cross-functional teams in a fast-paced environment.

• Proficient in accurately and succinctly documenting tasks using tools like Jira.

• Exceptional verbal communication skills with the ability to convey complex information to both technical and non-technical audiences.

• Commitment to enhancing educational access and equity.

• Submission of a cover letter and resume is required.

🏝️ Benefits

• Paid holidays.

• Sick and vacation leave.

• Medical, dental, and vision insurance.

• Employee Assistance Program.

• Defined benefit retirement plan (CalPERS).

• 403(b) and 457(b) retirement plans.

• $50,000 Basic Life and Accidental Death and Dismemberment insurance coverage for eligible employees.

• Flexible Spending Accounts.

• Tuition Reimbursement.

• Voluntary Life Insurance.

• Pet insurance.

• Remote work from home offices.

• Occasional internal in-person meetings and local/statewide travel for necessary business meetings.