Senior Infrastructure Engineer

This is a fully remote position, open to applicants in United States, +1 more country.

📋 Description

• Take ownership of the Databricks Lakehouse platform across various environments, including the regulated Upstart Bank environment.

• Enhance infrastructure as code for new Lakehouse functionalities while implementing a no-click-ops policy.

• Develop disaster recovery plans and apply least-privilege and attribute-based access controls.

• Ensure isolation of environments and address known security vulnerabilities.

• Close control gaps and automate evidence collection for OCC, SOX, and SOC audits.

• Minimize Databricks and AWS expenditures through cost attribution, waste detection, and platform-level optimizations.

• Assess and integrate new technologies, including open-source lakehouse alternatives.

• Boost developer productivity and infrastructure reliability in collaboration with Data Engineering, Machine Learning, Analytics, Compliance, and Security teams.

• Enhance Lakehouse security, usage efficiency, and observability.

• Advocate for coding and engineering best practices while mentoring junior engineers.

• Decompose complex projects and requirements into quarterly commitments and sprints.

• Manage Lakehouse infrastructure components comprehensively, from technical requirements and design to rollout and production support.

⛳️ Requirements

• Bachelor's degree in Computer Science or a related discipline, or equivalent practical experience.

• Over 5 years of experience in infrastructure engineering, DevOps, or data platform engineering.

• Practical experience operating a lakehouse or big data platform, such as Databricks, Snowflake, Apache Spark, Delta Lake, or Apache Iceberg.

• Proficient hands-on experience with AWS, including IAM, S3, EC2, or EKS.

• Experience with Python programming.

• Familiarity with infrastructure as code using Terraform or CDK.

• Exposure to CI/CD and Kubernetes.

• Preferred experience in implementing security and compliance controls in audited environments.

• Preferred experience in reducing costs related to cloud or data platforms.

• Proficiency in DevOps engineering and tools like Kubernetes, Jenkins, CDK, APM, and Terraform is preferred.

• Understanding of cloud networking fundamentals is preferred.

• Experience with AI coding assistants or agents is preferred.

• Familiarity with distributed systems, data architecture design, and big data technologies is preferred.

• Most employees can live and work anywhere in the U.S.; Canada is also recognized as an eligible office location, excluding Quebec.

🏝️ Benefits

• Competitive compensation package, including base salary, bonus opportunities, and annual equity grants that vest quarterly.

• 401(k) or Group Retirement Savings Plan with a company match of $2 for every $1 contributed, up to $15,000 annually.

• Employee Stock Purchase Plan (ESPP) with discounted stock purchase options for eligible employees (US only).

• Comprehensive medical, dental, vision, and wellness resources.

• Supplemental health coverage for Canada.

• Contributions to Health Savings Accounts for eligible plans (US only).

• Life insurance and disability coverage provided.

• Generous paid time off, sick leave, and company holidays.

• Paid family and parental leave available.

• Family-centered benefits for fertility, parenthood, and caregiving.

• Employee Assistance Program offering mental health support and life-centered resources.

• Financial planning tools and financial concierge service (US only).

• Annual wellness allowance provided.

• Annual productivity allowance for relevant tools and resources.

• Engaging team events, all-company updates, and employee resource groups.

• Catered lunches and fully stocked micro-kitchens at offices in the Bay Area, Austin, Columbus, and New York City.