Principal Software Engineer – Distributed Systems

This is a fully remote position, open to applicants in United States.

📋 Description

• Define, enhance, and take ownership of the architectural vision and roadmaps for key platform domains (e.g., orchestration, job execution, data pipelines, sidecar services).

• Conduct architecture reviews, evaluate proposals from teams, and assess designs for compliance with standards, scalability, performance, reliability, and security.

• Lead or actively participate in critical subsystems (e.g., orchestrator, scheduling, service mesh, job pipelines, stateful services).

• Promote and enforce architectural patterns across teams.

• Provide thought leadership by prototyping proofs-of-concept, assessing new tools/technologies, and fostering innovation within the platform.

• Mentor and guide architects, senior engineers, and support teams through architectural transformations.

• Collaborate with infrastructure, operations, SRE, security, and DevOps to ensure the architecture is operational, observable, and maintainable in production environments.

• Assist in defining and leading architecture governance (e.g., architecture review board, reference architectures, standards, guidelines).

• Engage in tech strategy and roadmap planning alongside product and engineering leadership.

• Identify and address technical debt or architectural "smells" throughout the codebase, ensuring consistency.

⛳️ Requirements

• Approximately 10+ years (or more) of software engineering experience.

• A minimum of 3–5 years in senior architect or principal-level roles (or similar).

• Demonstrated experience in designing and managing large-scale, distributed systems in a production setting.

• Extensive hands-on experience with orchestration, scheduler systems, job pipelines, or workflow engines.

• Strong expertise in one or more of the following: Java / JVM ecosystem, Python, Node.js, or comparable high-level backend languages.

• Familiarity with message/event systems (e.g., Kafka or similar), queuing, and streaming architectures.

• Knowledge of in-memory data stores and caching solutions (e.g., Redis or equivalents).

• Solid understanding of containerization, Kubernetes, sidecar patterns, service meshes, proxies, etc.

• Experience or understanding of split-plane architectures (control plane / data plane separation) is highly desirable.

• Strong skills in API design, data modeling, integration patterns, error handling, and consistency.

• Experience with observability (metrics, tracing, logging) and designing systems for operability and failure modes.

• Capability to conduct performance analysis, capacity planning, scaling, and fault tolerance strategies.

• Excellent communication skills—able to articulate tradeoffs, persuade stakeholders, write design documents, and lead design reviews.

• Ability to work independently, make decisions in uncertain situations, drive consensus across teams, and manage technology adoption or retirement as necessary.

• Strong judgment regarding when to optimize versus when to simplify.

• Familiarity with high-scale cloud deployments (e.g., AWS, Azure, GCP) and infrastructure-as-code is a plus.

🏝️ Benefits

• Medical, dental, and vision coverage.

• 401(k) with company match.

• Paid parental leave, caregiver leave, and flexible time off.

• Mental health support and wellness reimbursement.

• Career development and education assistance.

• Connectivity Plus stipend of $150 to assist with remote work-related expenses.

• An annual $200 reimbursement for home office expenses.