Kafka Engineer

This is a fully remote position, open to applicants in Virginia.

📋 Description

• Design, construct, manage, and sustain Kafka clusters across development, testing, and production settings.

• Oversee Kafka topics, partitions, brokers, replication, retention policies, and access controls.

• Monitor the performance, availability, throughput, and latency of Kafka; proactively identify and address issues.

• Conduct capacity planning, tuning, upgrades, patching, and disaster recovery planning for Kafka environments.

• Implement and uphold high availability and fault-tolerant configurations for Kafka.

• Develop and support event streaming pipelines utilizing Kafka for real-time and near-real-time data processing.

• Integrate Kafka with API Gateway (APIGW) based microservices and downstream backend systems.

• Design and implement Kafka producers, consumers, and connectors (e.g., Kafka Connect) to cater to system integration and ETL/data movement requirements.

• Collaborate with application teams to establish event schemas, topics, and data contracts.

• Ensure dependable message delivery, data integrity, and error handling throughout streaming workflows.

• Enforce Kafka security best practices, encompassing authentication, authorization, encryption in transit, and auditing.

• Guarantee that Kafka implementations adhere to CMS security, data governance, and operational standards.

• Support DevSecOps practices, CI/CD pipelines, and infrastructure-as-code methodologies wherever applicable.

• Engage in incident response, root cause analysis, and operational readiness initiatives.

• Collaborate closely with architects, developers, DevOps engineers, and system administrators to facilitate solution design and delivery.

• Document Kafka architectures, configurations, operational procedures, and integration patterns.

• Offer technical guidance, troubleshooting assistance, and knowledge transfer to internal teams.

⛳️ Requirements

• Bachelor’s degree in Computer Science, Information Technology, Engineering, or a related discipline.

• A minimum of 3 years of experience in developing, administering, and supporting Apache Kafka in enterprise environments.

• Practical experience in managing Kafka clusters, topics, partitions, and event streaming pipelines.

• Familiarity with integrating Kafka with microservices, API Gateways (APIGW), and backend systems.

• Strong grasp of event-driven architectures, messaging patterns, and data streaming principles.

• Experience with Linux-based environments and command-line management.

• Excellent troubleshooting and performance tuning abilities.

• Capability to articulate technical concepts clearly to both technical and non-technical audiences.

🏝️ Benefits

• Competitive salary and performance-based bonuses.

• Comprehensive health, dental, and vision insurance.

• Generous paid time off and flexible work arrangements.

• Opportunities for professional development and continuing education.

• Collaborative and inclusive work environment.