Kafka Engineer

This is a fully remote position, open to applicants in Virginia.

📋 Description

• Design, construct, oversee, and maintain Kafka clusters across development, testing, and production settings.

• Manage Kafka topics, partitions, brokers, replication, retention policies, and access controls.

• Monitor Kafka's performance, availability, throughput, and latency, while proactively identifying and resolving issues.

• Conduct capacity planning, tuning, upgrades, patching, and disaster recovery planning for Kafka environments.

• Implement and sustain high availability and fault-tolerant configurations for Kafka.

• Develop and support event streaming pipelines utilizing Kafka for real-time and near-real-time data processing.

• Integrate Kafka with API Gateway (APIGW)–based microservices and downstream backend systems.

• Design and implement Kafka producers, consumers, and connectors (e.g., Kafka Connect) to facilitate system integrations and ETL/data movement requirements.

• Collaborate with application teams to establish event schemas, topics, and data contracts.

• Ensure reliable message delivery, data integrity, and proper error handling throughout streaming workflows.

• Apply Kafka security best practices, including authentication, authorization, encryption during transit, and auditing.

• Ensure that Kafka implementations adhere to CMS security, data governance, and operational standards.

• Support DevSecOps practices, CI/CD pipelines, and infrastructure-as-code methodologies when applicable.

• Participate in incident response, root cause analysis, and operational readiness initiatives.

• Collaborate closely with architects, developers, DevOps engineers, and system administrators to facilitate solution design and delivery.

• Document Kafka architectures, configurations, operational procedures, and integration patterns.

• Provide technical guidance, troubleshooting assistance, and knowledge transfer to internal teams.

⛳️ Requirements

• Bachelor’s degree in Computer Science, Information Technology, Engineering, or a related discipline.

• A minimum of 3 years of experience in developing, administering, and supporting Apache Kafka within enterprise environments.

• Practical experience in managing Kafka clusters, topics, partitions, and event streaming pipelines.

• Experience in integrating Kafka with microservices, API Gateways (APIGW), and backend systems.

• Strong comprehension of event-driven architectures, messaging patterns, and data streaming principles.

• Familiarity with Linux-based environments and command-line administration.

• Strong skills in troubleshooting and performance tuning.

• Ability to effectively communicate technical concepts to both technical and non-technical audiences.

🏝️ Benefits

• Competitive salary and performance-based incentives.

• Comprehensive health, dental, and vision insurance coverage.

• Opportunities for professional development and continuous learning.

• Flexible work hours and remote work options.

• Collaborative and innovative work environment.