
Big Data Lead
Posted May 30

Posted May 30
This is a fully remote position, open to applicants in India.
• Integrate a hands-on Subject Matter Expert (SME) to utilize big data technologies for addressing intricate data challenges.
• Allocate nearly half of the time to practical coding tasks.
• Engage in extensive text data processing, create event-driven data pipelines, and perform in-memory computations while optimizing for CPU core, network I/O, and disk I/O.
• Leverage cloud-native services available in AWS and GCP.
• Over 7 years of practical experience in Software Development with an emphasis on big data and extensive data pipelines.
• At least 3 years of experience in developing services and pipelines using Python.
• Proficiency with various data processing systems, including streaming, event, and batch processing (such as Spark and Hadoop/MapReduce).
• Familiarity with at least one NoSQL database like MongoDB, Elasticsearch, or HBase.
• Knowledge of data models, sharding, and data location strategies for distributed data stores within large-scale, high-throughput, and high-availability environments, particularly in relation to non-structured text data processing.
• Experience in managing scalable and highly available systems using AWS or GCP.
• Preferred: Experience with Docker and Kubernetes.
• Exposure to Continuous Integration/Continuous Deployment (CI/CD).
• Understanding of Crawling and Scraping techniques.
• Fully remote work opportunity.
• Birthday leave policy.
• Flexible remote working arrangements.
AVENCORE
Smadex
ShipBob, Inc.
Get handpicked remote jobs straight to your inbox weekly.