Senior AWS Data Engineer

This is a fully remote position, open to applicants in India.

📋 Description

• Oversee data engineering initiatives, ensuring they align with business goals.

• Offer strategic insights on best practices in data engineering.

• Supervise a team of data engineers.

• Promote continuous enhancement of data processes.

• Design, construct, and uphold efficient, reusable, and dependable architecture and code for data pipelines and applications on AWS.

• Create robust data ingestion pipelines (from on-premises to AWS and within AWS) leveraging AWS services like Glue, Redshift, S3, Lambda, EMR/Spark, Kinesis, and SQS.

• Develop and oversee ETL/ELT processes to gather, process, and store data from various sources, ensuring data quality, integrity, and security.

• Architect and implement comprehensive data solutions (ingestion, storage, integration, processing, access) on AWS, emphasizing data lakes and warehouses.

• Engage in architectural and system design discussions for large-scale data engineering projects.

• Execute hands-on development, unit testing, and participate in code reviews to ensure compliance with best practices.

• Develop serverless applications using AWS Lambda, API Gateway, Step Functions, and other AWS technologies.

• Transition data from traditional relational databases, file systems, and APIs to AWS-based data lakes (S3), RDS, Aurora, and Redshift.

• Implement high-velocity streaming solutions utilizing Amazon Kinesis, SQS, and preferably Kafka.

• Design and execute CI/CD strategies for enterprise data platforms.

• Collaborate with product teams, operations, QA, and cross-functional groups throughout the software development lifecycle.

• Keep updated with emerging technology trends, execute POCs for new tools/technologies, and integrate them for practical applications.

• Identify and address performance challenges while continuously optimizing for cost, reliability, and scalability.

⛳️ Requirements

• Bachelor’s degree in Computer Science, Software Engineering, MIS, or equivalent education and experience.

• 5+ years of experience in implementing and supporting data lakes, data warehouses, and data applications on AWS for large organizations.

• Proficient programming skills in Python, Shell scripting, and SQL.

• Strong experience with AWS services: CloudFormation, S3, Athena, Glue, EMR/Spark, RDS, Redshift, DynamoDB, Lambda, Step Functions, IAM, KMS, and Secrets Manager.

• Experience in serverless application development and orchestration of data pipelines.

• Background in system analysis, design, development, and implementation of data ingestion pipelines in AWS.

• Understanding of ETL/ELT processes, data modeling, and big data technologies.

• Familiarity with data warehousing principles and cloud-based architecture.

• Strong analytical skills and meticulous attention to detail.

• Excellent communication and collaboration skills.

🏝️ Benefits

• Work From Home