AWS Data Architect

This is a fully remote position, open to applicants in United States.

📋 Description

• Develop and uphold a robust data architecture and cloud strategy that is in line with the organization's objectives and requirements.

• Design, execute, and oversee cloud-based data infrastructure on AWS, ensuring it is scalable, reliable, and cost-effective.

• Employ AWS services (S3, Glue, EMR, Redshift, Lambda, Kinesis, MWAA, etc.) to create and enhance data pipelines and storage solutions.

• Advocate for the implementation of data lakehouse architecture and enhance its performance for both analytical and operational tasks.

• Identify gaps and opportunities within the existing system, suggesting and implementing improvements to optimize processes and reduce costs.

• Lead and mentor data engineering teams in developing, maintaining, and refining ETL processes for data ingestion, transformation, and loading.

• Implement real-time data processing solutions utilizing technologies such as Apache Kafka and AWS Kinesis.

• Collaborate with data scientists, business stakeholders, and analysts to ensure data availability and quality, facilitating effective analytics and reporting.

• Use DBT for data modeling and transformation to enhance self-service analytics and support data governance.

• Design and implement data integration solutions for API ingestion, capturing, transforming, and incorporating data from various sources into our data lakehouse.

• Utilize Airbyte and custom APIs to guarantee efficient, reliable, and secure data transfers.

• Oversee data integration pipelines to support both real-time and batch data processing.

• Design, configure, and maintain workflow orchestration using Apache Airflow to automate ETL processes and data pipeline executions.

• Monitor and enhance job scheduling, error handling, and performance of data workflows.

• Enforce data security protocols, access controls, and encryption to protect sensitive information, particularly personally identifiable information (PII).

• Ensure adherence to data privacy regulations and industry standards.

• Work alongside cross-functional teams to comprehend data requirements and deliver data solutions that meet their needs.

• Maintain thorough documentation for data engineering and data architecture processes and solutions.

• Guide the team in establishing cloud infrastructure and automate using tools such as Terraform, CloudFormation, Jenkins, etc.

• Assist the operations team in implementing automated monitoring and alert mechanisms.

⛳️ Requirements

• A Bachelor's degree or higher in a relevant field.

• Over 6 years of demonstrated experience in data engineering, cloud architecture, and AWS services.

• In-depth knowledge of data lakehouse technologies, including Hudi, DBT, Airbyte, Redshift, Glue, Kinesis, and Apache Airflow.

• Strong proficiency in programming languages such as SQL and Python, as well as processing frameworks like PySpark.

• Significant expertise in real-time data processing.

• Excellent analytical and problem-solving abilities.

• Strong communication and teamwork skills.

• A passion for Sports, Gaming, or Entertainment is preferred.

🏝️ Benefits

• Competitive salary and performance-based incentives.

• Opportunities for professional development and continuous learning.

• Flexible work hours and remote working options.

• Comprehensive health benefits and wellness programs.

• Engaging work environment with a focus on collaboration and innovation.