Senior Data Engineer – AWS

This is a fully remote position, open to applicants in Brazil.

📋 Description

• Create and implement extensive data architectures within the AWS ecosystem, utilizing the Data Lakehouse approach.

• Act as the technical leader for the team, directing architectural choices and fostering best practices in data engineering.

• Design and advance robust, secure solutions with a focus on high availability, fault recovery, and scalability in data volumes.

• Develop and enhance distributed processing for large data volumes using EMR and Glue.

• Organize storage and analytics consumption tiers, including Redshift and the corporate data lake.

• Enforce data governance and security measures using Lake Formation, which includes cataloging and access controls.

• Lead efforts in cost optimization for data solutions, assessing trade-offs related to architecture, storage, and processing.

• Establish and uphold technical, quality, and documentation standards for the solutions delivered.

• Convert business goals into data architectures through collaboration with both technical and business stakeholders.

⛳️ Requirements

• Demonstrated experience as a Data Engineer working on intricate, large-scale data architectures.

• Proficient hands-on experience with the AWS data ecosystem (Lake Formation, EMR, Redshift, and Glue).

• In-depth understanding of the Data Lakehouse paradigm, with practical experience in designing or refining such architectures.

• Skilled in SQL for analytical queries, data transformation, and performance enhancement.

• Experience utilizing Python for data engineering and distributed processing tasks.

• Knowledgeable in large-scale distributed processing with Apache Spark.

• Advanced expertise in enterprise data governance, including cataloging, lineage, data quality, and access control.

• Proven track record in designing secure, resilient solutions emphasizing availability and fault recovery.

• Experience in cost optimization for cloud-based data environments.

• Capable of serving as a technical reference, leading architectural decisions while mentoring other engineers.

• AWS certifications focused on data or solution architecture.

• Familiarity with transactional table formats such as Apache Iceberg, Delta Lake, or Apache Hudi.

• Experience with infrastructure-as-code (Terraform or CloudFormation) for provisioning data resources.

• Proficient with orchestration tools such as Apache Airflow or AWS Step Functions.

• Understanding of FinOps as it applies to cloud data environments.

• Previous consulting or servicing experience with multiple clients.

• A mindset geared towards technical leadership, with a willingness to coach and develop other engineers.

• A strategic perspective to align technical decisions with business objectives.

• Strong critical thinking skills to assess trade-offs among performance, cost, security, and timelines.

• Excellent communication skills to effectively advocate for architectural decisions to both technical and non-technical audiences.

• Ability to independently drive initiatives from design through to support.

• A sense of ownership regarding the reliability, security, and cost-effectiveness of the solutions delivered.

🏝️ Benefits

• Ongoing training and opportunities for professional growth.

• A work environment and culture founded on ethics, transparency, quality excellence, teamwork, and responsibilities towards economic, social, and environmental aspects, along with strong interpersonal relations and credibility.

• Fully remote position.

• Position is also available to candidates with disabilities.