Senior Data Engineer

This is a fully remote position, open to applicants in Illinois.

📋 Description

• Partner with product managers, data scientists, engineering teams, and program management to establish product features, business deliverables, and strategies for data products.

• Collaborate with business partners, operations, senior management, and others to provide daily operational support.

• Assist in operational reporting, self-service data engineering initiatives, production data pipelines, and the business intelligence suite.

• Engage with diverse stakeholders to collect and comprehend business requirements, evaluate feasibility and impact, and deliver high-quality results promptly.

• Create suitable solutions and suggest alternative methods when necessary.

• Investigate, prototype, and recommend new tools and technologies in cloud data engineering and Generative AI.

• Construct pipelines that handle unstructured data, transform text, integrate models, and validate outputs for Generative AI projects.

• Manage large datasets, optimize database queries, and resolve intricate technical challenges.

• Contribute to multiple projects and demands simultaneously.

• Operate in a dynamic, collaborative, iterative, and agile/scrum environment.

• Apply independent judgment in methods and techniques to achieve results.

• Utilize cutting-edge technologies to acquire, ingest, and transform vast datasets.

• Oversee multiple data projects with planned coordination for timely delivery of quality products.

• Design, develop, and implement scalable and dependable data products and pipelines.

⛳️ Requirements

• Over 8 years of experience in data engineering or related technical fields, including business intelligence and analytics.

• Proficient in designing and constructing scalable and robust data pipelines.

• Strong understanding of the complete software development life cycle.

• Solid grasp of data warehousing principles and methodologies.

• Experience in creating data pipelines and ETL processes.

• Proven experience in developing high-volume data workflows within a cloud setting.

• Familiarity with building data warehouses and business intelligence projects.

• Skills in data cleansing, validation, and wrangling.

• Practical experience with AWS cloud and AWS native technologies such as Glue, Lambda, Kinesis, Lake Formation, S3, DynamoDB, Step Functions, and Athena.

• Hands-on API development experience utilizing AWS API Gateway, CloudWatch, Cognito, ECS, EKS, and Fargate.

• Experience with Business Intelligence tools such as Tableau, Cognos, and ThoughtSpot.

• Knowledge of relational databases like Oracle and Snowflake, including advanced SQL, Tasks/Streams, stored procedures, and performance optimization.

• Hands-on experience with Snowflake Cortex AI, AI SQL functions, or an equivalent production LLM platform.

• Proficient in SQL, PL/SQL, NoSQL, relational databases (RDBMS), database concepts, and dimensional modeling.

• Practical experience in building business logic and data ingestion and transformation workflows with Informatica IICS or similar tools, AWS pipelines, Snowpipe, streaming solutions, Kafka, and other industry-standard tools and processes is preferred.

• Hands-on experience with Python/shell scripting and SQL for creating scalable data pipelines.

• Familiarity with CI/CD tools such as GitHub Actions and/or Jenkins.

• Experience with infrastructure as code using Terraform and/or CloudFormation.

• Experience implementing LLM observability with Arize or a similar platform is advantageous.

• Excellent verbal and written communication skills.

• Strong analytical and problem-solving abilities, especially in Data Warehouse and Big Data environments.

• Capability to manage multiple projects and provide reports concurrently to different stakeholders.

• Meticulous attention to detail and accuracy.

• A Bachelor’s degree in engineering, Computer Science, Statistics, Economics, Mathematics, Finance, or a related quantitative field is preferred.

🏝️ Benefits

• Annual provision of complimentary hotel stays at Hyatt hotels globally.

• Flexible working hours.

• Work-life benefits, including wellbeing initiatives such as a complimentary Headspace subscription.

• Discounts available at the on-site fitness center.

• Global family assistance policy with paid leave following the birth or adoption of a child.

• Financial support for adoption.

• Paid Time Off.

• Medical coverage.

• Dental coverage.

• Vision coverage.

• 401K plan with company matching.

• Incentive awards.

• Annual bonus.