Lead Data Integration Engineer

This is a fully remote position, open to applicants in Texas.

📋 Description

• Collaborate with Technical Architects, Product Owners, and Business teams to convert requirements into technical designs for data modeling and integration.

• Apply expertise in data warehousing, data modeling, and ETL/ELT data processing methodologies.

• Create and develop ETL/ELT pipelines utilizing reusable patterns and frameworks.

• Design and construct efficient SQL queries to process and curate datasets in HANA, Azure, and Snowflake.

• Design and assess data ingestion frameworks using Python, Spark, Azure Data Factory, Snowpipe, and more.

• Develop Data Quality models and ABCR frameworks for data ingestion, validation, curation, and preparation for consumption.

• Evaluate functional domain and business requirements, proactively identifying any gaps before implementing solutions.

• Collaborate with platform teams to design and establish processes for automation in pipeline building, testing, and code migrations.

⛳️ Requirements

• Bachelor's degree in computer science, applied computer science, IT, or a related field.

• 10 years of experience in the offered position or similar roles.

• 10 years of experience in designing, developing, and constructing ETL/ELT pipelines, procedures, and SQL on MPP platforms (HANA, Snowflake, Greenplum, or Teradata).

• 5 years of experience in designing and creating metadata-driven data ingestion frameworks.

• 5 years of experience with SAP BO/Data Services, Azure Data Factory, SnowSQL, and Snowpipe.

• 5 years of experience with mini-batch, real-time, and event-driven data processing tasks.

• 5 years of experience in designing and developing object stores (including Azure ADLS, HDFS, and GCP Cloud Storage).

• 5 years of experience with Row and Columnar databases (including Azure SQL DW, SQL Server, Snowflake, Teradata, PostgresSQL, and Oracle).

• 5 years of experience with NoSQL databases (including CosmosDB, MongoDB, and Cassandra).

• 5 years of experience with ElasticSearch, Redis, and data processing platforms (Spark, Databricks, and SnowSQL) along with analytics development (Azure Stream Analytics, Azure Analysis Services, Data Lake Analytics, HDInsight, HDP, Spark, Databricks, MapReduce, Pig, Hive, Tez, SSAS, Watson Analytics, and SPSS for Data Analytics development).

🏝️ Benefits

• Comprehensive benefits package.

• 401(k) plan.

• Profit Sharing plan.

• Eligibility for target bonuses.