Senior Data Engineer

This is a fully remote position, open to applicants in Texas.

📋 Description

• Create scalable data pipelines and establish new integrations to manage the increasing data volume and complexity.

• Develop a comprehensive Data Lakehouse to support Vericast's marketing solutions and financial institution clients.

• Assist with pre-sales activities, execute campaigns, analyze campaign performance, and conduct benchmarking.

• Work collaboratively with AI, Analytics, and business teams to enhance data models that feed into AI and analytics tools.

• Implement mechanisms to monitor data quality, ensuring the availability and accuracy of production data.

• Conduct data analysis to identify and resolve data-related issues effectively.

• Offer post-deployment support and swiftly address any unforeseen production challenges.

• Collaborate with business units and engineering teams to define the long-term strategy for the data platform architecture.

⛳️ Requirements

• A Bachelor's degree in Computer Science, Information Technology, or a related discipline with over 5 years of applicable experience; or a Master's degree in Computer Science, Information Technology, or a related field (preferred).

• More than 5 years of experience in Data Engineering, ETL development, or Data Platform Engineering.

• Familiarity with environments such as Financial Institutions (FI), Banking, FinTech, MarTech, AdTech, Marketing Services, or Marketing Agencies.

• Proven experience managing customer, campaign, audience, marketing performance, attribution, advertising, or large-scale transactional data.

• Solid expertise in Python and PySpark for creating production-quality data pipelines.

• Background in designing, building, and maintaining Data Lakehouse environments.

• Proficient with tools including Apache Airflow, Iceberg, Hive, S3, and Trino.

• Experience with cloud platforms like AWS, Azure, or GCP.

• Knowledge of Agile software development methodologies.

• Familiarity with GitLab and CI/CD processes.

• Experience in supporting AI and machine learning applications.

• Understanding of machine learning models and the data needs of Data Science teams.

• Strong programming capabilities in object-oriented/functional scripting languages such as Python and PySpark, including testing, logging, and data observability skills.

• Experience with distributed systems and parallel data processing using Spark, PySpark, Hadoop, Kafka, and Hive.

• Proficient with relational databases.

• Solid understanding of Linux/Unix-based systems.

• Hands-on experience with Apache Ranger and Rancher/Kubernetes.

• Preferred: Experience in developing REST APIs.

• Exceptional analytical, conceptual, and problem-solving abilities.

• Strong communication skills that facilitate collaboration across teams.

🏝️ Benefits

• Medical, dental, and vision insurance coverage.

• 401K retirement plan.

• Generous paid time off (PTO) policy.

• Life insurance coverage.

• Employee assistance programs.

• Pet insurance options.