Data Engineer

This is a fully remote position, open to applicants in United States.

📋 Description

• Design, develop, and maintain Azure Databricks pipelines for the ingestion, transformation, integration, and downstream consumption of credit card data.

• Create scalable processing solutions using PySpark, Python, Spark SQL, SQL, Delta Lake, and ADLS to handle high-volume card datasets.

• Develop transformation logic for data related to cardholders, accounts, transactions, authorizations, payments, balances, fees, interest, rewards, and servicing.

• Convert business requirements and source-to-target mappings into reusable Databricks notebooks, workflows, jobs, and ETL/ELT processes.

• Assist in the modernization and conversion of credit card platforms, encompassing data mapping, historical/incremental loads, transformations, and reconciliation.

• Establish automated controls for data quality, balancing, reconciliation, cleansing, and exception handling across both source and target environments.

• Safeguard sensitive cardholder information, including PAN and PCI-sensitive data, through masking, tokenization, encryption, and appropriate data-handling practices.

• Diagnose and enhance Databricks pipelines for optimal performance.

• Collaborate with credit card subject matter experts, architects, analysts, and application teams.

• Forge innovative solutions and build relationships with teammates and clients while leveraging global capabilities.

⛳️ Requirements

• Over 5 years of experience in Data Engineering, ETL/Data Warehouse, or Big Data, including more than 3 years of practical Azure Databricks development.

• Significant hands-on experience with Databricks, PySpark, Python, Spark SQL, SQL, Delta Lake, and ADLS.

• Proven track record in designing and developing scalable ETL/ELT pipelines for large-volume transactional and financial datasets.

• Extensive experience with data transformation, source-to-target mapping, cleansing, reconciliation, data quality, and exception handling.

• Background in the credit card/payments domain, with knowledge of card accounts, transactions, authorizations, payments, balances, billing, and settlement processes.

• Familiarity with Azure DevOps, Git, CI/CD, and Agile/Scrum development methodologies.

• Strong experience in banking and financial services with a solid understanding of financial data controls and reconciliation processes.

• Preferred experience with TSYS or similar major credit card processing platforms.

• Prior experience supporting credit card conversions, migrations, or platform modernization is preferred.

• Familiarity with the complete card lifecycle is advantageous.

• Experience with Mastercard and/or Visa data, interfaces, reporting, rewards, or loyalty programs is preferred.

• Understanding of PCI DSS, PAN protection, tokenization/detokenization, encryption, and management of sensitive cardholder data is preferred.

• Experience with advanced Databricks features such as Unity Catalog, Workflows, Auto Loader, and Lakeflow/Delta Live Tables is preferred.

• Experience in modernizing Informatica PowerCenter or SSIS workloads to Databricks/PySpark is preferred.

• Azure and/or Databricks certifications and experience with large-scale banking conversion initiatives are preferred.

• Ability to successfully pass a background investigation.

• Qualified candidates must satisfy applicable U.S. employment requirements.

🏝️ Benefits

• Competitive compensation package.

• Comprehensive insurance options available.

• Matching contributions through the 401(k) plan and share purchase plan.

• Paid time off for vacations, holidays, and sick leave.

• Paid parental leave provided.

• Opportunities for learning and tuition assistance.

• Wellness and well-being programs offered.

• Ownership opportunity from day one through the CGI Partners model.

• Access to global capabilities and industry and technology expertise.

• Career development and skill-building opportunities available.

• Reasonable accommodations for qualified individuals with disabilities.