Senior Data Ingestion Engineer

This is a fully remote position, open to applicants in Poland, +4 more countries.

📋 Description

• Create and execute scalable ingestion pipelines for a large volume of both structured and unstructured insurance documents.

• Process various file types including PDFs, scans, emails, Word, Excel, and PowerPoint documents.

• Conduct OCR/document extraction, as well as text cleaning and normalization.

• Implement semantic chunking and apply metadata tagging.

• Prepare and store vector data for Retrieval-Augmented Generation (RAG).

• Validate extraction results and oversee automated error monitoring.

• Connect ingestion pipelines with downstream AI solutions and document sources.

⛳️ Requirements

• Minimum of 7 years of experience as a Data Engineer, proficient in Python and SQL.

• At least 5 years of experience working with AWS.

• Strong proficiency in Python, SQL, AWS, S3, Step Functions, and CloudWatch.

• Experience in building data/document ingestion pipelines.

• Familiarity with processing unstructured documents such as PDFs, scanned files, Word, Excel, PowerPoint, and emails.

• Knowledge of OCR/document extraction technologies, including AWS Textract or similar tools.

• Experience in developing connectors/integrations with SharePoint and email sources.

• Proficient in text cleaning and normalization techniques.

• Experience with metadata extraction and tagging processes.

• Background in automated error monitoring.

• Proven ability to validate OCR/extraction results.

• Experienced with Git, CI/CD, and testing methodologies.

• Practical experience in public-cloud data processing and document ingestion.

• Familiarity with vector databases, RAG, semantic chunking, vector-storage schema design, and retrieval mechanisms is a plus.

• Knowledge of Azure and Databricks is advantageous.

• Specific experience with AWS Textract is considered a plus.

• Previous experience in the insurance or financial services sector is beneficial.

• Experience in managing enterprise security and low-latency SLA requirements is desirable.

• Proficiency in English at Level C1.

• Duration of the mission: 1 year.

🏝️ Benefits

• Competitive salary.

• Opportunity to work with cutting-edge technology.

• Collaborative and dynamic work environment.

• Professional development opportunities.