This is a fully remote position, open to applicants in Poland.

📋 Description

• Become a part of Neurons Lab as a Data Engineer (part-time).

• Develop the infrastructure beneath the data context layer: handling data ingestion across multiple platforms.

• Execute identity and entity resolution as well as PII management.

• Conduct PII detection, redaction, and retention processes.

• Create monitored data pipelines instead of scripts using Airflow or Step Functions.

• Prepare runbooks for the client's team to ensure smooth operations after handover.

⛳️ Requirements

• Over 4 years of experience in data engineering, specifically with unstructured or semi-structured data (beyond just warehouse modeling).

• Proficient in Python and strong knowledge of SQL.

• Experience with unstructured data pipelines involving transcripts, emails, chats, and documents — including parsing, normalization, and deduplication.

• Knowledge of embedding and retrieval infrastructure: chunking strategies, vector stores (such as pgvector, OpenSearch, Pinecone-class), along with graph store loading.

• Experience in API and connector integration at scale: Google Workspace/M365, Slack, CRM; handling rate limits, pagination, incremental cursors, and webhooks.

• Expertise in entity resolution and record linkage (both deterministic and fuzzy) without a clean shared key.

• Familiarity with orchestration tools like Airflow or Step Functions, ensuring jobs are idempotent and restartable.

• Experience with AWS and/or GCP data stacks; comfortable working within a private or VPC deployment.

• Practical experience in PII detection, redaction, encryption, and retention.

• Strong written English skills; able to create documents for handover and collaborate effectively in a small distributed team.