Data Engineer

This is a fully remote position, open to applicants in United States.

📋 Description

• Evaluate and analyze four source datasets for structure, completeness, and quality, and document the results.

• Standardize and normalize addresses from all sources.

• Develop a tiered matching strategy that utilizes deterministic matching initially, followed by probabilistic linkage for remaining records.

• Document the matching process to ensure reproducibility.

• Assign and calibrate confidence scores according to client-approved thresholds.

• Direct records that fall below the threshold to an exception queue with specified reasons.

• Assign SIC and NAICS classifications as established from approved sources.

• Identify duplicate records and those that necessitate manual review.

• Create technical documentation, including a data dictionary, source-to-target mapping, transformation and business rules, and a summary of the methodology.

• Match around 19,000 commercial customer accounts with business license records and county parcel data.

• Enhance matched records with industry classifications and contact information.

• Provide clean files, exception queues, and documentation; no modifications to production systems, integrations, or application development are required.

⛳️ Requirements

• Experience in production environments with record linkage or entity resolution, covering blocking strategies, comparison levels, and match-model calibration.

• Familiarity with tools such as Splink, dedupe, recordlinkage, or a similar defensible solution.

• Experience in US address standardization using libpostal, usaddress, USPS Publication 28 conventions, or equivalent methods.

• Knowledge in handling directional, suffix, and unit-designator elements.

• Proficient in SQL and Python.

• Experience with DuckDB, polars, or pandas.

• Background in data quality assessment, including profiling and duplicate analysis.

• Ability to articulate data issues clearly to a non-technical audience.

• Technical writing experience, including the capability to provide a sample data dictionary, mapping document, or methodology description.

• Ability to employ explainable methods where each match can be traced back to a documented rule or interpretable score.

• Must be a US citizen and located in the US for work.

• Capability to perform all data processing within US-based infrastructure.

• Enrollment in E-Verify; companies must be part of the federal E-Verify program and provide a notarized subcontractor affidavit with E-Verify Company ID upon contract execution.

• Willingness to sign a data confidentiality agreement and adhere to restrictions on sharing, retaining, or using client data outside the engagement.

• Preferred: Knowledge of SIC and NAICS classification systems, including the 2022 NAICS revision and SIC crosswalks.

• Preferred: Experience with county parcel data and basic GIS handling, such as shapefiles, GeoJSON, and spatial joins.

• Preferred: Experience with utility customer information systems, billing data, or municipal government data.

• Preferred: Familiarity with Microsoft Fabric or Azure data tools.

• Preferred: Practical experience using LLMs for classification or exception triage within a documented, reproducible pipeline.

🏝️ Benefits

• Approximately 8-week contract/subcontract engagement.

• Flexible workload of about 12–15 hours per week.

• No travel expected.