Data Engineer

This is a fully remote position, open to applicants in New York.

📋 Description

• Construct extensive and intricate data sets that align with business needs by extracting, transforming, and loading data from diverse sources.

• Offer operational support and troubleshoot existing processes and systems effectively.

• Collaborate closely with architects, solution leads, data owners, Data Scientists, and key stakeholders to facilitate and manage the data platform backlog grooming, while prioritizing new feature requests in preparation for upcoming project activities.

• Implement automation and streamline processes to guarantee high-quality throughput and performance of the overall data and analytics platform.

• Ensure that data extraction, transformation, and loading meet the necessary data security and compliance standards.

• Connect with data source platform leads to acquire both tactical and strategic insights into the data sources required by the Agency Data Services AI/ML, as well as adhere to Data Office standards.

• Develop data tools for data scientists that aid in building and refining models.

⛳️ Requirements

• Bachelor's degree in Computer Science, Data Science, Engineering, or equivalent experience in software/services is required.

• 4+ years of experience with SQL, Snowflake, Databricks, Spark, and other big data technologies; 4+ years utilizing Python, SQL, PySpark, R, or similar languages to manipulate, process, and derive value from large, disparate data sets.

• 4+ years of experience in constructing and optimizing data pipelines, architectures, and datasets to address business inquiries and recognize opportunities for enhancement.

• 2+ years of experience in supporting large-scale data processing and storage using Azure Data Factory, Integration Runtime, Data Lake, Databricks, Spark, Azure ML, and Cosmos DB.

• 2+ years of experience in managing privacy, compliance, and security issues related to data storage and processing; and delivering data solutions within Agile environments.

• 2+ years of experience with software development and CI/CD methodologies and tools for automated infrastructure code and MLOps, as well as designing, implementing, and maintaining automation platforms and tools, including Ansible Tower, Azure, ARM, Terraform Enterprise, Azure DevOps, and GitHub Actions.

• 2+ years of experience with Salesforce FSC and Salesforce Data Cloud.

🏝️ Benefits

• Health insurance

• Paid holidays

• Flexible time off

• 401k retirement savings plan with company match, offering pre-tax and ROTH options

• Dental insurance

• Vision insurance

• Employer-paid disability insurance

• Life insurance and AD&D insurance

• Employee assistance program

• Flexible spending accounts

• Health savings account with employer contributions

• Accident, critical illness, hospital indemnity, and legal assistance

• Adoption assistance

• Domestic partner coverage