Data Operations Engineer II

This is a fully remote position, open to applicants in Pennsylvania.

📋 Description

• Develop, oversee, and sustain data ingestion processes and systems across various platforms, including Databricks, the Data Marketplace, and Ad Platform.

• Create, manage, and enhance Python, PowerShell, Bash, or similar scripts for automated data collection and monitoring.

• Take ownership of the operational state of designated data pipelines, which includes monitoring, incident response, maintenance, and escalation.

• Manage SQL databases for data loading and deployment purposes.

• Create and uphold unit test coverage for Data Ops codebases.

• Administer GoAnywhere and other managed file transfer platforms effectively.

• Develop and oversee comprehensive monitoring systems and automated alert mechanisms.

• Diagnose and resolve incidents, identify root causes, implement solutions, validate recovery, and document resolutions.

• Construct and maintain CI/CD pipelines utilizing GitHub Actions and Azure DevOps.

• Carry out scheduled maintenance, patches, updates, and preventive operational checks.

• Document scripts, automation, monitoring workflows, data pipelines, and deployment procedures meticulously.

• Enhance runbooks based on previous incidents.

• Collaborate with cross-functional teams regarding scripting needs and automation strategies.

• Plan and prioritize tasks independently, share knowledge, and provide informal coaching.

⛳️ Requirements

• Bachelor’s degree in Computer Science, Information Technology, or a related field is required.

• A minimum of 3 years of experience in data operations, scripting, and automation.

• Intermediate proficiency in Python, PowerShell, Bash, or similar scripting languages is necessary.

• Fully competent in Azure, Databricks, Snowflake, and SQL Server.

• Experience in managing relational databases such as SQL is essential.

• Preferred: proficiency in object-oriented programming languages like C++, Java, or similar.

• Preferred: familiarity with DataDog or similar monitoring and performance-tracking tools.

• Preferred: experience with CI/CD deployment automation tools and processes.

• Preferred: administrative experience with managed file transfer tools such as GoAnywhere.

• Preferred: experience with cloud platforms such as AWS, Azure, or GCP.

• Preferred: knowledge of version control systems such as Git.

• Exceptional analytical and problem-solving abilities.

• Capacity to debug systems and trace issues back to their source.

• Excellent communication and collaboration skills are required.

• Ability to adapt to changing priorities and manage multiple tasks effectively.

• Ability to provide clear incident updates on a defined schedule during active incidents.

🏝️ Benefits

• Option for remote work.

• Commitment to equal employment opportunity.

• Reasonable accommodations available for individuals with disabilities.