Data Engineer – Fixed Term Contract

This is a fully remote position, open to applicants in United Kingdom.

📋 Description

• Create and develop new data services.

• Oversee the migration of current solutions.

• Ensure the continuous performance and improvement of live data platforms.

• Design and implement data storage solutions and schemas on Azure.

• Construct and maintain scalable data pipelines for data movement, integration, and processing.

• Gather, transform, and load data using Azure Data Factory and Azure Databricks.

• Test and verify data quality and accuracy throughout the pipelines.

• Optimize and monitor pipeline performance, address bottlenecks, and refine processing algorithms.

• Develop and sustain data models and schemas.

• Collaborate with cross-functional teams to supply data for analytics and reporting.

• Ensure compliance with security and privacy requirements, including GDPR, in Azure data processing.

• Work with datasets containing hundreds of millions to billions of rows and conduct data analysis.

• Stay informed about Azure features, updates, and best practices.

• Perform additional tasks assigned by the line manager as necessary.

⛳️ Requirements

• Demonstrated experience in Python programming for data engineering and analytics.

• Strong expertise in SQL, including the Microsoft T-SQL dialect.

• Practical experience with Azure data storage solutions.

• Familiarity with data integration and processing frameworks such as Azure Data Factory and Azure Databricks.

• Comprehensive understanding of data modeling concepts and best practices for schema design.

• Exceptional analytical and problem-solving abilities, with a keen attention to detail.

• Strong communication skills and the ability to work effectively within cross-functional teams.

• Must be eligible to work without employer-sponsored work visa, as Livestock Information Ltd. is unable to sponsor work visas at this moment.

🏝️ Benefits

• A fully remote and flexible working environment.

• 25 days of annual leave, increasing by 1 day for each year of service up to a maximum of 30 days (plus Bank Holidays).

• 1 Privilege Day.

• Life Assurance equivalent to 4 times the annual salary.

• Market-leading pension scheme provided through Legal & General.

• Enhanced Family Friendly Policies.

• Excellent opportunities for learning, training, and career development.

• 24/7 access to our Employee Assistance Programme.

• A diverse and inclusive culture where everyone is respected and valued.