Data Engineer - Stoke On Trent

At Dee Set we are undergoing an exciting digital transformation that includes building a strong Data Science and Analytics team to drive data integration, modelling and inference utilising the best-in-class Data Analytics, Engineering and Machine Learning tools and techniques.
The role will require working within the Data Science and Analytics team and with a range of business functional teams, including Tech and Change. You will focus on development and maintenance of the end-to-end data pipelines and data warehouse, implementations of systems and standards for service quality, maintenance of code repositories. You will be a strong independent problem solver capable of working through complex problems with minimal guidance. You will build and manage the data infrastructure and environments and implement best practice for the storage and processing of large-scale data sets, enabling the team to succeed.
Key Accountabilities and Responsibilities:
Ingestion of data from source systems, identifying and resolving data quality issues
Design and develop ETL processes to create robust data assets
Design, build and maintain the end-to-end data pipelines and data models to serve analytics solutions
Automation of pipelines
Support to the business in the implementation and usage of Data Engineering applications and tools
Taking ownership of our data lake and warehouse platform
Take responsibility for the quality of the code base ensuring that the data is reliable and matches the user needs
Collecting and analysing user requests and estimating required effort
Be a strong advocate for quality, ensuring that data and common repositories of code are delivered and maintained to a high standard
WHAT YOU'LL NEED
We are looking for an outstanding problem solver with an inquisitive mind, good eye for details and real hands-on experience. You will love working with data and documentation. Having self-motivation to learn and keep pace with numerous analytics and software tools with an ambition to become an expert in applying such technologies is essential.
Knowledge, Skills and Experience required:
Knowledge of the processes, tools and techniques for extracting, joining, cleaning and shaping data from multiple sources for subsequent analysis, ability to create robust data models to help drive business insights.
Collect and translate the business requirements for analytics solutions into required data structures
Implement and operate multiple types of data structures and troubleshoot complex problems associated with these structures
Anticipate underlying problems and limitations in the data structures
Design and develop the Data Engineering steps required to provide the necessary data using complex queries and transforms (MS-SQL, Python, Snowflake, Azure services: Blob storage, Cosmos DB, Elasticsearch, etc.)
Orchestrate data pipeline steps into logical executable sequence (preferably using Matillion)
Identify reasonable timescales and data owner(s) input need for data transformation exercise
Utilise a systematic approach to projects and tasks by using the most efficient means of generating consistent and optimum results
Optimise the use of compute and storage in Data Engineering platform
Optimise query, design efficient code, perform code reviews, utilise version control and collaborative development (GitLab, Snowflake, Azure, Python)
Incorporate design, build, test and defect resolution methodology and operational documentation at the Development stage
Desirable Skills and Competencies:
Experience building and maintaining a live cloud data lake and data warehouse
Star-Schema warehouse design and implementation
Experience using the Matillion, Snowflake and Azure ecosystems
CI/CD experience
Machine learning model deployment experience is a plus
WHY US?
Bring your expertise and realise your potential to further your career with a world-class team. This is a brilliant opportunity to join the market leading and award-winning Dee Set Group and play a key role in delivering the digital transformation. We offer flexible and remote working, opportunities to learn and grow professionally and personally, access to money-saving perks via our Colleague Hub and a competitive pension plan.
With an AI-powered recruitment process we give you the best possible application experience and provide personalised feedback.
We are proud of our fairness and equality and are committed to ensuring equal opportunities for all.
This makes Dee Set a perfect place to continue your journey. We strongly advise you to submit your application as early as possible.
Closing date for applications is 8th January 2024