Address
Form of workWe are currently recruiting for a Data Engineer to work for a well respected utilities client on a 6 month contract
Data Engineer
Contract
Inside IR35
6 Months (most likely to extend to 12)
Hybrid working (Mostly remote with occasional visit to site)
(Must be Scotland Based)
Project
We have a large number of data products that have been developed over the last 4 years within a traditional data warehouse model, with our ETL being carried out by Azure Data Factory and SQL stored procedures on an Azure SQLServer DB. This has given us a reasonable batch loaded platform that served us well for traditional Business Insights use cases, which typically we publish via a self service PowerBI strategy.
However we recognise now that we need a big shift towards faster data pipelines that can serve greater workloads and operational use cases. We also recognise that our current solution stores the data twice(in a data lake staging area and a SQL modelled area) which is costly and makes governance overly cumbersome.
The Lakehouse approach will be faster, more easily governed and better value for money.
Role
The role will be to be an accomplished Data Engineer who would describe Databricks as their main skillset. In particular building pipelines using Python notebooks. You will be able to recognise everything that is happening within SQL stored procedures, and convert this where necessary but also re-write this where you see it being advantageous. You will have strong data modelling skills, be expert in DevOps and using CI pipelines to test & deploy your code. You will be working alongside an in-house team of 4 other Data Engineers, who are used to working in a highly collaborative agile scrum approach. Learning from each other is expected
**If this role is of interest, please submit an up to date CV highlighting all relevant experience clearly**
Rullion celebrates and supports diversity and is committed to ensuring equal opportunities for both employees and applicants.
