Data Engineer - Abinitio And Ms Azure

Data Engineer - AbInitio and MS Azure - 6 Month Contract - £480/day - IR35 TBC - Hybrid - 1 Day/week in Reading

I am currently supporting a key client looking for a Data Engineering Consultant with Telecom Domain and Smart Metering Experience.

Experience Required:

• Telecom Domain and Smart Metering expert
  • Accountable for Designing & Developing Cloud Solution for Smart Metering - Strategic Data Reporting which provides the Network Performance Measure reports of service provide's network to Data and Communication Company (DCC) and Smart Metering service users (Energy Suppliers).
  • Understanding and knowledge of Reporting Data Store (RDS) which is required to support the reporting of Data and Communication Company (DCC)
• Should be able to work in the SCRUM team which delivers the ETL, Report and Dashboard solution using MS Azure.
• Closely work with Product owner, Business Analyst to create user stories required for the sprint.
• Create design data flows/pseudo code to enable developers to develop Azure ETL/Power BI Reports based on that
• Provide required design input and artefacts to the Data Engineers and Data Scientist.
• Responsible for the data model and defining Raw, Bronze, Silver and Gold storage layers.
• Developing code using Azure functions, Spark and SQL Stored procedures in Azure SQL Pool
• Doing code review of ETL code developed using Azure Function
• Capacity planning to understand how and when to scale up and scale down
• Developing complex Abinitio graphs for the Smart Metering Reporting.
• Performance tuning of poorly performing Ab Initio graphs/graphs handling huge volume of data.
• Doing high level/low level impact assessment for new change requests.
• Designing/Developing ETL solution considering recovery, stability, reliability, scalability and performance.
• Translating the reporting requirements to Detailed Design.
• Creating Source to Target mapping for various source feeds to target Dimensional Model.
• Producing document/prototype graphs with complex file transformations and handing over to developers to complete development.
• Conducting Design Walkthrough meetings/ Requirement Clarification meetings with all the stake holders
• Providing inputs such as capacity requirement, GDPR compliance related information required for networking, IT Platform, DBA and security teams.
• Coordinating regularly with offshore team and ensure they make progress on the tasks given to them.
• Doing Ab Initio code review to check if the code follows best practices and meets required standards and provide performance improvement suggestions.
• Expertise in ETL tool Ab initio
• Should be Expert in MS Azure
  • Azure Analytics
  • Azure Function
  • Power BI
  • Data Factory
  • Data Brick
• Expertise in Business intelligence
• Knowledge of Shop For Data
• Knowledge of Data Profiler 3.1.4
• Knowledge of XML Manipulation
• Software development
• Analysis and design
• Unix/Linux/Windows

If this role is of interest, please apply below for more information.