Data Engineer – Abinitio / Ms Azure

Data Engineer – Abinitio / MS Azure

Whitehall Resources currently require an experienced Data Engineer with Abinitio and Azure experience to work with a key client

**Please note this role is INSIDE IR35 and will be a hybrid role with 1-2 days onsite in reading weekly**

Required Experience:
* Should have Telecom Domain and Smart Metering experience
* Accountable for Designing & Developing Cloud Solution for Smart Metering – Strategic Data Reporting which provides the Network Performance Measure reports of service provide’s network to Data and Communication Company (DCC) and Smart Metering service users (Energy Suppliers).
* Understanding and knowledge of Reporting Data Store (RDS) which is required to support the reporting of Data and Communication Company (DCC)
* Should be able to work in the SCRUM team which delivers the ETL, Report and Dashboard solution using MS Azure.
* Closely work with Product owner, Business Analyst to create user stories required for the sprint.
* Create design data flows/pseudo code to enable developers to develop Azure ETL/Power BI Reports based on that
* Provide required design input and artefacts to the Data Engineers and Data Scientist.
* Responsible for the data model and defining Raw, Bronze, Silver and Gold storage layers.
* Developing code using Azure functions, Spark and SQL Stored procedures in Azure SQL Pool
* Doing code review of ETL code developed using Azure Function
* Capacity planning to understand how and when to scale up and scale down
* Developing complex Abinitio graphs for the Smart Metering Reporting.
* Performance tuning of poorly performing Ab Initio graphs/graphs handling huge volume of data.
* Doing high level/low level impact assessment for new change requests.
* Designing/Developing ETL solution considering recovery, stability, reliability, scalability and performance.
* Translating the reporting requirements to Detailed Design.
* Creating Source to Target mapping for various source feeds to target Dimensional Model.
* Producing document/prototype graphs with complex file transformations and handing over to developers to complete development.
* Conducting Design Walkthrough meetings/ Requirement Clarification meetings with all the stake holders
* Providing inputs such as capacity requirement, GDPR compliance related information required for networking, IT Platform, DBA and security teams.
* Coordinating regularly with offshore team and ensure they make progress on the tasks given to them.
* Doing Ab Initio code review to check if the code follows best practices and meets required standards and provide performance improvement suggestions.
* Expertise in ETL tool Ab initio
* Should be Expert in MS Azure
* Azure Analytics
* Azure Function
* Power BI
* Data Factory
* Data Brick
* Expertise in Business intelligence
* Knowledge of Shop For Data
* Knowledge of Data Profiler 3.1.4
* Knowledge of XML Manipulation
* Software development
* Analysis and design
* Unix/Linux/Windows
* Team Leading capabilities

All of our opportunities require that applicants are eligible to work in the specified country/location, unless otherwise stated in the job description.

Whitehall Resourcesare an equal opportunities employer who value a diverse and inclusive working environment. All qualified applicants will receive consideration for employment without regard to race, religion, gender identity or expression, sexual orientation, national origin, pregnancy, disability, age, veteran status, or other characteristics.