Data Engineer

Your Role:

Apollo is establishing a set of truly innovative data services to propel the business to the next level. We are capturing, transforming, and analysing a vast variety of data through a modern data platform to produce fabulously rich insight for our business. You are the enthusiastic, skilful, and creative Data Engineer we need to make this happen.

This role will be part of our Data Engineering and Application Development team in our data family, getting hands-on with our data and collaborating directly with colleagues across the business to deliver the data, applications, and services needed. You will bring the best of your Data Engineering and solution design skills to deliver this, either through new data pipelines in our data platform, modelling data to maximise its ease of use or using technology to automate key business tasks.

By working directly with the business and project teams, you will be perfect positioned to build relationships to support our operational responsibilities such as maintaining our data quality, timeliness, and spotting opportunities to continually improve the way we transform our data. Where the challenges and requirements need wider support, the rest of the data family will be there to help, backing you up with IT, reporting and data governance skills.

The data space is ever evolving, and along with the team’s application development responsibilities, there will also be opportunities to gain experience and new skills and innovative solutions along the way.

What you will do:

Stakeholder Engagement

• Build relationships with our business colleagues, both as data suppliers and consumers, listening to their requirements and providing the benefit of your design expertise.
• Be comfortable examining and profiling raw data, operational systems, and business processes to get to the essence of what is needed.
• Be able to communicate the Apollo approach to Data Engineering to ensure project teams have a clear idea of the options and solutions available.

Data Solution Design

• Work with our IT teams to design access to our transactional systems and other data sources.
• Model data in line with the medallion architecture principles of our data platform.
• Design ETL pipelines that produce datasets that can easily be consumed by our analytical teams, used as sources for our systems and complement our wider data architecture.
• Be a subject matter expert in the use and management of our data platform tools such as Azure Data Factory, Azure Databricks and Unity Catalogue and SQL Server.
• Apply best practice to ensure our pipelines are transparent, our data lineage is understood, and our data quality is maximised.
• Use any other approved tools and skills required to design automation and user interface applications to complement our data solutions.
• Ensure our data governance and regulatory responsibilities are upheld by keeping our data safe and secure.

Data Solution Delivery

• Be a hands-on developer, using the tools and implementing code in line with good practice and Apollo standards.
• Be an active contributor to our development process, refining code control, quality assurance and testing capability and standards.
• Plan and set expectations for when work will be complete, released and delivered.
• Work in our agile delivery process to ensure changes are built and implemented alongside the rest of the data family’s activity.
• Be comfortable working directly with our wider business colleagues as part of testing, warranty period or user training activities.
• Ensuring they have the knowledge and skills to make full use of our solutions.

Data Operations

• Understand our wider data strategy so that you can spot and recommend engineering opportunities that will improve our current data coverage.
• Work with the rest of the data family to monitor and identify problems with our data sets.
• Work with data stewards/owners to feedback where source data needs to be improved.

What we are looking for:

• Strong experience with ETL/pipeline tools including: Databricks (Administration, Unity Catalog, Python scripting and SQL) Azure Data Factory SQL Server (SSIS, T-SQL, SSAS etc.) General python scripting
• Strong working knowledge of wider Azure components, their management and use in data applications such as Azure Functions, Logic Apps, API Management Working experience with Azure DevOps (Repos, CI/CD etc.)
• Good understanding of medallion architecture patterns
• Good understanding of data modelling and wider data architecture principles
• Good understanding of Data Governance principles and their application including: Microsoft Purview Databricks Unity Catalog Master Data Management (MDM) Data Quality monitoring tools and processes
• Excellent stakeholder management skills including requirements capture.
• Excel/VBA for data presentation