Address
Form of workWe are Smart DCC
Together we're building a greener tomorrow
Connecting every home and business to a single, secure smart meter network is a huge undertaking. So you can imagine the scale – and variety – and opportunities for the people who'll help us do it.
Together we're building a greener tomorrow
Connecting every home and business to a single, secure smart meter network is a huge undertaking. So you can imagine the scale – and variety – and opportunities for the people who'll help us do it.
The role
DataOps Engineer
PERMANENT ROLE WITH SMART DCC
Based in Manchester
Competitive Salary plus benefits
DataOps Engineer
PERMANENT ROLE WITH SMART DCC
Based in Manchester
Competitive Salary plus benefits
Role
In our Data and Analytics team, we're focused on using data to drive our organization forward. Our organization is in search of a DataOps Engineer, tasked with revolutionizing and streamlining our data workflows, pipelines, and architectures to bolster our advanced analytics, machine learning, and artificial intelligence efforts. In this critical role, you're expected to harness your engineering prowess to enhance data process efficiencies, ensuring data is seamlessly, reliably, and efficiently managed across our analytics platforms. Your contributions will be vital in enabling quick access to insights, promoting a culture of continuous improvement, and driving innovation within our data practices.
As a DataOps Engineer, your responsibilities will span the development and implementation of automated solutions for data integration, quality control, and continuous delivery. This role demands a solid grounding in software engineering principles, fluency in programming languages such as Python or Scala, and an adeptness with DevOps tools. You'll play a crucial role in constructing and maintaining sophisticated data pipelines that support the organization's data science and analytics ambitions.
Collaboration is a cornerstone of this position. You will work closely with teams across the organization, assimilating their data requirements and challenges, and crafting agile, robust data solutions. Your efforts in implementing best practices in DataOps will aim to eliminate bottlenecks, elevate data quality, and ensure that data management processes are in tight alignment with our strategic analytics and decision-making objectives.
In this role, automating data pipelines and implementing scalable solutions will be just the beginning. You will also ensure data availability and integrity through effective governance, advocate for DataOps methodologies alongside IT and data teams, and continuously monitor, troubleshoot, and optimize data systems for superior performance.
This DataOps Engineer position is perfectly suited for those with a passion for leveraging technology to tackle data challenges, a desire to enhance data analytics capabilities, and a drive to influence business outcomes through streamlined data operations. If you're eager to contribute to the data infrastructure backbone of our organization and possess a robust background in software engineering and data pipeline automation, this role offers a compelling opportunity to advance within our Data and Analytics team.
In our Data and Analytics team, we're focused on using data to drive our organization forward. Our organization is in search of a DataOps Engineer, tasked with revolutionizing and streamlining our data workflows, pipelines, and architectures to bolster our advanced analytics, machine learning, and artificial intelligence efforts. In this critical role, you're expected to harness your engineering prowess to enhance data process efficiencies, ensuring data is seamlessly, reliably, and efficiently managed across our analytics platforms. Your contributions will be vital in enabling quick access to insights, promoting a culture of continuous improvement, and driving innovation within our data practices.
As a DataOps Engineer, your responsibilities will span the development and implementation of automated solutions for data integration, quality control, and continuous delivery. This role demands a solid grounding in software engineering principles, fluency in programming languages such as Python or Scala, and an adeptness with DevOps tools. You'll play a crucial role in constructing and maintaining sophisticated data pipelines that support the organization's data science and analytics ambitions.
Collaboration is a cornerstone of this position. You will work closely with teams across the organization, assimilating their data requirements and challenges, and crafting agile, robust data solutions. Your efforts in implementing best practices in DataOps will aim to eliminate bottlenecks, elevate data quality, and ensure that data management processes are in tight alignment with our strategic analytics and decision-making objectives.
In this role, automating data pipelines and implementing scalable solutions will be just the beginning. You will also ensure data availability and integrity through effective governance, advocate for DataOps methodologies alongside IT and data teams, and continuously monitor, troubleshoot, and optimize data systems for superior performance.
This DataOps Engineer position is perfectly suited for those with a passion for leveraging technology to tackle data challenges, a desire to enhance data analytics capabilities, and a drive to influence business outcomes through streamlined data operations. If you're eager to contribute to the data infrastructure backbone of our organization and possess a robust background in software engineering and data pipeline automation, this role offers a compelling opportunity to advance within our Data and Analytics team.
What will you be doing?
At the DCC, we believe in making Britain more connected, so we can all lead smarter, greener lives. That desire to make a difference is what drives us every day and it wouldn’t be possible without our people. Each person at the DCC brings a special kind of power to the business, and if you join us, we’ll give you the means to unleash yours. Here, we depend on each other and hold each other accountable. You have the power to challenge and make change, to take the initiative and enjoy real responsibility. Whether it’s doing purposeful work, helping us grow or building the career you want – we’ll give you the support to do it all. Our secure network for smart meters is transforming Britain’s energy system and helping the country’s fight against climate change: we want you to be part of our journey.
- Data Pipeline Development:
- Detailed ETL Expertise: Design and implement complex ETL processes to efficiently extract, transform, and load data from disparate sources into target data stores, ensuring data quality and integrity.
- Streaming Data Processing: Develop real-time data processing pipelines using technologies like Apache Kafka or cloud-native streaming platforms, enabling low-latency data ingestion and analysis for time-sensitive applications.
- Batch Processing Optimization: Optimize batch processing workflows using distributed computing frameworks such as Apache Spark or Apache Flink, fine-tuning parallelism, resource allocation, and data partitioning strategies for performance and scalability.
- Infrastructure Automation:
- Infrastructure as Code (IaC): Implement infrastructure automation using tools like Terraform or Ansible to provision, configure, and manage cloud resources, data clusters, and containerized environments, ensuring reproducibility and consistency across environments.
- Adopting Cloud-Native Services: Evaluate, select, and integrate cloud-native services and technologies (e.g., AWS, Azure) to address DataOps challenges efficiently and cost-effectively to reduce effort and improve consistency.
- Integration with Data Lakes and Data Warehouses: Integrate cloud-native data services with existing data lakes, data warehouses, and analytics platforms to establish seamless data pipelines, support hybrid architectures, and enable cross-cloud data integration for multi-cloud deployments.
- Continuous Integration and Deployment (CI/CD):
- Automated Testing: Develop automated test suites and integration tests for data pipelines and analytics applications, validating data transformations, business logic, and data quality constraints throughout the CI/CD pipeline.
- Pipeline Orchestration: Orchestrate CI/CD pipelines using tools like Jenkins, GitLab CI/CD, or Apache Airflow, automating build, test, and deployment tasks for data engineering artifacts and workflows.
- Monitoring and Alerting:
- Comprehensive Monitoring: Implement comprehensive monitoring solutions using tools like Prometheus, Grafana, or Datadog to monitor data pipeline health, performance metrics, and system resource utilization in real-time.
- Anomaly Detection: Configure anomaly detection algorithms and threshold-based alerting mechanisms to proactively identify and address issues such as data latency, pipeline failures, or resource contention.
- DevOps Collaboration:
- Cross-functional Collaboration: Collaborate with DevOps teams, data engineers, data scientists, and business stakeholders to align DataOps practices with broader DevOps principles, fostering a culture of collaboration, transparency, and shared responsibility across teams.
- Agile Methodologies: Embrace agile methodologies such as Scrum or Kanban, participating in sprint planning, backlog grooming, and retrospective meetings to prioritize tasks, iterate on development efforts, and adapt to evolving requirements.
- Performance Optimization:
- Query Optimization: Profile and optimize SQL queries, data processing workflows, and distributed computing jobs to improve query performance, reduce latency, and minimize resource utilization across data infrastructure components.
- Resource Management: Monitor and optimize resource allocation, container resource limits, and cluster configurations to maximize resource utilization, minimize costs, and ensure optimal performance under varying workloads.
- Continuous Improvement:
- Professional Development: Stay abreast of industry trends, emerging technologies, and best practices in DataOps through continuous learning, participation in conferences, webinars, and online communities, and pursuing relevant certifications.
- Process Optimization: Identify opportunities for process improvements, tool enhancements, and automation initiatives to streamline DataOps workflows, reduce cycle times, and enhance productivity, leveraging feedback from stakeholders and retrospectives to drive iterative improvements.
- Advanced proficiency in database technologies such as SQL Server, Oracle, MySQL, or PostgreSQL for data management and querying.
- Expertise in implementing and managing data pipelines.
- Strong understanding of data warehousing concepts, data modelling techniques, and schema design for building and maintaining data warehouses or data lakes.
- Proficiency in cloud platforms such as AWS, Azure, or Google Cloud for deploying and managing scalable data infrastructure and services.
- Knowledge of DevOps principles and practices for automating infrastructure provisioning, configuration management, and continuous integration/continuous deployment (CI/CD) pipelines.
- Strong scripting and programming skills in languages like Python, Bash, or PowerShell for automation, data manipulation, and orchestration tasks.
- Ability to collaborate with cross-functional teams including data engineers, data scientists, and business stakeholders to understand requirements, design data solutions, and deliver projects.
- Excellent communication skills to effectively convey technical concepts to non-technical stakeholders and collaborate with team members.
- Strong problem-solving skills to troubleshoot data issues, optimize performance, and improve reliability of data pipelines and infrastructure.
- Ability to stay updated with emerging technologies, trends, and best practices in the field of DataOps and data engineering.
- Initiative and drive to continuously improve skills, automate repetitive tasks, and streamline data operations processes for increased efficiency and productivity.
- University degree or equivalent experience in business or a STEM subject
- Experience in a relevant regulated industry.
At the DCC, we believe in making Britain more connected, so we can all lead smarter, greener lives. That desire to make a difference is what drives us every day and it wouldn’t be possible without our people. Each person at the DCC brings a special kind of power to the business, and if you join us, we’ll give you the means to unleash yours. Here, we depend on each other and hold each other accountable. You have the power to challenge and make change, to take the initiative and enjoy real responsibility. Whether it’s doing purposeful work, helping us grow or building the career you want – we’ll give you the support to do it all. Our secure network for smart meters is transforming Britain’s energy system and helping the country’s fight against climate change: we want you to be part of our journey.
Company benefits:
The DCC’s continued success depends on our people. It’s important to us that you enjoy coming to work, and feel healthy, happy and rewarded. In this role, you’ll have access to a range of benefits which you can choose from to create a personalized plan unique to your lifestyle.
If there are any questions you’d like to ask before ing, please contact Stephanie.Owen@peregrineresourcing.comcomplete your application, so we can learn more about you. Your application will be carefully considered, and you’ll hear from us regarding its progress.
Join the DCC and discover the power of you.
The DCC’s continued success depends on our people. It’s important to us that you enjoy coming to work, and feel healthy, happy and rewarded. In this role, you’ll have access to a range of benefits which you can choose from to create a personalized plan unique to your lifestyle.
If there are any questions you’d like to ask before ing, please contact Stephanie.Owen@peregrineresourcing.comcomplete your application, so we can learn more about you. Your application will be carefully considered, and you’ll hear from us regarding its progress.
Join the DCC and discover the power of you.
What to do now
Choose ‘ now’ to fill out our short application, so that we can find out more about you. If you have any questions you’d like to ask before ing, please contact Stephanie.Owen@peregrineresourcing.com
Choose ‘ now’ to fill out our short application, so that we can find out more about you. If you have any questions you’d like to ask before ing, please contact Stephanie.Owen@peregrineresourcing.com
Capita Opportunity Statement
The parent company, Capita Plc*, are a leading UK provider of technology enabled business services. We’re supporting and improving the lives of millions of people every day and we can only do this with the right people in place, working towards a shared goal. We encourage an open, honest working environment where everyone can be true to themselves, and people are valued for their differences. We’re always challenging each other to learn and improve, because we know when we work together, we can deliver better outcomes. We work across such a huge range of businesses and sectors, that you’ll have the opportunity to grow and develop your career in any number of directions. You’ll also become part of a network of 63,000 experienced, innovative, and dedicated individuals across multiple disciplines and sectors. There are countless opportunities to learn new skills and develop in your career, and we’ll provide the support you need to do deliver. Our purpose is to create a better outcome for you.
Smart together
Our values and behaviours
Make a difference
We believe in our purpose. Together, we work as one team, always looking for ways to improve and support DCC’s role in a low-carbon future.
Act with integrity
We treat everyone with kindness, consideration and respect. We keep our promises to our customers. And if something doesn’t feel right, we speak up.
Be accountable
You can depend on our word. We’re clear about our roles and we take the initiative, while respecting the processes we have in place.
Rewards & Benefits
Get perks that work
Personal Development
Our training and development opportunities help you be your best self – from regular 1-2-1s all the way up to professional qualifications.
Group income protection
Provides a continuous income if you're unable to work due to illness or injury for more than 26 weeks.
Pension Plans
Our Capita Choice partners will match the salary contributions you make to your pension up to 5%, depending on your plan and time with us.
Flexible Holidays
Need more time off? Increase your standard 25 days of leave by buying extra to rest, spend time with family or satisfy your wanderlust.
Cycle to Work Scheme
We’re big on keeping things green. If you want to cycle to work, we'll loan you the gear through salary sacrifice to make it tax-efficient.
Flexible working
As we all transition to a new normal, we can help you find the hybrid working pattern that suits you and your team best.
Additional Benefits
Add more to your core
Unmind
Travel season ticket loan
Bonus
Medical cash plans
Life cover
Charity days
Childcare vouchers
Health screening
RAC discount
Vision Express discounts
Mobile phone discounts
Discounted gym membership
Retail and leisure discounts
The parent company, Capita Plc*, are a leading UK provider of technology enabled business services. We’re supporting and improving the lives of millions of people every day and we can only do this with the right people in place, working towards a shared goal. We encourage an open, honest working environment where everyone can be true to themselves, and people are valued for their differences. We’re always challenging each other to learn and improve, because we know when we work together, we can deliver better outcomes. We work across such a huge range of businesses and sectors, that you’ll have the opportunity to grow and develop your career in any number of directions. You’ll also become part of a network of 63,000 experienced, innovative, and dedicated individuals across multiple disciplines and sectors. There are countless opportunities to learn new skills and develop in your career, and we’ll provide the support you need to do deliver. Our purpose is to create a better outcome for you.
Smart together
Our values and behaviours
Make a difference
We believe in our purpose. Together, we work as one team, always looking for ways to improve and support DCC’s role in a low-carbon future.
Act with integrity
We treat everyone with kindness, consideration and respect. We keep our promises to our customers. And if something doesn’t feel right, we speak up.
Be accountable
You can depend on our word. We’re clear about our roles and we take the initiative, while respecting the processes we have in place.
Rewards & Benefits
Get perks that work
Personal Development
Our training and development opportunities help you be your best self – from regular 1-2-1s all the way up to professional qualifications.
Group income protection
Provides a continuous income if you're unable to work due to illness or injury for more than 26 weeks.
Pension Plans
Our Capita Choice partners will match the salary contributions you make to your pension up to 5%, depending on your plan and time with us.
Flexible Holidays
Need more time off? Increase your standard 25 days of leave by buying extra to rest, spend time with family or satisfy your wanderlust.
Cycle to Work Scheme
We’re big on keeping things green. If you want to cycle to work, we'll loan you the gear through salary sacrifice to make it tax-efficient.
Flexible working
As we all transition to a new normal, we can help you find the hybrid working pattern that suits you and your team best.
Additional Benefits
Add more to your core
Unmind
Travel season ticket loan
Bonus
Medical cash plans
Life cover
Charity days
Childcare vouchers
Health screening
RAC discount
Vision Express discounts
Mobile phone discounts
Discounted gym membership
Retail and leisure discounts
