Data Engineer Manager

We are currently working on an exciting opportunity for a reputable and innovative business, who enable investment into low carbon technology projects, starting in proven technologies such as wind and solar and moving into Hydrogen and Carbon Capture, Usage and Storage. Their mission is to accelerate the delivery of Net Zero. Through their success, they are currently in a period of growth and are looking to invest in new talent and are looking to onboard a Data Engineering Manager into their Technology Hub. The Data Engineer Manager is responsible for driving the design, development, and optimization of data solutions within the data infrastructure. In addition to fostering the growth of a skilled team, you will play a pivotal role in delivering data applications, infrastructure, and services, ensuring they align with organisational goals and industry best practices. As part of the Technology Hub within the business, the Data Engineer Manager will work very closely with all teams across the business. The role is instrumental in defining and upholding a clear vision for the integrity of data life cycle management. Key responsibilities: * Mentor the data engineering team to design and implement complex, tailored data solutions that support processing of high volumes of data across all schemes and applications. * Establish and support the technical vision and strategy for a robust data architecture that aligns with LCCC’s overall strategy, with a strong focus on ensuring security for all structured data. * Establish and maintain robust operational support and monitoring systems to ensure the reliable performance of critical systems in live environments. * Facilitate the adoption and implementation of continuous delivery practices while advocating for the use of cloud solutions. * Design, implement, and optimize end-to-end data pipelines and solutions on Azure, ensuring data quality, reliability, and security throughout. * Oversee the integration of both structured and unstructured data sources. * Oversee project timelines, scope, and deliverables to ensure successful execution, while actively monitoring progress and addressing risks proactively. * Implement best practices for process improvements, cost optimization and monitoring. Continuously evaluate and improve the Azure data platform to enhance performance and scalability. * Collaborate with stakeholders to understand business requirements and translate them into technical solutions. * Develop and implement a comprehensive data governance framework to ensure data quality, security, and compliance across the data applications. The successful candidate will come from a solid Data Engineering or Data Architecture and governance background, with at least 5-6 years’ experience in a senior role. They must hold strong proficiency in Python, preferably PySpark, along with hands-on experience with Azure; ADLS, Databricks, Stream Analytics, SQL DW, Synapse, Databricks, Azure Functions, Serverless Architecture, ARM Templates, DevOps. In addition, they will be a credible and confident leader, with line management experience. AWS experience will be considered, providing the candidate is open to working with Azure. This employer is unable to provide sponsorship at this time