As heavy industries continue to adopt sophisticated IoT based sensors capable of outputting big data sets, there is now a great potential to deepen our understanding of complex engineering processes through automated diagnostic analysis and reporting. NEL are therefore looking to recruit a Data Engineer to join our existing 'Digital Services' group, to help transform the use of data and digital technologies within the business and create new digital services for all industry sectors. The role will be to drive better exploitation of the vast historical data reserves within our business as well as the analysis of live data streams from ongoing R&D and commercial projects, whilst creating new service-based products we can offer to market. This role will work closely with a team of Data Scientists and software engineers and is responsible for integrating existing models developed in programming languages such as Python and transform outputs into a robust software that is capable of performing live data analysis. Although sitting within the Digital Services group, you will work with scientists and engineers from across the business to deliver a range of projects covering all industry sectors.

The role has great freedom to innovate, and we are looking for someone with new ideas and the drive to convert these into tangible results. As such, the position will suit someone who is a self-starter, enjoys working in multi-disciplinary teams on complex problems and has a passion for finding answers to difficult question from complex data.

The role will be based at our facilities in East Kilbride, and the candidate must have the ability to work in the UK and be able to travel when required.

Responsibilities and authority:

* Transform the use of data and digital technologies within the business and the services we offer across all industry sector

* Build infrastructure and data pipelines using software engineering best practices, data management tools, and the most practical frameworks in distributed systems and Big Data technologies.

* Design, implement and document stable, scalable, low-cost ETL/ELT pipelines that reshape and persist data.

* Develop an automated system of workflows (tasks, jobs) to transfer data from different sources

* Integrate the management infrastructure to monitor and maintain data-driven applications.

Essential:

* High proficiency in Python

* Proficiency in SQL and data warehousing concepts

* Knowledge of professional software engineering practices & best practices for the full software development life cycle, including coding standards, code reviews, source control management, build processes, testing, and operations.

* Experience with Agile web application development.

* Experience or demonstrated interest in frontend frameworks like Angular, React or equivalent.

* Experience or demonstrated interest in backend frameworks like Django, Laravel, NestJS or equivalent

Desirable:

* Experience or demonstrated interest in DevOps, Docker, cloud/AWS, automated testing

* Software Development Lifecycle tools like Git

* Experience creating workflows and data pipelines using tools like Apache Airflow, Apache Spark or Kafka