Data Engineer / London / Milton Keynes

Sopra Steria Recruitment Limited
London (Greater)
16 May 2018
15 Jun 2018
Contract Type
Full Time
Data Engineer

Milton Keynes or London


Sopra Steria Recruitment are seeking 4 experienced Data Engineers to work for a major Client in the Milton Keynes or London area.

The Role:

·Load data from a range of different batch & streaming data sources located internally and externally and deal with all of the typical Data Wrangling challenges.

·Liaise with source system teams to understand their data and construct optimal data pipelines.

·Develop Data Quality routines to cleanse and reconcile the data.

·Work with the Data Architect to combine different data streams into common data structures.

·Work with Analysts and Data Scientists across the company to deliver data that will allow them to quickly provide actionable intelligence to the business.

·Provide subject matter expertise with Big Data technologies to constantly challenge and improve the Big Data solutions by adding new capabilities, enhancing performance or improving robustness and solution availability.

Essential Skills:

·An in-depth understanding of the Apache Hadoop ecosystem (Cloudera preferred), including Hive, Impala and HBase

·Development experience with Apache Spark

·Programming in one or more of the following languages: Python, Java, Scala

·Data Warehouse Design and Development skills (OLAP, Star Schema, Kimball)

·Database Design & Development in one or more of the following: Oracle, Microsoft SQL Server, IBM DB2, MySQL, PostgreSQL

·ETL Experience constructing data flows from multiple source systems (Talend or Pentaho preferred)

·Data Analytics experience (e.g. Regression, Clustering, Decision Trees, Forecasting, Statistics)

·SQL development (e.g. PL/SQL or T-SQL) including HiveQL

·Experience working with Structured, Semi-Structured and Unstructured data

·Linux Shell scripting

Desirable Skills:

·Streaming data and experience with Kafka or RabbitMQ

·Software Deployment and Continuous Integration with Git, Jenkins & Docker

·Data Visualisation (e.g. Microsoft PowerBI or Tableau)

This job was originally posted as

Similar jobs

Similar jobs