Big Data Engineer - Hadoop, Kafka, TDD, BDD, Spark, Agile

People Source Consulting Limited
London (Greater)
12 Jan 2018
10 Feb 2018
Contract Type
Full Time
Big Data / Cloudera Hadoop / Spark / Kafka / Scala / Python / Unix / Agile / Central London

£80,000-£90,000 + fantastic benefits package including healthcare, Pension and company bonus

A fantastic opportunity has arisen for a Big Data Engineer to join a global giant and work on one the largest Big Data projects in the UK.

The company
A market leader, the company has accomplished a strong reputation as an employer of choice. Operating in over 25 countries, the company is always committed to deliver useful and inspiring innovation. A brand that loves change - if it's not happening naturally then they will create it themselves.

Key responsibilities of the role:
* Expert level experience in designing, building and managing applications to process large amounts of data in a Hadoop ecosystem;
* Extensive experience with performance tuning applications on Hadoop and configuring Hadoop systems to maximise performance;
* Experience building systems to perform real-time data processing using Spark Streaming and Kafka, or similar technologies;
* Experience with common SDLC, including SCM, build tools, unit testing, TDD/BDD, continuous delivery and agile practises
* Experience working in large-scale multi tenancy Hadoop environments

You'll have experience of the following:
* Extensive experience with Hadoop ecosystem (Spark, Hive/Impala, HBase, Yarn); desirable experience with Cloudera distribution;
* Strong software development experience in Scala and Python programing languages
* Experience with Unix-based systems, including bash programming
* Experience with columnar data formats
* Experience with other distributed technologies such as Cassandra, Solr/ElasticSearch, Flink, Flume would also be desirable.

The package
£80k-90k + fantastic benefits package including healthcare, Pension and company bonus

Role is based in central London

***If you are interested but your salary expectations exceed this level please do apply stating your requirements as there may be other suitable roles***

People Source Consulting Ltd is acting as an Employment Agency in relation to this vacancy.

People Source specialise in technology recruitment across niche markets including Information Technology, Digital TV, Digital Marketing, Project and Programme Management, SAP, Digital and Consumer Electronics, Air Traffic Management, Management Consultancy, Business Intelligence, Manufacturing, Telecoms, Public Sector, Healthcare, Finance and Oil & Gas.
This job was originally posted as