Data Engineer - Big Data architect / Hadoop / AWS - 6 month FTC

The Resourcing Hub
London (Greater)
08 Mar 2018
16 Mar 2018
Contract Type
Full Time

Job Title Data Engineer - Big Data architect using Hadoop on AWS platforms - 6 month FTC

One of our most prestigious clients is looking to hire an AWS experienced Candidate who has an understanding of how to offload/co-exists DWH/BI Architectures, Functional processes within Big Data Architectures. This role is an FTC for 6 months and this role does sit within IR35. Must be SC/DV Cleared or eligible to gain SC clearance without hindrance.

Some of the key factors are as follows -

  • Big Data practitioner - hands-on experience is essential. A key to success in this role is a unique blend of technical skills, business knowledge and working with and influencing multiple cross-functional teams within the company and clients on the appropriate solutions.
  • Strong technical experience as a Big Data Architect using the Hadoop ecosystems Cloudera on a Cloud Platforms (AWS) and associated technologies.
  • Strong experience of distributed and NoSQL databases
  • Core Technical Skillset: Java, SPARK, Scala, Python, and have worked on Hadoop ecosystem


Title of Role: Data Engineer

Key Purpose of the Role(s)

The Data Engineer will form a key part of the Strategic Data Analytics Capabilities (SDAC)

programme team specialising in the delivery of high-end data and analytics solutions in the

FCA s cloud data centre. This role requires a broad range of skills and the ability to step into

different roles depending on the size and scope of the data product being delivered to the

end customer. The primary day-to-day responsibility is to design, implement and maintain

Analytics tools and supporting relative services to support business-driven actionable insight.


The use of data and analytics is becoming increasingly critical to making informed business

decisions, building strategies, and improving operational effectiveness. The rapid pace of

analytics technology evolution and the strong supply of skilled individuals entering the global

workforce provide opportunities for the FCA to improve process efficiency and better

diagnose harm in markets, in line with the FCA mission statement.

In March 2016, the FCA acted upon the opportunity, approving a proposition to address the

increasing need for data analytics with the first phase of the SDAC programme known as

SDAC I. While the transformation of the FCA s data and analytics capability was deemed a

multi-year journey, the focus for SDAC I was on quickly building useful capabilities to allow

future business demand to be met efficiently in an enduring manner complementary to the

FCA strategic aims and Data Strategy.

In FY 16/17, SDAC I targeted a set of priority business needs for data analytics. Through

these priorities SDAC I delivered a rudimentary cloud-based technology platform that is

being used by a small population across the FCA with use highest in EMO and S&C for,

amongst things, various market studies, exploration of order book data to identify connected

parties in wholesale markets, and replacement of the neighbourhood watch tool in Market


In order to expand analytics adoption, the focus for SDAC II needs to shift to the

development of the FCA s people and skills in advanced analytics. Research carried out in

SDAC I revealed that:

  • Self-reliant, advanced analysts are in short supply. Where skills do exist, colleagues

operate independently and undertake the duplicative effort.

  • For the most part, data is extracted and reported using basic technology and

inconsistent processes.

  • In order to become a more efficient, data-led organisation the FCA needs to:

o significantly increase the number of more sophisticated analytical users

(through attracting, developing and retaining talent);

o adopt a more flexible resourcing model for addressing business issues with


o embrace an agile approach and mindset; and,

o embed more consistent methods and techniques.

SDAC II will be about increasing user capability to drive analytics demand to existing tools

before further expanding the technology base.

Person Specification


  • Must be SC/DV Cleared or eligible to gain SC clearance without hindrance.
  • Experience with generalist data and analytics roles.
  • Demonstrable experience in creating and modifying jobs to support delivery of data.
  • Experience in using AWS database/warehouse technologies (RDS, Redshift, Aurora,

and Oracle).

  • Great working knowledge of Linux and ability to complete common administrative


  • Understanding of ETL/databasing concepts, architecture and data warehouse

.development methodologies and best practices.

  • Governance and uploading of data using multiple tools.
  • Knowledge of Python, SQL, Java.
  • Excellent verbal and written communication skills.
  • Ability to be a key team player delivering in an agile environment.
  • Strong analytical skills and an eagerness to learn new technologies.


  • Experience with RStudio, Spotfire, Stata and other leading statistical computing tools.
  • Functional and Non Functional Requirements capture and elicitation.
  • Red Hat knowledge/experience.

Key Accountabilities & Responsibilities (other responsibilities may be assigned)

  • Working closely with our FCA Architects, Analysts, Data Scientists, functional

practitioners, and other stakeholders, the successful candidate will take time to

understand data usage requirements and take full responsibility to propose the best

route to live for data.

  • Automate everything! The successful candidate will need to invest time in automating

all the tasks they do repeatedly, boosting the reliance of data led solutions and

providing business continuity.

  • Working with the SDAC Solution Architect, DevOps Engineers, Test Manager the

successful candidate will be involved in designing, developing, testing, supporting and

debugging new and existing analytics environment.

  • Suggesting enhancements to the platform and optimise where the opportunity arises.
  • Performing data administration, managing granular data access security groups and

migrating across environments.

  • Monitoring and optimising our RDS/Redshift databases using industry-standard tools

and practices.

  • Thoroughly document everything regarding the build and maintenance of our data

solutions, job scheduling, data audit and dependencies.

  • Demonstrate curiosity and ability to learn new concepts, processes, tools, and best

practices. The successful candidate is going to need to adapt to different software

tools or techniques as the world of data and analytics continues to present new and

exciting options for the FCA.

  • Provide technical assistance to the SDAC application development team as needed

using your Linux knowledge and skills.

  • Provide end to end problem solving and technical troubleshooting.

All applicants.

Should you feel that your skill set matches the above requirements, please submit your CV along with evidence of SC/DV clearance for review. We look forward to speaking with you.

This job was originally posted as

Similar jobs

Similar jobs