Data Engineer - Big Data architect / Hadoop / AWS - 6 month FTC
Job Title Data Engineer - Big Data architect using Hadoop on AWS platforms - 6 month FTC
One of our most prestigious clients is looking to hire an AWS experienced Candidate who has an understanding of how to offload/co-exists DWH/BI Architectures, Functional processes within Big Data Architectures. This role is an FTC for 6 months and this role does sit within IR35. Must be SC/DV Cleared or eligible to gain SC clearance without hindrance.
Some of the key factors are as follows -
- Big Data practitioner - hands-on experience is essential. A key to success in this role is a unique blend of technical skills, business knowledge and working with and influencing multiple cross-functional teams within the company and clients on the appropriate solutions.
- Strong technical experience as a Big Data Architect using the Hadoop ecosystems Cloudera on a Cloud Platforms (AWS) and associated technologies.
- Strong experience of distributed and NoSQL databases
- Core Technical Skillset: Java, SPARK, Scala, Python, and have worked on Hadoop ecosystem
Title of Role: Data Engineer
Key Purpose of the Role(s)
The Data Engineer will form a key part of the Strategic Data Analytics Capabilities (SDAC)
programme team specialising in the delivery of high-end data and analytics solutions in the
FCA s cloud data centre. This role requires a broad range of skills and the ability to step into
different roles depending on the size and scope of the data product being delivered to the
end customer. The primary day-to-day responsibility is to design, implement and maintain
Analytics tools and supporting relative services to support business-driven actionable insight.
The use of data and analytics is becoming increasingly critical to making informed business
decisions, building strategies, and improving operational effectiveness. The rapid pace of
analytics technology evolution and the strong supply of skilled individuals entering the global
workforce provide opportunities for the FCA to improve process efficiency and better
diagnose harm in markets, in line with the FCA mission statement.
In March 2016, the FCA acted upon the opportunity, approving a proposition to address the
increasing need for data analytics with the first phase of the SDAC programme known as
SDAC I. While the transformation of the FCA s data and analytics capability was deemed a
multi-year journey, the focus for SDAC I was on quickly building useful capabilities to allow
future business demand to be met efficiently in an enduring manner complementary to the
FCA strategic aims and Data Strategy.
In FY 16/17, SDAC I targeted a set of priority business needs for data analytics. Through
these priorities SDAC I delivered a rudimentary cloud-based technology platform that is
being used by a small population across the FCA with use highest in EMO and S&C for,
amongst things, various market studies, exploration of order book data to identify connected
parties in wholesale markets, and replacement of the neighbourhood watch tool in Market
In order to expand analytics adoption, the focus for SDAC II needs to shift to the
development of the FCA s people and skills in advanced analytics. Research carried out in
SDAC I revealed that:
- Self-reliant, advanced analysts are in short supply. Where skills do exist, colleagues
operate independently and undertake the duplicative effort.
- For the most part, data is extracted and reported using basic technology and
- In order to become a more efficient, data-led organisation the FCA needs to:
o significantly increase the number of more sophisticated analytical users
(through attracting, developing and retaining talent);
o adopt a more flexible resourcing model for addressing business issues with
o embrace an agile approach and mindset; and,
o embed more consistent methods and techniques.
SDAC II will be about increasing user capability to drive analytics demand to existing tools
before further expanding the technology base.
- Must be SC/DV Cleared or eligible to gain SC clearance without hindrance.
- Experience with generalist data and analytics roles.
- Demonstrable experience in creating and modifying jobs to support delivery of data.
- Experience in using AWS database/warehouse technologies (RDS, Redshift, Aurora,
- Great working knowledge of Linux and ability to complete common administrative
- Understanding of ETL/databasing concepts, architecture and data warehouse
.development methodologies and best practices.
- Governance and uploading of data using multiple tools.
- Knowledge of Python, SQL, Java.
- Excellent verbal and written communication skills.
- Ability to be a key team player delivering in an agile environment.
- Strong analytical skills and an eagerness to learn new technologies.
- Experience with RStudio, Spotfire, Stata and other leading statistical computing tools.
- Functional and Non Functional Requirements capture and elicitation.
- Red Hat knowledge/experience.
Key Accountabilities & Responsibilities (other responsibilities may be assigned)
- Working closely with our FCA Architects, Analysts, Data Scientists, functional
practitioners, and other stakeholders, the successful candidate will take time to
understand data usage requirements and take full responsibility to propose the best
route to live for data.
- Automate everything! The successful candidate will need to invest time in automating
all the tasks they do repeatedly, boosting the reliance of data led solutions and
providing business continuity.
- Working with the SDAC Solution Architect, DevOps Engineers, Test Manager the
successful candidate will be involved in designing, developing, testing, supporting and
debugging new and existing analytics environment.
- Suggesting enhancements to the platform and optimise where the opportunity arises.
- Performing data administration, managing granular data access security groups and
migrating across environments.
- Monitoring and optimising our RDS/Redshift databases using industry-standard tools
- Thoroughly document everything regarding the build and maintenance of our data
solutions, job scheduling, data audit and dependencies.
- Demonstrate curiosity and ability to learn new concepts, processes, tools, and best
practices. The successful candidate is going to need to adapt to different software
tools or techniques as the world of data and analytics continues to present new and
exciting options for the FCA.
- Provide technical assistance to the SDAC application development team as needed
using your Linux knowledge and skills.
- Provide end to end problem solving and technical troubleshooting.
Should you feel that your skill set matches the above requirements, please submit your CV along with evidence of SC/DV clearance for review. We look forward to speaking with you.
This job was originally posted as www.jobsite.co.uk/job/960293492