Lead Data Engineer Architect - Python, Spark, AWS

Recruiter
Harnham
Location
Farringdon
Posted
15 Apr 2018
Closes
20 Apr 2018
Contract Type
Permanent
Hours
Full Time

Lead Data Engineer + Architect - Python/AWS/Spark

London

Up to £105,000 + benefits

This is the chance to join one of the fastest growing companies in the UK, and lead their data engineering and architecture efforts.

With an advanced data platform based on Spark and AWS, you will join the team to lead their efforts in increasing the scalability and robustness of the platform.

THE COMPANY:

One of the fastest growing companies in the UK is looking for a talented data leader to join their team. They are based in the world of online sport media, and are currently expanding into countries across Europe and the Rest of the World.

THE ROLE:

As a Lead Data Engineer + Architect, you will:

  • Be responsible for the design and development of the data warehouse/data lake combination and surrounding infrastructure.
  • Be responsible for choosing optimal solutions to support data sets and develop the data warehouse/data lake.
  • Be highly experienced in data engineering and architecture to recognise business needs and technical solutions.

YOUR SKILLS AND EXPERIENCE:

As a Lead Data Engineer + Architect, you will:

  • Be responsible for evolving the exisitng ETL pipelines (python + luigi + pandas) and the AWS redshift database.
  • Ensure the infratructure can meet business demands and scalability requirements.
  • Improve the performance of the redshift data warehouse.
  • Migrate ETL processing to EC2 instances to a continous delivery model.
  • Research and POC new technologies (eg Spark, Kafka).

THE BENEFITS:

  • The chance to be an established leader within one of the fastest growing businesses in the UK.
  • Take ownership of the data warehouse and data lake solution.
  • A salary of up to £105,000 + a great benefit package.

For more information on the role, please get in touch with Josh Carter from Harnham, and apply via the link below.

Similar jobs

Similar jobs