Big Data Engineer - Scala

Recruiter
Harnham
Location
London, UK
Salary
Competitive
Posted
27 Jul 2019
Closes
31 Jul 2019
Ref
1205154741
Sector
Retail
Contract Type
Permanent
Hours
Full Time
Big Data Engineer - Scala
£600 per day
3-months
Central London

As a Big Data Engineer, you will be migrating away from a Redshift data warehouse and introducing Spark using Airflow for orchestration. This will be for a food-based E-commerce company.

THE COMPANY:
This company focus on using data to drive their success. They are an E-commerce company who supply food and beverages across the UK. They are using Data Science algorithms to improve efficiency and reduce waste. They have established a data platform that allows Data Scientists to build their models on, however, they are not ready to use big data just yet. This is where you are required to come and make their platform, big data ready.

THE ROLE:
You will be working alongside Data Scientists to ensure they have the relevant data for them to build their models. You are required to have expertise building ETL pipelines in Python. Ideally, you will have worked with big data using Spark applications and should be familiar with using Airflow. They have an AWS environment but are moving away from Redshift.

YOUR SKILLS AND EXPERIENCE:
The ideal Big Data Engineer will have:
  • A good understanding of AWS components: Redshift, S3, Lambdas, EC2
  • Strong coding skills in Python
  • Experience using big data: Spark
  • Worked with orchestration tools like Airflow


HOW TO APPLY:
Please register your interest by sending your CV via the Apply link on this page.

Similar jobs

Similar jobs