Big Data Engineer - London - Software House - Hadoop, Hive, Java, Scala, Python, Spark, AWS, ETL
Big Data Engineer highly skilled in high-volume production systems/architectures using Hadoop, Hive, Spark, AWS and managing ETL workflows using API's, Python, Java/Scala is urgently required to join a Product Software House based in Central London.
As a Big Data Engineer, you'll be part of a new technical team working on a Greenfield project to understand the requirements for and to evolve and implement timely, relevant and accurate engineering processes. Your key technical stack will be using the full Hadoop ecosystem (Hadoop, Hive, HBase, Spark) whilst further using relevant API's and languages such as Java, Scala and Python in managing and optimising ETL and workflows. The project's architecture is cloud-based in AWS, Google Cloud Platform and Google Big Query. As part of a small Agile team, you will further keep current with the wider Big Data ecosystem technologies and Analytics community to ensure leading practises are utilised.
Previous experience working on large data-sets and analytics within an Agile eCommerce/FinTech environment is highly desired.
Essential Technical & Soft Skills
- Previous experience building, managing or large scale Big Data initiatives, including Big Data
- 3+ years commercial experience using Python, Spark, Java/Scala
- 3+ years experience of ETL tooling including Talend, Informatica, Pentaho
- Exposure to Hadoop ecosystem (Hadoop, Hive, Hbase)
- Understanding in linking business objectives to data KPI's
- API integration
- Strong understanding in Cloud-Platforms such as AWS, Google Cloud Storage, Google Big Query
- Agile development experience (Scrum/Kanban)
- Team player
- Strong communication skills
This is a great opportunity to join a market-leading software house for an exciting and innovative Greenfield project as part of a new technical team. You'll be part of a welcoming environment with a reputation for entrepreneurialism, game-changing ideas and unique culture.
Interested? Get in touch with an updated copy of your CV to learn more.