Big Data Engineer - (Pipeline Messaging Team) - Bangkok

Recruiter
Big Wednesday Digital
Location
South West London
Salary
£45000 - £75000 per annum + Negotiable
Posted
09 Feb 2017
Closes
09 Mar 2017
Contract Type
Permanent
Hours
Full Time
*Please no C2C / 3rd party applications*

Please note this role is based in Thailand - full relocation and visa sponsorship provided

Our client is the largest and fastest growing online hotel booking platform in Asia, and are part of the largest online travel company in the world.

About You:

  • Willing to relocate to Bangkok, Thailand
  • You'll probably have a B.Sc. in Computer Science / Information Systems / Computer Engineering or related field
  • You have two plus years of industry experience, preferred at a tech company
  • A passion for Big (Petabytes worth) Data
  • Good knowledge of data architecture principles
  • You have operational experience debugging production issues
  • You have experience with at least one of the following: Scala, Java, C#, GoLang and/or any functional language
  • An experienced coder, who can stand your ground with experience building systems with purpose that are flexible, well-tested, maintainable and scale
  • You have no issues being on-call and working at odd hours as needed
  • You can communicate in technical English with fluidity, both verbal and written
  • Good understanding of how Kafka works
  • Experience producing messages to Kafka from any one of the following languages: Java, Scala, C#, GoLang
  • Understands Concepts relating to Schema Registry and Schema Evolution
  • Experience working with Serialization Formats either with ProtocolBuffers, Avro or Thrift
  • Knows how to use ElasticSearch proficiently
  • Development experience on Hadoop (MapReduce, Spark, Hive, Impala, SparkSql)
  • Experience with data ingestion from Kafka into Hadoop, ElasticSearch, other Distributed Systems
  • Strong systems administration skills in Linux
  • Worked on or contributed to Open Source Project

The role:

The Bangkok team is looking for top quality passionate engineers to build products across their next gen data platform products.

Our systems scale across a multitude of data centers, totaling a few million writes per second and managing petabytes of data. They deal with problems from real-time data-ingestion, replication, enrichment, storage and analytics. They are not just using Big Data technologies; they are pushing them to the edge.

Why Messaging Platform Team?

You will be joining the team that handles the data pipeline infrastructure that is the core backbone for all Data Event Logging and is crucial to real time monitoring of all Systems across geographically distributed Data Centers.

They are a small passionate team and they are looking for exceptional individuals to be a part of designing, building, deploying (and probably debugging) their Data Pipeline.

Some tech you will use:

Kafka, ElasticSearch, Scala, Akka, Java, GoLang, C#, Python, Ruby, Hadoop, Spark, Hive/Impala/SparkSql, Avro, Parquet, Schema Registry, Sensu, Postgres, MSSQL, Graphite, Grafana

Day to Day:

  • You will build, administer and scale data pipelines that process billions of messages a day spanning over multiple data centers
  • You will be comfortable navigating the following technology stack: Linux, JVM, Java/Scala, C#, Golang, Kafka, scripting (Bash/Python), Hadoop, ElasticSearch
  • You will develop and expand upon existing frameworks that is used by Teams throughout Agoda to produce messages to the data pipeline
  • You will build and manage data ingestion into multiple systems (Hadoop, ElasticSearch, other Distributed Systems)
  • You will build tools that monitor high data accuracy SLAs for the data pipeline
  • You will fix production problems
  • You will profile for performance, self-recovery and stability
  • You will collaborate with other teams and departments
  • You will automate system tasks via code as needed
  • You will explore available new technologies that improve upon our quality of data, processes and data flow
  • You will develop quality software through design review, code reviews and test driven development

If that's the kind of team you want to join, let's talk!

Our client is an equal opportunity employer and values diversity. They do not discriminate on the basis of race, religion, color, national origin, gender, sexual orientation, age, marital status or disability status.

Please note this role open to local and international applications. Full visa sponsorship and relocation assistance available.