Data Engineer

Recruiter
hackajob Client
Location
Victoria
Posted
02 Nov 2018
Closes
17 Nov 2018
Contract Type
Permanent
Hours
Full Time
Preferred Skills: Java, Python, Scala, Hadoop, HBase, Elastic Search, Kibana, Spark, PySpark, Spark Scala, Apache Pig, Apache ZooKeeper, Talend, Cloudera, Apache Impala, Kafka. Oracle, MySQL, ETL, Big Data, Agile.

Essential Skills:

  • An in-depth understanding of the Apache Hadoop ecosystem (Cloudera preferred), including Hive, Impala and HBase
  • Development experience with Apache Spark
  • Programming in one or more of the following languages: Python, Java or Scala
  • Data Warehouse Design and Development skills (OLAP, Star Schema, Kimball)
  • Database Design & Development in one or more of the following: Oracle, Microsoft SQL Server, IBM DB2, MySQL, PostgreSQL
  • ETL Experience constructing data flows from multiple source systems (Talend or Pentaho preferred)
  • Data Analytics experience (e.g. Regression, Clustering, Decision Trees, Forecasting, Statistics)
  • SQL development (e.g. PL/SQL or T-SQL) including HiveQL
  • Experience working with Structured, Semi-Structured and Unstructured data
  • Linux Shell scripting

Desirable Skills:

  • Streaming data and experience with Kafka or RabbitMQ
  • Software Deployment and Continuous Integration with Git, Jenkins & Docker
  • Data Visualisation (e.g. Microsoft PowerBI or Tableau)
  • Monitoring expertise with Splunk and Kibana
  • NoSQL development (e.g. MongoDB)
  • Cloud storage, including both AWS S3 and Microsoft Azure Blob Storage
  • An understanding of Data Security, Encryption and GDPR
  • Agile Development and Scrum

Similar jobs

Similar jobs