Data Engineer

Recruiter
Dynamix Recruitment Limited
Location
Ipswich, UK
Salary
Competitive
Posted
11 Jul 2019
Closes
24 Jul 2019
Ref
1205516387
Contract Type
Permanent
Hours
Full Time
As a data engineer within the exciting, new claims advanced analytics capability, you will be building big data solutions to solve some of the organization's toughest problems and delivering significant business value. This is a really exciting time to join as you will be helping to shape the big data analytics architecture and technology stack within a new cloud based data lake

Responsibilities:
  1. Shape the portfolio of business problems to solve by building detailed knowledge of data sources (internal and external)
  2. Model data landscape, obtain data extracts and define secure data exchange approaches
  3. Acquire, ingest, and process data from multiple sources and systems into Cloud Data Lake
  4. Operate in fast-paced, iterative environment while remaining compliant with Information Sec policies/standards
  5. Collaborate with data scientists to map data fields to hypotheses and curate, wrangle, and prepare data for use in their advanced analytical models
  6. Help architect the strategic advanced analytics technology landscape
  7. Build re-usable code and data assets

Codify best practices, methodology and share knowledge with other data engineers/scientists in the organisation

Measures:
  • Become expert in claims data sources
  • Framework set up across the company to define best practice in data engineering space
  • Robust data sources in the data lake with increasing proportion of data held in the lake
  • No unexpected issues arise
  • Successful delivery of cloud projects

"Single version of the truth" tables and views in the cloud that are used by a wide variety of end users providing accurate re-producible

Skills & Experience:
  • Meaningful experience (2+ years) with at least two of the following technologies: Python, Scala, SQL, Java
  • Experience and interest in Cloud platforms such as:, Azure, AWS or Databricks
  • The ability to work across structured, semi-structured, and unstructured data, extracting information and identifying linkages across disparate data sets
  • Meaningful experience in at least one database technology such as:

-Distributed Processing (Spark, Hadoop, EMR)

-Traditional RDBMS (MS SQL Server, Oracle, MySQL, PostgreSQL)

-MPP (AWS Redshift, Teradata)

-NoSQL (MongoDB, DynamoDB, Cassandra, Neo4J, Titan)
  • Understanding of Information Security principles to ensure compliant handling and management of data
  • Experience in traditional data warehousing / ETL tools (Informatica, Talend, Pentaho, DataStage)

Ability to clearly communicate complex solutions

Similar jobs

Similar jobs