Senior Data Engineer

JOB AD:

Job Title: Senior Data Engineer

Type: Permanent

Location: London, UK - 100% REMOTE

Responsibilities:

- Analyse source systems, define underlying data sources and transformation requirements, design suitable data models and document the design/specifications

- Build and maintain robust, scalable, automated cloud-based data pipelines using industry best practises

- Ensure quality of technical solutions as data moves across environments

- Recommend ways to improve data reliability, efficiency, and quality

- Partake in cross team code reviews to ensure all solutions developed meet required standards

- Contribute to the ongoing formation of the Data Engineering team's processes and skill sets

Key Requirements:

- Excellent coding skills in Python, PySpark and SQL

- Passionate about Test Driven Development

- Create and maintain architecture and systems documentation

- Experience of developing AWS data pipelines using Glue, Step Functions, Lambda to different data stores such as S3, Redshift and RDS.

- Big Data ecosystem data design and processing - including file systems, data structures, automation, security, messaging, movement etc.

- In production cloud data infrastructure - including design, implementation, and maintenance (preferable AWS)

- With SQL development across multiple language variations - PostgreSQL, Amazon Athena

- Good communications skills across all levels of stakeholder (business & technical)

- Continually keep up with advancements in data engineering practises

- 5+ years of experience

- Mentor other members of the team in the principles of data engineering.

- Expertise with version control e.g. Git

Desirable:

- Experience of encrypting and obfuscating sensitive data

- Knowledge of CI\CD best practices

- Experience of deploying Infrastructure as Code preferably using Terraform

- Experience working with both structured and unstructured data in various formats - such as JSON, AVRO, CSV, TSV, PDF etc.