Data Engineer, Remote Working, Manchester, £45-55K + 10% bonus + £5.8K Car Allowance + bens. You must be an experienced Data Engineer with solid experience with SQL and Python. Any experience of AWS, CI/CD pipelines, Snowflake, Apache Airflow, Kinesis, Kafka, Docker, Kubernetes, TDD, XP would be advantageous.
Reporting into the Data Engineering Technical Lead, the role involves creating data solutions to process data at scale, both in batch and real-time pipelines, to support a wide range of data-driven projects and support our transformation into an AI-ready organisation.
The key responsibilities of the Data Engineer are:
*Implement workflows to ingest data into a Snowflake data warehouse for a variety of data sources
*Implement data transformation pipelines in real-time and batch environments
*Support all product teams in adopting our data engineering tech stack to generate new data streams
*Collaborate with Data Science and Business Intelligence teams to identify requirements and develop the necessary data workflows to deliver against those requirements
Experience: (some of the following)
*University degree in Computer Science, Software Engineering or related subjects
*Experienced in Python or Scala. Any Java would be a plus
*Relational and non-relational databases. (SQL, SQL Server, NoSQL). Experience with Snowflake a plus.
*Batch processing frameworks, such as DBT, Flink, Apache Airflow, etc.
*Message brokers / stream processing technologies (Kinesis, Kafka, Storm, Spark Streaming, Flink, etc.)
*Familiarity with AWS, Docker, Kubernetes, Amazon EKS
*Continuous Integration with Jenkins
*Test-Driven Development and XP
This role involves close collaboration with data scientists, data analysts and product engineers.
This Data Engineer position, Manchester is remote working and is paying c£45-55K + 10% bonus + Bens, depending on relevant experience.