We appreciate the value of data within ours and client's business. We generate more data than ever before but we recognise that clean and meaningful data streamed at a rate which can give us a competitive edge is the key. Our Data Engineers are key to activating our innovative solutions. The role requires an individual who can work with other engineers and analysts to create solutions which integrate with our advertising technology stack. The role will be client facing, with the collation and documentation of requirements.
We would expect the individual to achieve the Certified Google Data Engineer status shortly after joining (if this is not already attained) and maintain this qualification. Training will be provided where necessary.
We also run GCP Training Courses and it would be expected that the individual would take part in giving those courses to our clients (after attaining the GCP Trainer accreditation).Responsibilities
Knowledge & Experience Requirements
- All extract, transform and load (ETL) processes and the creation of applications that can connect to remote APIs. Preferably including streaming data into environments such as BigQuery.
- Management of multiple processes and applications, performance reporting and error checking.
- Management of all data created within client applications, the structure of data held and the views of data created.
- Recommending the correct technologies to be used and in the most cost effective manner.
- Design and creation of data led strategies which provide clients with opportunities to leverage their data for greater insight or performance.
- Thought leadership with regards to best practice and use of the google cloud platform.
- 80% development, 20% new business & scoping (technical specifications)
- Be willing to take ownership of projects and to be able to present your proposed solutions to the wider team and to non-technical staff in an understandable way
- Automating repetitive tasks
- Proactively suggest, plan and integrate new tools and methods to improve productivity, efficiency and reduce costs
- Be willing to work in a research capacity - to find and test and critically review open-source frameworks and technologies
- Maintain and enforce a good standard of coding and adhere to the same conventions as the rest of the team
- Leave code better than you found it (and fully commented)
- Helping acquire and respond to new business requests
- Solid Python programming skills. Java / other languages beneficial.
- Strong SQL experience, any flavour.
- Must have experience of building robust pipelines and data environments to support database or machine learning based applications.
- Google Cloud Platform experience is beneficial
- Official Google Data Engineer Certification is beneficial.
- Data Engineering / BI Development / Data Warehousing experience.
- Knowledge of serverless infrastructure beneficial
- Prior management experience
- Ability to scope a project based on a technical brief and work with the DevOps and QA teams to provide a detailed project plan including:
- Data Flow Diagrams for process flow
- Database Schemas & Normalisation
- Scalable environment architecture suggestions
- A complete and accurate estimate for the project
- Ability to assess new business and respond with a full list of targeted questions to ensure accurate estimates are created
- Ability to research solutions to technical problems
- Experience scheduling/automating scripts
- Experience with streaming data beneficial
- Experience on Linux command line and Bash scripting
- Experience with Git/GitHub
- Experience with Dataflow, Google PubSub or other queuing software beneficial
- Good experience of parsing data formats such as XML/JSON and using 3rd party API's
- An understanding of how data can benefit the wider business, and how to translate technical requirements to non-technical stakeholders.
- Good time management skills
- Pro-active & self-motivated with a sense of ownership
- Strong verbal and written communication skills - able to communicate with clients effectively
- Adopts a "leave code better than you found it (and fully commented)" attitude
- Ability to perform effectively to tight deadlines, work well both independently and as part of a team
- Self-managing within the scope of the role
- Educated to degree level in Computer Science or equivalent
- A good understanding of the change request process