Senior Data Engineer (Spark/Kinesis/Hadoop/AWS/Agile)
Senior Data Engineer (Spark/Kinesis/Hadoop/AWS/Agile) - Bristol - c£Negotiable plus benefits
Our global client are looking for an experienced and motivated cloud/backend Senior software engineer who is passionate about Lean Software development and code craftsmanship. You will be working alongside a growing team of like minded engineers in our clients rapidly expanding IoT platform engineering group. You will be responsible for building world-class technology employing the latest continuous delivery and devops practices to deliver the next generation of connected apps and systems.
If you thrive on working in a collaborative, stimulating, fast moving environment and relish difficult software engineering challenges, then our client would love to speak with you!
Our client is building world-class cross functional Agile teams and adopting the latest technology and techniques to ensure they can deliver their ambitious vision in the connected space. You’ll be working to create a world-class user experience in one of the fastest moving technology domains, alongside other engineers, designers, commercial strategists and electronics engineers.
- Highly experienced in data processing techniques and analytics with Strong SQL experience.
- Proven track record of building and maintaining scalable ETL data processing systems (Spark, Storm, DataTorrentRTS, GoogleBigQuery)
- Great software development skills in (Java, Python, C#, Scala)
- Passion for lean agile development of well tested code in a containerised/cloud environment.
- Good knowledge of Big Data querying tools, such as Presto (Athena), Pig, Hive, and Impala.
- Good knowledge of data visualization tools (Tableau, Qlik, Spotfire, Power BI)
- Knowledge of AWS data tools (EMR, Redshift, Elasticsearch, Athena, Kinesis)
- ML / AI experience with MapReduce, clustering and MLLib/Pandas
- Knowledge of DevOps practices and have used them with CI/CD pipelines.
- Knowledge and development using Cloud based systems (AWS/Azure or Docker-based container management system)
- Experience of M2M or IoT data applications
A senior data engineer will be expected to demonstrate the following:
- Push code daily that will be relied upon by our ever-growing fleet of connected users and products
- Answering business needs with data analytics
- Developing new features and extending existing data platform using Java, Python and a range of deployment automation and monitoring tools
- Support and coaching of software developers and BI data engineers through advice, guidance and pairing
- Improving team efficiency by analysing working practices and procedures
- Work effectively as a key member of an agile development team utilising Scrum based methodologies and tool suites e.g. Atlassian JIRA/Stash
- Steering the direction of development in order to assist platform growth and feature enrichment
- Review the code of others for accuracy and functionality and to offer guidance for improvement if needed
- Monitor and assist with the deployment of code through test environments towards production and the handling of any issues that arise
- A passionate approach to work and a desire to push boundaries and constantly improve quality.
- Self-starter, able to take an idea from concept through to production
- Keen to assist others and both learn from and teach those around you
- Have a keen sense of ownership of code produced by you and your team members
- Demonstrate a continuing interest in emerging technologies and practices
- Able to communicate clearly and effectively with teams locally and in different time zones, as well as non-technical colleagues
Our client would love to talk to you about any personal or open source projects you might be working on, talks you may have given or articles you have written. If you have a profile on a website like Github, please send this along with your CV for consideration.