Excellent fully-remote technology role for an Apache Airflow Specialist to join a global player in providing data insights to the advertising & media industry.
Remote Apache Airflow Specialist – ETL
Plus Benefits Package & Full Remote Working
Who we are:
We are a provider of SaaS/software to advertising and media companies, providing the most influential agencies, media owners and planners with innovative cloud-based tools to help create, plan and predict the most successful media campaigns. With over 50 years of experience providing data and analytical tools for the marketing and advertising industry, our easy-to-use software products and solutions support over 8,000 databases, making it the industry’s most trusted third-party data analysis software globally.
Apache Airflow Specialist Role Summary:
Reporting to the Engineering Resource Manager in the role of Apache Airflow Specialist, you will be responsible for designing, expanding, automating and optimizing our data pipeline architecture, as well as optimising data flow and collection.
You will be responsible for the Airflow architecture to design, build and manage automated data pipelines built upon environments in AWS and pioneer ways of working that other team members can adopt
Using Apache Airflow for respondent survey pipeline building, you will work to automate components of a manual load and then add these to the pipeline. The aim is to reduce manual intervention, and increase processing speed, so that we can repeatedly process more data, at higher frequencies in the future. Where needed you may choose to add manual human tasks to the automation workflow.
You will work with Data Engineers, Python Engineers and Product Managers within Agile to deliver secure, performant and maintainable data automation. You will use best-practice continuous integration and continuous deployment methodologies to ensure that the build and deployment pipelines are fast, robust and secure. You will champion good code quality and architectural practices.
+ Significant experience of large scale implementation using Apache Airflow
+ Be an expert in the concept of DAGs (Directed Acyclic Graph) and Operators to schedule jobs
+ Working knowledge of message queuing, stream processing, and highly scalable “big data” data stores.
+ Experience of manipulating, processing and extracting value from large disconnected datasets.
+ Prior experience with customer data platforms.
+ Experience in performing root cause analysis on internal/external data and processes.
+ Prior experience with data analysis and data warehousing
+ Technical expertise with data models, data mining, and segmentation techniques
+ Proficiency in scripting languages (especially Python)
+ Be able to investigate current data loading procedures, planning pipelines and required steps in order to automate data extraction, transformation, and loading (ETL) processes
+ Proficiency in understanding of GitHub for source code repositories to maintain daily operation, integrity and security of source code.
+ Experience of conducting code reviews against acceptance criteria
+ Knowledge of Amazon Web Services (AWS) infrastructure & services e.g. Redshift, EC2, RDS, S3, Lambda, EMR, Batch or Athena
+ Excellent Linux scripting skills
+ Experience with data modelling, data processing and ETL
+ Passionate about the power of data to drive better business outcomes for our customers.
+ A working knowledge of SQL, query authoring and a working familiarity with a variety of relational databases.
+ Experience with Agile methodologies and change management i.e. JIRA and be able to define technical acceptance criteria for stories
+ Experience working with external partners to drive product delivery.
+ Excellent problem-solving skills
+ Proven ability to work effectively in a distributed working environment
+ Outstanding written and verbal communication skills
+ Ability to estimate effort of own tasks and those of others in expertise domain
+ Organized, detail-oriented, and deadline-driven
+ Strong interpersonal skills and the ability to work proactively, independently and as a team player
+ Ability to work efficiently and productively in a fast-paced environment
+ Willingness to learn new skills
+ Be confident with numbers
Interested? Apply here for a fast-track path to the Hiring Manager.
If you have any pre-application questions please contact us first quoting the job title & ref. Good luck, Team RR.