Airflow is a renowned tool for data engineering. It helps with orchestrating ETL workloads and it's well regarded amongst machine learning engineers as well. So, how does Airflow work and how is it applied to MLOps? In this episode, Demetrios and David are joined by Simon Darr, a Managing Consultant at Servian, with many years of experience using Airflow, along with Byron Allen, a Senior Consultant at Servian, specializing in ML. The group discusses how Airflow works, its pros, and cons for MLOps and how it is used in practice along with a short demo.
In this episode
Senior Consultant , Servian UK
Byron is a Senior Consultant at Servian - a data consultancy in Australia that also has a footprint across APAC as well as the UK. Byron is based in the London office where he helps organisations discover and build competitive advantage through their data. His focus is on client advisory and consulting delivery related to Experiments and ProductionML (i.e. data science, experimental design, ML model development, MLOps). Byron has written about a wide range of topics including the divide between data engineer and scientist, the role of ML in the post-covid world, and Kubeflow vs. MLflow.
Managing Consultant, Servian UK
I am a data and analytics specialist with a keen interest in data engineering and software development. I have a broad range of experience delivering technical solutions as well as supporting, building and maintaining data platforms. Specifically, I have experience in the following technologies: Building - Snowflake Data Warehouse - SQL, Python, Scala - Fivetran & Apache Airflow - Apache Flink & Spark - ELK stack - Oracle OBIEE Supporting - Snowflake Data Warehouse - AWS & GCP infrastructure - Oracle appliances (Exadata, Big Data Appliance, ZFS and OVM appliances) - General Linux administration & scripting
Demetrios is one of the main organizers of the MLOps community and currently resides in a small town outside Frankfurt, Germany. He is an avid traveller who taught English as a second language to see the world and learn about new cultures. Demetrios fell into the Machine Learning Operations world, and since, has interviewed the leading names around MLOps, Data Science, and ML. Since diving into the nitty-gritty of Machine Learning Operations he felt a strong calling to explore the ethical issues surrounding ML. When he is not conducting interviews you can find him making stone stacking with his daughter in the woods or playing the ukulele by the campfire.
David is one of the organizers of the MLOps Community. He is an engineer, teacher, and lifelong student. He loves to build solutions to tough problems and share his learnings with others. He works out of NYC and loves to hike and box for fun. He enjoys meeting new people so feel free to reach out to him!