KServe Live Coding Session
We start with the serialization of TensorFlow/PyTorch/SKLearn models into files and the deployment of an inference service on a Kubernetes cluster. Great MLOps means great model monitoring, so then we look at inference service metrics, model server metrics, payload logs, class distributions. For AI ethics on production, we use the explainers pattern with many different explainers, fairness detectors, and adversarial attacks. For integrations, we use the transformer pattern to process as well as to enrich the inference request with online features from a feature store.
Take-aways
In this episode
Theofilos Papapanagiotou
Data Science Architect, Prosus
Theo is a recovering Unix Engineer with 20 years of work experience in Telcos, on internet services, video delivery, and cybersecurity. He is also a university student for life; BSc in CS 1999, MSc in Data Coms 2008, and MSc in AI 2017. Nowadays he calls himself an ML Engineer, as he expresses through this role his passion for System Engineering and Machine Learning. His analytical thinking is driven by curiosity and a hacker’s spirit. He has skills that span a variety of different areas: Statistics, Programming, Databases, Distributed Systems, and Visualization.
Demetrios Brinkmann
Host
Demetrios is one of the main organizers of the MLOps community and currently resides in a small town outside Frankfurt, Germany. He is an avid traveller who taught English as a second language to see the world and learn about new cultures. Demetrios fell into the Machine Learning Operations world, and since, has interviewed the leading names around MLOps, Data Science, and ML. Since diving into the nitty-gritty of Machine Learning Operations he felt a strong calling to explore the ethical issues surrounding ML. When he is not conducting interviews you can find him making stone stacking with his daughter in the woods or playing the ukulele by the campfire.
Alexey Grigorev
Host
Alexey lives in Berlin with his wife and son. He’s a software engineer with a focus on machine learning. He works at OLX Group as a Lead Data Scientist. Alexey is a Kaggle master and he wrote a couple of books. One of them is “Mastering Java for Data Science” and now he’s working on another one — “Machine Learning Bookcamp”.