KServe Live Coding Session

We start with the serialization of TensorFlow/PyTorch/SKLearn models into files and the deployment of an inference service on a Kubernetes cluster. Great MLOps means great model monitoring, so then we look at inference service metrics, model server metrics, payload logs, class distributions. For AI ethics on production, we use the explainers pattern with many different explainers, fairness detectors, and adversarial attacks. For integrations, we use the transformer pattern to process as well as to enrich the inference request with online features from a feature store.

Take-aways

We'll learn how to deploy models in KServe, monitor their metrics, explain the predictions and detect fairness, check adversarial robustness, enrich requests with online features and use the KServe SDK to create your custom model server.

In this episode

Theofilos Papapanagiotou

Data Science Architect, Prosus

Theo is a recovering Unix Engineer with 20 years of work experience in Telcos, on internet services, video delivery, and cybersecurity. He is also a university student for life; BSc in CS 1999, MSc in Data Coms 2008, and MSc in AI 2017. Nowadays he calls himself an ML Engineer, as he expresses through this role his passion for System Engineering and Machine Learning. His analytical thinking is driven by curiosity and a hacker’s spirit. He has skills that span a variety of different areas: Statistics, Programming, Databases, Distributed Systems, and Visualization.

Twitter

Demetrios Brinkmann

Host

Demetrios is one of the main organizers of the MLOps community and currently resides in a small town outside Frankfurt, Germany. He is an avid traveller who taught English as a second language to see the world and learn about new cultures. Demetrios fell into the Machine Learning Operations world, and since, has interviewed the leading names around MLOps, Data Science, and ML. Since diving into the nitty-gritty of Machine Learning Operations he felt a strong calling to explore the ethical issues surrounding ML. When he is not conducting interviews you can find him making stone stacking with his daughter in the woods or playing the ukulele by the campfire.

Alexey Grigorev

Host

Alexey lives in Berlin with his wife and son. He’s a software engineer with a focus on machine learning. He works at OLX Group as a Lead Data Scientist. Alexey is a Kaggle master and he wrote a couple of books. One of them is “Mastering Java for Data Science” and now he’s working on another one — “Machine Learning Bookcamp”.