The 7 Lines of Code You Need to Run Faster Real-time Inference

You've already trained your great neural network. It reaches 99.9% of accuracy and saves the world, so you would like to deploy it. However, it must run in real time and process data locally, and you don't want to build a web API. After all, you are a Data Scientist, not a Web Developer… So, is it possible to automatically optimize and run the network fast on the local hardware you have, not the hardware you wish you had? Absolutely! During the talk, Adrian will present the OpenVINO Toolkit. You'll learn how to automatically convert the model using Model Optimizer and run the inference with the Runtime. The magic with only seven lines of code. After all, you'll get a step-by-step jupyter notebook to try at home.

Take-aways

- you don't need to buy any additional hardware to use OpenVINO - even if you just use your CPU, your performance will be better than PyTorch or Tensorflow - the best way to learn OpenVINO is the OpenVINO Notebook repository available on GitHub

In this episode

Adrian Boguszewski

AI Software Evangelist, Intel

AI Software Evangelist at Intel. Adrian graduated from the Gdansk University of Technology in the field of Computer Science 6 years ago. After that, he started his career in computer vision and deep learning. As a team leader of data scientists and Android developers for the previous two years, Adrian was responsible for an application to take a professional photo (for an ID card or passport) without leaving home. He is a co-author of the LandCover.ai dataset and he was teaching people how to do deep learning. His current role is to educate people about OpenVINO Toolkit. In his free time, he’s a traveler. You can also talk with him about finance, especially investments.

Demetrios Brinkmann

Host

Demetrios is one of the main organizers of the MLOps community and currently resides in a small town outside Frankfurt, Germany. He is an avid traveller who taught English as a second language to see the world and learn about new cultures. Demetrios fell into the Machine Learning Operations world, and since, has interviewed the leading names around MLOps, Data Science, and ML. Since diving into the nitty-gritty of Machine Learning Operations he felt a strong calling to explore the ethical issues surrounding ML. When he is not conducting interviews you can find him making stone stacking with his daughter in the woods or playing the ukulele by the campfire.

Ben Epstein

Host

Ben was the machine learning lead for Splice Machine, leading the development of their MLOps platform and Feature Store. He is now a founding software engineer at Galileo (rungalileo.io) focused on building data discovery and data quality tooling for machine learning teams. Ben also works as an adjunct professor at Washington University in St. Louis teaching concepts in cloud computing and big data analytics.