Coffee Sessions #170

All the Hard Stuff with LLMs in Product Development

Explore the intricate challenges associated with the implementation of Large Language Models (LLMs), encompassing critical aspects like security concerns and the comprehensive measures implemented to effectively mitigate potential attacks. Highlight the pivotal role played by the collaborative efforts between skilled ML engineers and astute product managers, underscoring their synergy in achieving a triumphant implementation. Probe deeper into the strategic significance of not only identifying leading indicators but also meticulously quantifying the Return on Investment (ROI) for diverse AI initiatives. This entails a profound assessment of the tangible value brought forth by these initiatives through a thorough evaluation of their performance and impact on organizational objectives.

In this episode

Phillip Carter

Phillip Carter

Principal Product Manager, Honeycomb

Phillip is on the product team at Honeycomb where he works on a bunch of different developer tooling things. He's an OpenTelemetry maintainer -- chances are if you've read the docs to learn how to use OTel, you've read his words. He's also Honeycomb's (accidental) prompt engineering expert by virtue of building and shipping products that use LLMs. In a past life, he worked on developer tools at Microsoft, helping bring the first cross-platform version of .NET into the world and grow to 5 million active developers. When not doing computer stuff, you'll find Phillip in the mountains riding a snowboard or backpacking in the Cascades.



Demetrios Brinkmann

Demetrios Brinkmann


Demetrios is one of the main organizers of the MLOps community and currently resides in a small town outside Frankfurt, Germany. He is an avid traveller who taught English as a second language to see the world and learn about new cultures. Demetrios fell into the Machine Learning Operations world, and since, has interviewed the leading names around MLOps, Data Science, and ML. Since diving into the nitty-gritty of Machine Learning Operations he felt a strong calling to explore the ethical issues surrounding ML. When he is not conducting interviews you can find him making stone stacking with his daughter in the woods or playing the ukulele by the campfire.