Earlier today I had the privilege of giving a talk in the Department of Computer Science at the University of Oxford. The topic was “real-time machine learning at industrial scale” which is core to what we are doing here at TUMRA. More specifically it was about the epic battle between the accuracy of machine learning models, and the latency of their answers.

The key concept I wanted to emphasise is given the choice of a “best guess” answer now, versus a “better answer” later – I’m going to take the “best guess” almost every time. When you are interacting with people through a web/mobile app you have their attention for a very short period of time. Choosing algorithms, techniques, and even cheats/hacks that shave milliseconds off your processing time are an everyday necessity. However, there are definitely some instances where you are prepared to wait seconds, minutes, hours or even days for the “best answer”; but more often than not a “best guess” is good enough.

Abstract

Right now in institutions around the world, some of the greatest minds in computer science and statistics are coming up with amazing new algorithms and mathematically beautiful solutions. However it’s entirely possible that the solutions they conceive will be impracticable in industry. The reason is simple; “the best answer is useless if it arrives too late to do anything with it”. The key principle here is the compromise between ‘accuracy’ and ‘latency’. In this presentation I describe examples where this holds true, and how I am using real-time machine learning models to solve challenges in eCommerce, Financial Services and Media companies.

Slides



Errata

- The Oxford vs Cambridge graph in Slide #30 can be downloaded as a Scalable Vector Graphic image (8.6MB), alternatively I’ve rendered it as a 6,000px by 10,000px PNG image (82MB).
- Sadly we ran out of time (and had no internet connectivity) but I did mention the real-time demographics demo which can be found here with a write-up here.