Scaled Machine Learning


Bill Dally, Nvidia

Matroid and the Stanford center for image system engineering ran the 3rd year of the ScaledML conference yesterday, 24 March 2018. It was a concentrated survey of work in progress in machine learning, with admirably little overt advertising. The overall impression is of enormous actual potential, orthogonal to enormous hype and inflated expectations, significant uncertainty about what will actually get done, and of a lot of work in progress on necessary infrastructure in hardware, architecture, languages, systems, and education.

Agenda and speaker list

08:45 – 09:00 Introduction Reza Zadeh Matroid
09:00 – 10:00 Ion Stoica Databricks
10:00 – 11:00 Reza Zadeh Matroid
11:00 – 11:30 Andrej Karpathy Tesla
11:30 – 12:00 Jennifer Chayes Microsoft Research
13:00 – 14:00 Jeff Dean Google
14:00 – 14:30 Anima Anandkumar Amazon
14:30 – 15:00 Ilya Sutskever Open AI
15:00 – 15:30 Francois Chollet Google
16:00 – 17:00 Bill Dally Nvidia
17:00 – 17:30 Simon Knowles Graphcore
17:30 – 18:00 Yangqing Jia Facebook

From my notes :
The successor to the AMP Lab at Berkeley is RISE lab, building Real-time Intelligent Secure Explainable applications to make low-latency decisiongs on live data with strong security. (Ion Stoica). Note the remark about Explainable; this came up as a common theme.
Being able to examine detector errors and mistakes came up again in Reza Zadeh’s Matroid demonstration – this was the only live product shown. A user can build a detector with multiple attributes to pick out images from streaming video.
Bill Dally (Chief Scientist, Nvidia) reckons that Moore’s Law is dead; Simon Knowles (Graphcore) gave a more reasoned explanation about possible performance gains from hardware improvements over the next 10 years.


Graphcore hardware, use of BSP – Simon Knowles

Jeff Dean’s slides

Bill Dally’s slides

Anima Anandkumar

Ion Stoica

Francois Cholet on Keras

Ilya Sutskever

Jennifer Chayes

Yangqing Jia

Transporting people like IP packets


Transportation as a Service might be better thought of as transportation as a system; the items to be transported are people, and things.
An Economist special report, ‘Reinventing Wheels ‘, from early March 2018, discusses autonomous vehicles, urban planning, and possible changes in how people live, while ignoring a significant part of the overall transportation system ; there’s very little mention of pedestrians, and no mention of bicycles. This ignores the actual flexibility in the system if it allows for people to walk or bicycle for part of their journey. There are many urban and suburban trips where the time to drive and park a vehicle exceeds the time to walk or cycle the same trip.
Contrast this with ‘tim in Graz’ (Austria).
“At selected public transport stops, the tim locations, the Graz Lines bundle additional mobility services as a supplement to public transport:

  • e-car sharing
  • conventional car sharing
  • rental car for longer distances or long-term use
  • e-taxis with exclusive stand
  • public charging stations for private electric cars
  • bicycle parking

This offer makes it easier to dispense with your own car because it makes it easy and convenient to access a car when needed. You can also park your own e-car at the e-charging station and change to bus or train. Bicycle parking makes the change from the bike to public transport comfortable.”
Being able to change, at a location for which there is a business model that scales up, between different modes of transport is going to be important to the improvement of frequent, flexible, movement of people. Amazon, FedEx, DHL and the other shipping companies use this model for moving things.

TIM – Graz, Austria

Some people are usually early


Here’s something I’d like to be able to do. I’m usually early for things, and especially for flights, given the uncertainty of travel to airports, and the uncertainty of clearing security at airports. Some flight segments have very frequent service; I’d like to be able to get to the airport, clear security, then take the next flight with space available (whether space with legroom, or not) to where I’m going (which is maybe another airport for a connecting flight). I’d pay more for a ticket with that flexibility. Trying to get onto a different flight that the one you originally booked while you are at the airport usually requires going back outside security, and queuing for check-in along with all the people who have luggage to check; that’s not what I do.

This would require the ticket I booked originally to be treated like currency, so that it could be used as credit for the earlier seat; there should be a value to the airlines who can support this rebooking, because airline seats have a value up until the flight closes. If a flight leaves with empty seats for which someone would have paid, that’s a lost sale.

This app could start out by being be a representation of “same day flight changes” , kept up to date with all the policies and constraints described here.