The Dragon

Robert Heinlein, in the story ‘The Man who sold the Moon’ , predicted the forming of a private company which built a rocket to go to the Moon, and enabled the formation of a colony there – in 1950. I didn’t read it until much later – my first Heinlein story was ‘The Moon is a Harsh Mistress’, set on the moon colony.

In August 2020, the Dragon spacecraft built by Space X successfully returned two astronauts from the Space Station to a splashdown landing in the Gulf of Mexico. Images which follow are screen captures from the NASA livestreams.

On the pad
The capsule
21 seconds up
Returned

Never waste a good crisis

The Scottish government has a committee, the Scottish Advisory group on Economic recovery  “to provide independent expert advice on supporting the sectors and regions of Scotland’s economy to recover from the impact of Covid-19”. I was asked to contribute to the consultation which took place last week (closing date 31 May)

Gathering together my responses, paraphrasing the questions, and adding a list of references : 

Q: What will be the shape and form of the recovery from the crisis and what will be the implications for the future growth and structure of the economy?

What are the opportunities to boost private consumption to stimulate the recovery ? 

Which aspects of regulation might be relaxed to stimulate growth ?

A: View from Silicon Valley :  The construction business is back in full swing.  Large companies (Facebook, Google, Cisco, Microsoft) have moved smoothly to Working from Home and are going to continue doing that. Their user conferences and most other conferences have changed to being remote only. Sectors of the economy which have been impacted most severely, which have laid off most people, are non-essential retail stores, followed by restaurants. 

There’s an opportunity for a dramatic re-think of the concept of the ‘working week’, following on from WFH and the income support which has happened.  4 day weeks, enabling more businesses to be open every day, with support for scheduling so that people can usually spend their days off with family and friends, would enable more people to have flexibility in their lives between paid and volunteer work.

Trying to stimulate growth in take out food and non-essential retail would be a mistake, although income support for the employees is critical. There are opportunities for employment in essential retail, which needs more people and hours to maintain distancing and cleanliness.  

Q: What are the medium- to long-term consequences of the lockdown on businesses, including loss of employees, debt overhang, loss of markets, reduced investment and unemployment? 

A: The medium to long term consequences can be very positive; there has been a forced acceleration of companies retraining their teams to work in a distributed, remote, highly digital world.  Doing this well and early can be a competitive advantage; Scottish companies could offer remote teams who deliver work all over the world without having to travel there.  

Q: What can be done now to ensure the transition to a wellbeing-oriented, inclusive economy on a transition to net zero ?

How can the wellbeing of the people of Scotland flourish and what are the environmental implications of the crisis?

A: Three big things : infrastructure investment in Internet communications; enabling conversion of office and retail space to insulated residential accommodation with room for WFH; rethink transport policies to encourage electric bicycle and economical small scale delivery and pickup for physical goods going to consumers.

  1. Provision of fibre for broadband network has been a political minefield and remains inadequate for WFH over much of the country. Private only provision has not been effective. Cellular networks are not an adequate substitute.  Sweden’s strategy, for “market-driven development, completed by public efforts” is worth emulating.  

There’s a proposal for the creation of a National Investment Authority (NIA) by Cornell University law professors Saule Omarova and Robert C. Hockett, worth considering for the Scottish Investment Bank.

  1. WFH, made possible by improved communication infrastructure, reduces commuting costs, which reduces pollution and energy use as well as saving time and frustration. Every effort should be made to encourage this to continue.  This implies that office space should be able to be converted into residential accommodation.  The collapse in non-essential retail is an opportunity to take advantage of those existing buildings, particularly in small towns where the high street has been hollowed out, to convert them for residential use, with requirements in building codes for insulation and heat pumps. 
  1. Continuing the reduction in vehicle traffic which happened in the lockdown while enabling people to have freedom of movement requires a system scale re-think. Encouraging cycling, particularly e-bike use (as in the Netherlands and Germany) for personal transport and local deliveries, would contribute to well-being. 

References

Call for views https://consult.gov.scot/economic-development/call-for-views-advisory-group-on-economic-recovery/

https://blogs.microsoft.com/on-the-issues/2020/05/21/broadband-gap-covid-19-airband/

https://www.government.se/496173/contentassets/afe9f1cfeaac4e39abcdd3b82d9bee5d/sweden-completely-connected-by-2025-eng.pdf

http://www.circleid.com/posts/20200525-public-and-private-infrastructure-investment-alternatives/

https://www.oreilly.com/tim/21stcentury/

https://www.economist.com/business/2020/05/30/working-life-has-entered-a-new-era

https://www.blog.google/inside-google/working-google/working-from-home-and-office/ Sundar Pichai, CEO of Google and Alphabet Published May 26, 2020

https://www.specialityfoodmagazine.com/news/are-we-on-the-brink-of-a-british-food-revolution

Not yet the new normal

Snapshot for Silicon Valley: updated to add the link to the recording.

Had been asked to do a status report on the Valley for compatriots in Scotland with Christine Esson for the Scottish Business Network insights series..

Here’s the recorded interview https://www.youtube.com/watch?v=poXljpfVNFk

References

The California Consumer Privacy Act, discussed on the China Law blog.

A summary of a representative Venture Capital viewpoint from the point of view of the investors, rather than from founders looking for investment.

Elad Gil’s advice for founders.

Linley Spring Processor Conference Goes Virtual The Linley Group held their conference which gathers good quality marketing information about imminent AI hardware developments on Zoom, to excellent effect. They report 700 part-time attendees, compared to the usual 300 maximum in person for the event. Listening to it it seemed that the quality of technical information imparted was at least as good as it would have been at an in-person event.

Sherwood Partners doing two to five business windups per day. “This is the great unwinding,” Pichinson said. “We don’t know what’s happening, but we do know everything we believed in is changing. Everything we thought to be true may not be true.”

He expects the number of startup companies that he is ending from the coronavirus downturn to exceed the carnage left behind in the dot-com bust.

What I’m seeing isn’t layoffs – what I’m seeing is business continuing, though not business as usual. One company was acquired by a public company for cash, another couple of public companies are being acquired by much bigger companies, another needs advice about splitting off the original technology it started with, with the potential to re-capitalize and convert to a much more sustainable business model while compensating the original investors.

Delivery robots are out on the street in Mountain View

There have now been 100 reported deaths in Santa Clara County attributed to Covid-19 Vehicle traffic continues to be much lighter than before the lock-down; many more people are walking and cycling along the residential streets. We ride and walk for exercise, too.

Juiced E-bike, Trek road bike

Testing, tracing and tracking an outbreak in Seoul, South Korea. Paper It’s not clear when Santa Clara County is going to be able to implement this kind of tracing and follow up; without it removing the lockdown has a high likelihood of raising the case count and death rate again.

Call to action from Jim Yong Kim : social distancing, contact tracing, testing, isolation, and treatment are essential. Massachusetts is getting started on this.

ScaledML 2020 notes

26 – 27 Feb at the Computer History Museum in Mountain View

This was the 5th year of this conference. Metroid, who put it on, pick a useful mixture of academic and industry speakers, from people working at the front edge of getting products using machine learning into early adoption.  The most entertaining talk was from Josh Bloom, on applying ML in astrophysics. The most significant talks were from Jim Keller (Intel) on how Moore’s Law continues, and from Dennis Abts on his 14th chip design, Groq’s Tensor Streaming Processor. 

http://scaledml.org/2020/

——

Oren Etzioni Allen Institute for AI 

Paper https://www.technologyreview.com/s/615264/artificial-intelligence-destroy-civilization-canaries-robot-overlords-take-over-world-ai/ 

———

Metroid  turning research into product

Look for definition and understanding of ‘adequate accuracy’ for the context of the problem

Understand the rate of change of adequate accuracy – predictor of time to develop and investment requirements. Army of annotators available ? 

Can do model compression and optimization to match low capability edge AI chips. 

————-

Megan Kacholia VP engineering Google. https://ai.google/ 

https://www.blog.google/outreach-initiatives/accessibility/impaired-speech-recognition/

Tensor Flow TF 2.1 released Jan 2020 https://github.com/tensorflow/tensorflow/releases

————–

Andrej Karpathy Tesla https://www.tesla.com/autopilotAI 

Aiming for full self driving. Using the fleet (of customers’ cars) for data gathering.

——————

Ilya Sutskever Open AI 

Dactyl robot hand manipulation Rubik’s cube https://openai.com/blog/learning-dexterity/

Musenet for music . AI Dungeon game discussion at https://www.reddit.com/r/AIDungeon/

—————-

Wes McKinney Ursa Labs  forum for discussion https://discuss.ossdata.org/ 

Opensource support model Apache Arrow Looking for Swift and Julia developers

—————-

Savin Goyal Netflix Framework for AI development https://metaflow.org/ opensource

Sandbox, free, at AWS

———–

Panel discussion.  Another framework https://mlflow.org/  MLsys conference 

————–

Posters : Pure Storage;  Logical Clocks (Sweden) ; Samsung Iot chip, no system design

———–

Ion Stoica UC Berkeley  https://rise.cs.berkeley.edu/ 

————

David Aronchick Microsoft Leads open source machine learning at Azure

https://www.davidaronchick.com/  david_aronchick@microsoft.com

Structured schemas required for ML Ops. Design and test discipline for both data and algorithms.

———–

Joshua Bloom UC Berkeley 

https://bids.berkeley.edu/events/physics-machine-learning-workshop

Towards Physics-informed ML Inference In Astrophysics 

Searching for Planet 9   One hot encoding

Physics informed deep learning https://arxiv.org/abs/1711.10561

Physics Informed Deep Learning (Part I): Data-driven Solutions of Nonlinear Partial Differential Equations:  Maziar Raissi, Paris Perdikaris, George Em Karniadakis

Reverse-Engineering Deep ReLU Networks https://arxiv.org/abs/1910.00744

David Rolnick, Konrad P. Kording

——————-

Matei Zaharia Databricks Scaling Machine Learning Development with MLflow 

More ml devtools . Reproducible runs Auto logging to support data versioning

———

Jim Keller Intel 

Moore’s law continues  1000 scalars. Abstraction layers are critically important 

Extreme ultraviolet lithography is the next phase of chip manufacture.

EUV is a step function enables 100x finer printing

Once you have stable data and a stable platform, the platform can evolve from CPU to GPU to special purpose accelerator.

————–

Josh Romero Nvidia Scaling Deep Learning on the Summit Supercomputer 

Used Horovid (Uber) framework for DL training.  Needed hierarchical all reduce 

——–

Peter Mattson Google MLPerf: driving innovation by measuring performance 

Need benchmarks for training, inference, mobile. Hard to get contributors. MLCommons non-profit formed to encourage innovation. People’s Speech dataset aiming for 100k hours of transcribed speech by diverse speakers.

Sean Lie Cerebras Wafer-Scale ML 

——–

Dennis Abts  Ditching the ‘C’ in CPU: Groq’s Tensor Streaming Processor (TM)

https://research.google/people/author36240/

Dataflow in a superlane 220 Mibytes shared SRAM

Memory has an address and a direction 1 Teraop/sec/mmsquared

Deterministic instruction time INT8, FP16 

14th chip.  No arbiter, no replay mechanism, no flow control in chip, no hardware interlocks – orchestrated by the compiler. 

Click to access Groq-Rocks-NNs-Linley-Group-MPR-2020Jan06.pdf

Slides and video are now available at https://info.matroid.com/scaledml-media-archive-preview Matroid ask for an email address in exchange for access.

———–

Understanding the new business of AI

Adding to the agreement and reaction to the useful Andreesen Horowitz post on The New Business of AI from last week ..

Image from the 2017 AI Index, Stanford Institute for Human -centered AI

Reaching the sunlit uplands of AI is going to be rather harder than many of its investors and protagonists have predicted. https://a16z.com/2020/02/16/the-new-business-of-ai-and-how-its-different-from-traditional-software/

In particular, many AI companies have:

  • Lower gross margins due to heavy cloud infrastructure usage and ongoing human support;
  • Scaling challenges due to the thorny problem of edge cases;
  • Weaker defensive moats due to the commoditization of AI models and challenges with data network effects.

Adding a couple of examples to illustrate some of the systems design issues :

Frank Denneman, from VMware, on paralellism used for scaling training models https://frankdenneman.nl/2020/02/19/multi-gpu-and-distributed-deep-learning/

Emily Potyraj, from Pure Storage, on optimizing ECG data layout to improve deep learning training performance. https://towardsdatascience.com/what-format-should-i-store-my-ecg-data-in-for-dl-training-bc808eb64981

All of this points to a continuing requirement for a high degree of skilled problem analysis and systems design in order to make best use of AI/ML . There’s an opportunity for existing services companies to dramatically improve with judicious use of ML/AI .