new icn messageflickr-free-ic3d pan white
Introducing MosaicML — Accelerating AI Algorithmically | by jurvetson
Back to photostream

Introducing MosaicML — Accelerating AI Algorithmically

It's good be investing in Naveen Rao and team again! I led the first investment in his last company, Nervana, which Intel soon acquired and promoted Naveen to run all AI teams at Intel. He left to help democratize corporate access to cutting edge machine learning across cost-effective cloud compute platforms.


His new company, MosaicML came out of stealth today: founder's blog & Forbes: "Given the increasing cost and environmental impact of AI computation, his timing may be perfect once again. Rao believes he and his team can spread a revolution in training AI models by offering model optimization as a service."


In short, Mosaic improves ML training efficiency algorithmically. By integrating the latest advances from academia and industry and testing on diverse hardware platforms, their mosaic of methods has already seen a 4x improvement in the price/performance of training. We expect Mosaic’s Law to continue for the near future, where learning/$ quadruples every year.


Mosaic’s first tools for the ML community are the open-source Composer and Efficiency Explorer to visualize the tradeoffs between cost, time, and quality in training neural nets. This will lead to training tools and optimizers that are cross-cloud and hardware agnostic.


Our group photo is from the all hands team meeting at SF office opening.


P.S. The improvement of algorithms outpacing Moore’s Law reminds me of conversations I had with Jesse Levinson, co-founder of Zoox, about computer chess: the improvements to Stockfish algorithms outpaced hardware improvements over 2014-2017. And 10 years earlier, in 2007, Geordie Rose, co-founder of D-Wave, gave a more stark comparison from the domain of factoring integers: the latest algorithm running on a 30-yr old Apple ][ would beat the 30-year old algorithm running on an IBM Blue Gene/L (the supercomputer of the day). And in quantum computing, algorithmics advances are so important, they carry the names of their inventors (Shor, Grover, etc.).


And from OpenAI last year: "We’re releasing an analysis showing that since 2012 the amount of compute needed to train a neural net to the same performance on ImageNet classification has been decreasing by a factor of 2 every 16 months. Compared to 2012, it now takes 44 times less compute to train a neural network to the level of AlexNet (by contrast, Moore’s Law would yield an 11x cost improvement over this period). Our results suggest that for AI tasks with high levels of recent investment, algorithmic progress has yielded more gains than classical hardware efficiency."

5 faves
1 comment
Taken on June 15, 2021