Today’s AI models can for the most part be trained to do just one task.
When retraining a neural network, it will most likely “forget” all previously acquired knowledge.
Google is developing a “next-generation AI architecture” that should be much more versatile. Last Thursday, Jeff Dean, senior vice president of Google Research, blogged about Pathways, an AI project aimed at making it easier to develop neural networks that can learn thousands, possibly millions, of tasks.
At the same time, the knowledge gained in learning how to perform one task will be generalized to help in solving subsequent tasks. A typical neural network today can process text, audio, or video, but not all three types of data. The Pathways project envisions the creation of neural networks that can combine different types of information to make more accurate decisions. “The result is a model that is prone to deeper inference, less prone to error and bias,” Dean wrote.
In addition to the already mentioned advantages, the new architecture will also be faster and much more energy-efficient, since it will only activate those parts of the neural network that are absolutely necessary for the given task. Dean said applying a similar “sparse activation” technology to Switch Transformation’s natural language processing model and another AI model, called GShard, reduced their energy consumption by a factor of ten. Google’s replacement of multiple AI models with a single multipurpose neutron network will provide important business benefits.
One of the outcomes of the Pathways project could be the improvement of Waymo’s autonomous driving system by increasing the efficiency of information processing. In the public cloud, providing customers with more versatile neural networks could help Google make its AI-driven services more competitive. In addition, Pathways will enable the company to improve the services it provides to consumers, in particular Google Search.
“Pathways will allow a single AI system to summarize thousands or millions of tasks, understand different types of data, and do it with amazing efficiency,” said Dean. It is such a system that can lead us from the stone age of artificial intelligence, “the era of single-purpose models that simply recognize patterns”, to a model in which “general-purpose smart systems will provide a deeper understanding of our world and can adapt to new needs.”
