Neural networks also need states that resemble dreams. This conclusion was reached by researchers from the Los Alamos National Laboratory. The results of their research are published by EurekAlert!
The instability associated with continuous self-learning in artificial intelligence is solved by introducing the system into a state similar to sleep cycles. Such rest periods cause a positive response from the algorithms.
“We study pulsed neural networks – systems that learn just like a living brain,” said computer scientist from Los Alamos Laboratory Jizing Watkins. “We were fascinated by the prospect of learning a neuromorphic processor in a way similar to how people and other biological systems learn from the environment as children develop.”
However, Watkins and the research team found that network modeling became unstable after long periods of unattended training. But after scientists exposed the neural networks to conditions similar to the waves that the living brain experiences during sleep, stability was restored. “It looked like we were giving neural networks something like an analogue of a quiet night’s rest,” Watkins said.
“The question of how to prevent instability in learning systems arises only when trying to use neuromorphic processors,” said computer scientist from Los Alamos and research co-author Garret Kenyon. “The vast majority of machine learning, deep learning, and AI researchers never run into this problem because in the very artificial systems they study they can afford to perform global mathematical operations that affect the overall dynamic gain of the system.”
The decision to let the network rest was almost the last thing scientists could come up with. They experimented with various types of noise, roughly comparable to the static that you might encounter between stations when tuning in the radio. The best results were obtained when the researchers used waves of the so-called Gaussian noise. They suggest that noise mimics the entry of biological neurons into the slow sleep phase. The results show that slow sleep can partly help keep cortical neurons stable and not hallucinate.
The next goal of the group is to implement their algorithm on the neuromorphic Intel Loihi chip. Scientists hope that by letting him sleep from time to time, they will make his work stable. If the results confirm the need for sleep in an artificial brain, it is expected that the same will be true for androids and other intelligent machines that will appear in the future.