Artificial neural networks sleep for better learning

According to age, a person needs 7 to 13 hours of complete sleep in 24 hours a day. During the process of sleeping, many things happen in the body, including slow heartbeat and breathing, slowing down of the body's metabolism and regulation of hormone levels. In such a state, the body is completely at rest, but in the meantime, the brain takes a different behavior.

The brain is very busy during sleep, because it learns what we have learned during the day. It repeats better. In fact, sleep helps to reorganize information and memories in order to present them in the most efficient way possible when awake. It is interesting to know that the same procedure is true for artificial neural networks, these strange human creations! They are used by numerous technologies and systems, from basic science and medicine to finance and social networks. In this regard, it is interesting to know that although in some ways, these human creatures have achieved superhuman functions, such as very high computing speed, but they have failed in one key aspect. When artificial neural networks learn things sequentially, they overwrite new information on top of previous information, such a phenomenon is called "catastrophic forgetting".

In contrast to artificial neural networks, the human brain is constantly learning and incorporating new data alongside existing knowledge. In this learning style, when new teachings are combined with regular periods of sleep, it helps to consolidate information better and learn more deeply. So on November 18, 2022, computational biologists are looking further into how biological models can help reduce the risk of catastrophic forgetting in artificial neural networks and increase their utility in a wide range of research fields.

Sleeping helps artificial neural networks learn better. It artificially mimics the behavior of natural nervous systems. In fact, in this new approach, instead of the information being constantly transferred and the entire system constantly connected, the information is transferred as discrete events (in the form of clusters) at specific times.

During this procedure of scientists They found that when these clustered networks learn new information in the form of occasional, non-continuous periods (similar to sleep), the rate of catastrophic forgetting is reduced. According to the authors of the study, like the human brain, sleep allows neural networks to recall old memories without explicitly using old training data. In the human brain, memories are represented by patterns called synaptic weight, which means the strength or range of connection between two neurons.

When the cluster networks receive new information in the form of occasional periods and out of Continuous state (which is similar to sleep.) they learn, the rate of catastrophic forgetting is reduced.

Basically, when we are learning new information, neurons light up in a certain order, and this issue is also a factor for increasing synapses among different neurons. During sleep, the same synaptic patterns (which are carried out in the awake state) repeat themselves, which is called reactivation or replay.

The small space between two neurons is called a synapse, through which the nerve impulse It is transmitted from transmitter to receiver by neurotransmitters.

Synaptic flexibility, which includes the capacity to change or reshape connections, is also present during sleep and can strengthen synaptic weight patterns that represent memory and help prevent forgetting or rewriting information over new information. The researchers of this study applied this approach to artificial neural networks and found that such a procedure helps these networks avoid catastrophic forgetting. The result of this research means that these networks can continuously learn new information, just like humans or animals.

In the end, it should be noted that understanding how the human brain processes information during sleep that can Helping to strengthen memory is one of the fields of study that can lead to significant progress in improving artificial neural networks.

• The way artificial intelligence learns is similar to the way our brain works!
• The main source of the article

Source: Science Daily