The art of forgetting. It’s been well depicted by human beings on numerous occasions. But what about machines? Are they capable of forgetting data as well?
This article answers that question.
As human beings, we are blessed with several cognitive skills. Including the habit of forgetting. People say it’s a boon to forget what’s unwanted. And a sin to forget the things that are important.
For example, people forget names, places and directions all the time! Forgetfulness can be caused by a number of factors. Such as retrieval failure, interference, failure to store or deliberate motivation.
Is it the same for machines?
We all know that artificial intelligence in machines try to mimic the natural cognitive process. With all the improvements and technical advancements made in machine learning as a field, mimicking the behaviour of the human brain, should be easier.
However, one question arises. Is the advancement enough to teach machines the habit of forgetting?
Yes. It is. Machines can and do forget what they learn in the past. There’s even a name for it. It’s called the “Catastrophic forgetting problem”. And it happens more often in artificial neural networks and deep learning.
Can it be resolved?
Interestingly enough, there are other strategic ways a modeller can force an algorithm to forget data. In systematic ways. Some of these techniques include LSTM — Long short term memory, Elastic Weight Consolidation (EWC) and the Bottleneck Theory.
The catastrophic forgetting problem can be resolved by providing samples from previous data. This data will be used by the model to retain the knowledge it has gained. This technique is called “Pseudo rehearsal”.
