Utilizing the Learned Latent Structure from Dimensionality Reduction Algorithms to Prevent the Effects of Catastrophic Forgetting in Neural Networks

Overview

In recent years, the neural network has become one of the most powerful tools in the field of machine learning. However, while neural networks are extremely good at learning a singular task, they struggle when being trained on more than one. In my research, I attempt to overcome this problem by automatically separating the different tasks which a machine learning model is required to learn. My solution takes advantage of the different appearance of samples across tasks, which can be automatically quantified using dimensionality reduction algorithms, such as PCA (Principal Component Analysis). By using these algorithms to separate a dataset based on its content, we can approximate a separation of each task that needs to be learned. In order to test the effectiveness of this approach, I tested this novel algorithm's ability to learn to identify handwritten Arabic numerals, in both their western and Persian forms. Compared to simply training a single neural network on the combined dataset, the novel approach was only able to achieve a 0.58% increase in accuracy and was able to split the dataset into separate tasks with 95.68% accuracy. Through my experimentation, I was unable to find a practically useful improvement in accuracy with the novel algorithm. However, the performance increase which was achieved, along with the high accuracy of the task separation, shows some promise in the tested algorithm. Through improvements to the process by which tasks are separated, it’s possible that a more significant performance increase could be achieved. TENNESSEE

Competition history

• JSHS 2023 Category not listed

Resources

• National JSHS abstract book ↗

Related projects

Closest projects by meaning, across every fair and year in the corpus.