This paper introduces and evaluates a novel training method for neural networks: Dual Variable Learning Rates (DVLR). Building on insights from behavioral psychology, the dual learning rates are used to emphasize correct and incorrect responses differently, thereby making the feedback to the network more specific. Further, the learning rates are varied as a function of the network's performance, thereby making it more efficient. DVLR was implemented on three types of networks: feedforward, convolutional, and residual, and two domains: MNIST and CIFAR-10. The results suggest a consistently improved accuracy, demonstrating that DVLR is a promising, psychologically motivated technique for training neural network models.

PDF

In Proceedings of the International Joint Conference on Neural Networks, 2021.

@inproceedings{liner:ijcnn21, title={Improving Neural Network Learning Through Dual Variable Learning Rates}, author={Elizabeth Liner and Risto Miikkulainen}, booktitle={Proceedings of the International Joint Conference on Neural Networks}, month={ }, url="http://www.cs.utexas.edu/users/ai-lab?liner:ijcnn21", year={2021} }