The architecture design and training optimization of spiking neural network with low-latency and high-performance for classification and segmentation

Abstract

Spiking Neural Networks (SNNs) are the new third generation of bio-mimetic neural networks suitable for large-scale parallel computation due to its advantages of low power consumption and low latency. However, most of the training algorithms and network architectures of existing SNNs are designed on the basis of traditional Artificial Neural Networks (ANNs), which require a large number of time-steps for inference and have high requirements for membrane potential storage space, resulting in large latency and consuming large memory resources. In this paper, we propose a spiking neurons-shared ResNet network (Spiking-NSNet) for image classification and a spiking semantic segmentation network (Spiking-SSegNet) for image segmentation based on our designed neurons-shared architecture and hybrid attenuation strategy. Firstly, a novel Neurons-Shared Block (NS-Block) are designed for locally sharing membrane potential parameters of neurons to realize the reduction of parameters and accelerate the inference speed. Secondly, different attenuation factor are set for neurons in different NS-Blocks, so that different neurons have different activities and are more in line with the biological dynamic characteristics. Finally, a temporal correlated(TC) loss algorithm is designed to optimize the SNN direct training process for faster convergence and better performance. Based on above improvements, the Spiking-NSNet and the Spiking-SSegNet are designed by using the architectures of ResNet and UNet, respectively, and are trained by realizing the pre-training and transfer learning of SNNs for the first time. The experiments show that the proposed Spking-NSNet obtains high recognition accuracies of 94.65 %, 77.4 % and 79 % with lower latency of four time steps on static dataset of CIFAR-10, CIFAR100 and dynamic dataset of DVS-CIFAR-10. The mIoUs of designed Spiking-SSegNet can achieve 43.2 % and 53.4 % on static dataset of PASCAL-VOC2012 and dynamic dataset of DDD17. Thus, under the recognition and segmentation tasks, the proposed methods can effectively reduce the number of time steps and model parameters for model's training and inference comparable to that of traditional ANN models.

Keywords: Hybrid attenuation factors; Neurons-shared architecture; Semantic segmentation; Spiking neural network; Temporal correlated loss.