• Hongguang Pan, Yunpeng Fu, Zhuoyi Li, Fan Wen, Jianchen Hu, and Bo Wu.
• College of Electrical and Control Engineering, Xi'an University of Science and Technology, Xi'an 710054, China; Key Laboratory of Industrial Internet of Things & Networked Control, Ministry of Education, Chongqing 400065, China. Electronic address: hongguangpan@163.com.
• Neuroscience. 2023 Jan 15; 509: 103112103-112.

AbstractReconstructing visual stimulus images from the brain activity signals is an important research task in the field of brain decoding. Many methods of reconstructing visual stimulus images mainly focus on how to use deep learning to classify the brain activities measured by functional magnetic resonance imaging or identify visual stimulus images. Accurate reconstruction of visual stimulus images by using deep learning still remains challenging. This paper proposes an improved deep generative multiview model to further promote the accuracy of reconstructing visual stimulus images. Firstly, an encoder based on residual-in-residual dense blocks is designed to fit the deep and multiview visual features of human natural state, and extract the features of visual stimulus images. Secondly, the structure of original decoder is extended to a deeper network in the deep generative multiview model, which makes the features obtained by each deconvolution layer more distinguishable. Finally, we configure the parameters of the optimizer and compare the performance of various optimizers under different parameter values, and then the one with the best performance is chosen and adopted to the whole model. The performance evaluations conducted on two publicly available datasets demonstrate that the improved model has more accurate reconstruction effectiveness than the original deep generative multiview model.Copyright © 2022 IBRO. Published by Elsevier Ltd. All rights reserved.

12 keywords...

hide…