Weight Initializations in Sigmoidal Feedforward Artificial Neural Networks

Abstract

The Proposed Weight Initialization, CONEXP is presented such that, the weights to hidden node are initialized in a way that the region of interest alternately contracts and expands statistically. The Proposed Weight Initialization, CONEXP is compared to conventional random weight initialization for 5 function approximation problems, and the proposed weight initialization technique performs better than conventional random weight initialization routine. To establish results student t-test is conducted to test the significance of results, and the results indicate that proposed technique is better than random weight initialization technique. The Proposed Weight Initialization, SRINWIT is presented, that initializes the input to hidden weight with small random values and hidden to output weights using Moore-Penrose inverse. The Proposed Weight Initial ization, SRINWIT is compared to conventional random weight initialization technique for 10 function approximation tasks. On comparing performance of these functions on statistics like mean, median, the performance of proposed technique is found to be supe rior to random and inverse weight initializations. It is thus, concluded that the proposed method is better weight initialization technique for sigmoidal feedforward artificial neu ral networks for at least these 10 function approximation task.