Accession Number : AD1040293


Title :   Deep Gate Recurrent Neural Network


Descriptive Note : Conference Paper


Corporate Author : University of Waikato Hamilton New Zealand


Personal Author(s) : Gao,Yuan ; Glowacka,Dorota


Full Text : https://apps.dtic.mil/dtic/tr/fulltext/u2/1040293.pdf


Report Date : 22 Nov 2016


Pagination or Media Count : 16


Abstract : This paper explores the possibility of using multiplicative gate to build two recurrent neural network structures. These two structures are called Deep Simple Gated Unit (DSGU) and Simple Gated Unit (SGU), which are structures for learning long-term dependencies. Compared to traditional Long Short-Term Memory (LSTM) and Gated Recurrent Unit (GRU), both structures require fewer parameters and less computation time in sequence classification tasks. Unlike GRU and LSTM, which require more than one gate to control information flow in the network, SGU and DSGU only use one multiplicative gate to control the flow of information. We show that this difference can accelerate the learning speed in tasks that require long dependency information. We also show that DSGU is more numerically stable than SGU. In addition, we also propose a standard way of representing the inner structure of RNN called RNN Conventional Graph (RCG), which helps to analyze the relationship between input units and hidden units of RNN.


Descriptors :   ARTIFICIAL NEURAL NETWORKS , network topology , information processing , mathematical models , graphs , classification


Subject Categories : Information Science


Distribution Statement : APPROVED FOR PUBLIC RELEASE