Accession Number:

AD1040293

Title:

Deep Gate Recurrent Neural Network

Descriptive Note:

Conference Paper

Corporate Author:

University of Waikato Hamilton New Zealand

Personal Author(s):

Report Date:

2016-11-22

Pagination or Media Count:

16.0

Abstract:

This paper explores the possibility of using multiplicative gate to build two recurrent neural network structures. These two structures are called Deep Simple Gated Unit DSGU and Simple Gated Unit SGU, which are structures for learning long-term dependencies. Compared to traditional Long Short-Term Memory LSTM and Gated Recurrent Unit GRU, both structures require fewer parameters and less computation time in sequence classification tasks. Unlike GRU and LSTM, which require more than one gate to control information flow in the network, SGU and DSGU only use one multiplicative gate to control the flow of information. We show that this difference can accelerate the learning speed in tasks that require long dependency information. We also show that DSGU is more numerically stable than SGU. In addition, we also propose a standard way of representing the inner structure of RNN called RNN Conventional Graph RCG, which helps to analyze the relationship between input units and hidden units of RNN.

Subject Categories:

  • Information Science

Distribution Statement:

APPROVED FOR PUBLIC RELEASE