Title : Recent Advances in Hierarchical Recurrent Neural Networks

 

Speker : 안 성 진 Ph.D (University of Montreal)

The recent resurgence of Recurrent Neural Networks (RNN) has achieved remarkable advances in sequential modeling. However, we are still missing many key abilities of RNN required to model more challenging yet important natural phenomena. In this talk, I introduce some recent advances in this direction, focusing on two new RNN architectures: the Hierarchical Multiscale Recurrent Neural Networks (HM-RNN) and the Neural Knowledge Language Model (NKLM).  In the HM-RNN, each layer in a multi-layered RNN learns different time-scales adaptively to the inputs. The NKLM deals with the problem of incorporating factual knowledge provided by a knowledge graph into RNNs.  I argue the advantages of these models and conclude the talk with a discussion on the key challenges that lie ahead.