Deterministic Nonmonotone Training
Peng, Chun-Cheng
In this book, we propose novel deterministic RNN training algorithms that adopt a nonmonotone approach. This allows learning behaviour to deteriorate in some iterations, nevertheless the overall learning performance is improved over time. The nonmonotone RNN training methods, which take their theoretical basis from the theory of deterministic nonlinear optimisation, aim at better exploring the search space and enhancing the convergence behavio...