Sequence to Sequence Learning with Neural Networks

This academic paper presents a novel approach to sequence-to-sequence learning using deep Long Short-Term Memory (LSTM) neural networks, addressing the limitations of traditional Deep Neural Networks (DNNs) on variable-length inputs and outputs. The core method involves using one LSTM to encode the input sequence into a fixed-dimensional vector and another deep LSTM to decode the target sequence from that vector. A significant finding is that reversing the order of the source sentence words dramatically improves performance by introducing more short-term dependencies. The authors demonstrate the effectiveness of their approach on an English to French machine translation task, achieving results that surpass a phrase-based statistical machine translation system and perform well on long sentences.