Better Binarization for the CKY Parsing - Song, Xinying; Ding, Shilin; Lin, Chin-Yew (2008)
We present a study on how grammar binarization empirically affects the efﬁciency of the CKY parsing. We argue that binarizations affect parsing efﬁciency primarily by affecting the number of incomplete constituents generated, and the effectiveness of binarization also depends on the nature of the input. We propose a novel binarization method utilizing rich information learnt from training corpus. Experimental results not only show that different binarizations have great impacts on parsing efﬁciency, but also conﬁrm that our learnt binarization outperforms other existing methods. Furthermore we show that it is feasible to combine existing parsing speed-up techniques with our binarization to achieve even better performance.