頁籤選單縮合
題 名 | A Pragmatic Chinese Word Segmentation Approach Based on Mixing Models |
---|---|
作 者 | Jiang, Wei; Guan, Yi; Wang, Xiao-long; | 書刊名 | International Journal of Computational Linguistics & Chinese Language Processing |
卷 期 | 11:4 民95.12 |
頁 次 | 頁393-415 |
分類號 | 312.23 |
關鍵詞 | Word segmentation; N-Gram; Maximum entropy model; Rough sets; Word cluster; Machine learning; |
語 文 | 英文(English) |
英文摘要 | A pragmatic Chinese word segmentation approach is presented in this paper based on mixing language models. Chinese word segmentation is composed of several hard sub-tasks, which usually encounter different difficulties. The authors apply the corresponding language model to solve each special sub-task, so as to take advantage of each model. First, a class-based trigram is adopted in basic word segmentation, which applies the Absolute Discount Smoothing algorithm to overcome data sparseness. The Maximum Entropy Model (ME) is also used to identify Named Entities. Second, the authors propose the application of rough sets and average mutual information, etc. to extract special features. Finally, some features are extended through the combination of the word cluster and the thesaurus. The authors’ system participated in the Second International Chinese Word Segmentation Bakeoff, and achieved 96.7 and 97.2 in F-measure in the PKU and MSRA open tests, respectively. |
本系統中英文摘要資訊取自各篇刊載內容。