頁籤選單縮合
題 名 | Analyzing the Performance of Message Understanding Systems |
---|---|
作 者 | Bagga,Amit; Biermann,Alan W.; | 書刊名 | International Journal of Computational Linguistics & Chinese Language Processing |
卷 期 | 3:1 1998.02[民87.02] |
頁 次 | 頁1-26 |
分類號 | 312.121 |
關鍵詞 | Information extraction; Domain complexity; Analysis of systems; Message understanding conferences; |
語 文 | 英文(English) |
英文摘要 | In this paper we describe a method of classifying facts (information) into categories or levels; where each level signifies a different degree of difficulty of extracting the fact from a piece of text containing it. Based on this classification mechanism, we propose a method of evaluating a domain by assigning to it a "domain number" based on the levels of a set of standard facts present in the articles of that domain. We then use the classification mechanism to analyze the performances of three MUC systems (BBN, NYU, and SRI) based on their ability to extract a set of standard facts (at different levels) from two different MUC domains. This analysis is then extended to analyze the role of coreferencing in the performance of message understanding systems. The evaluation of a domain based on the "domain number" assigned to it is a big step up from methods used earlier (which used vocabulary size, average sentence length, the number of sentences per document, etc.). Moreover, the use of the classification mechanism as a tool to analyze the performance of message understanding systems provides a deeper insight into these systems than the one provided by obtaining the precision and recall statistics of each system. |
本系統中英文摘要資訊取自各篇刊載內容。