頁籤選單縮合
題 名 | 中文資訊檢索測試集之設計與製作=The Design and Implementation of the Chinese IR Benchmark |
---|---|
作 者 | 陳光華; 江玉婷; | 書刊名 | 資訊傳播與圖書館學 |
卷 期 | 6:3 2000.03[民89.03] |
頁 次 | 頁61-80 |
分類號 | 028.7 |
關鍵詞 | 資訊檢索測試集; 資訊檢索評估; 文件集; 查詢主題; 相關判斷; Information retrieval benchmark; Information retrieval test collection; Information retrieval evaluation; Document set; Topic; Relevance judgment; |
語 文 | 中文(Chinese) |
中文摘要 | 在國內資訊檢索研究已日趨受到重視,卻缺少合適的測試評估機制之背景下,本 文參考國外各測試集的結構、特性與建構經驗,設計中文資訊檢索測試集的方法與程序,發 展華文世界第一套中文資訊檢索測試集,實際地進行測試集的規劃建置工作。本研究所建立 的測試集包括 132,207 篇新聞文件、50 個查詣主題、以及文件與查詣主題間的相關判斷, 平均每個查詣主題有 16.32 篇相關文件。 研究結果顯示,以統計抽樣的觀點,本測試集的 文件數量具有一定的效度;查詣主題呈現詳盡且多樣化的查詣需求,能反映真實的檢索情況 ;由三位判斷者進行的相關判斷具有顯著的一致性,推斷它們是具有可佔度的。本測試集不 僅具有可應用性,並能作為進一步資訊檢索與評估研究之基礎。 |
英文摘要 | The research and development of information retrieval has made considerable progress recently.However, there is not any applicable test mechanism for system evaluation in the Chinese research society. This paper reports our research on the design and implementation of the first Chinese information retrieval benchmark. According to the framework and contents of the existing foreign benchmarks, we develop a methodology to establish the Chinese IR benchmark. An IR benchmark consists of three parts: document set, topic set, and relevance judgments. our document set contains 132,207 documents collected from news web sites, the topic set contains 50 topics transformed from real users' information needs, and each topic has on the average16.34 related documents as a result of the relegance judgments. The results of our research show that the quantity of document set is valid from the viewpoint of sampling statistics. The topics reveal multiple kinds of information need, and they also reflect certain real retrieval environment. Besides, the judgments given by three judges have exhibited significant consistency, so we conclude their reliability. Although the benchmark is in its first edition, it possesses a complete structure and medium scale. On this basis, it is readily feasible to expand this benchmark's current scale to a proper large one in the near future. |
本系統中英文摘要資訊取自各篇刊載內容。