本文關鍵詞：Web信息智能抽取技術的研究與實現　出處：《電子科技大學》2009年碩士論文　論文類型：學位論文

  更多相關文章： 信息抽取 規則生成器 模板生成器 增量/多頁處理

【摘要】： 隨著我國經濟的迅速發展,國家信息基礎設施建設強度加大加強和人民生活質量的提高,網絡已經深入人們生活的方方面面,成為工作或生活中不可缺少的一部分,怎樣快速有效的獲取Web上的信息,已經成為了一個重要的研究課題。但是網絡上的信息種類繁多、網頁結構形式多變,大多數網頁上還包含了許多廣告、導航、熱點鏈接等噪音信息,這些問題給研究者帶來了很大的困擾。而目前的信息抽取技術還存在很多不足:如僅能處理一種類型網頁,提取的信息細化程度低,準確率與效率矛盾、人工干預與智能化操作、不支持增量信息處理等問題。這就迫切需要一種全新的信息提取方法來解決這些問題,本課題就是在這種需求下產生的。 本文主要采用的是模板化的信息提取算法,先利用規則生成器識別網頁上的目標實體分隔符,然后由模板生成器把這些分割標記配置到模板中,最后由信息抽取器根據模板提取該站點的相關信息。具體創新點或關鍵技術如下: 1、通過分析的站點網頁結構,分析網頁結構布局形式和標簽的分布規律,并結合目前國內外的信息抽取技術,發明了一套可以定義任何網頁結構形式的模板,并設計出了一套模板自動配置方案; 2、設計了信息抽取器:實現了讀取模板,以及根據模板配置進行信息抽取的方法,并在此過程中增加了信息增量/多頁處理算法:采用增量/多頁算法來解決同一主題的內容分布在多個網頁的問題,即需要進行融合計算,以及解決不同時間段,主題網頁內容動態更新的問題,即要進行增量提取;去重處理算法:處理站點間相似或相同主題重復問題; 3、結果的結構化存儲:根據模板的配置,提取相關的信息,并采用結構化的形式進行保存;設計一個可動態擴展的信息提取系統:根據不同的需要,動態配置模板,不需要更改代碼。 本文在理論上提出了一套依據模板能自動提取各種類型網頁的信息抽取方案,并開發了相應的系統IWIES。實踐結果證明,本方案相對于常見的Web信息抽取技術方法具有更好的提取速度以及更高的準確率、召回率。
[Abstract]:With the rapid development of our country's economy, the strengthening of the national information infrastructure construction and the improvement of the people's quality of life, the network has gone deep into all aspects of people's life. Become an indispensable part of work or life, how to quickly and effectively obtain information on Web, has become an important research topic, but there are many kinds of information on the network. The structure of the web page is changeable, and most web pages also contain a lot of noise information, such as advertisement, navigation, hot link and so on. These problems have brought a great deal of trouble to the researchers. However, the current information extraction technology still has many shortcomings: only one type of web pages can be processed, the degree of information refinement is low, and the accuracy and efficiency are contradictory. Artificial intervention and intelligent operation do not support incremental information processing and so on. Therefore, a new information extraction method is urgently needed to solve these problems. This paper mainly uses the template-based information extraction algorithm, first using the rule generator to identify the target entity separator on the web page, and then the template generator to configure these segmentation tags into the template. Finally, the information extractor extracts the relevant information of the site according to the template. The specific innovation points or key technologies are as follows: 1. Through the analysis of the structure of the web page, the layout of the page structure and the distribution of tags, and combined with the current information extraction technology at home and abroad. A set of templates can define any form of web page structure, and a set of template automatic configuration scheme is designed. 2. The information extractor is designed: the method of reading the template and extracting the information according to the configuration of the template is implemented. In this process, the information increment / multi-page processing algorithm is added: the incremental / multi-page algorithm is used to solve the problem that the content of the same topic is distributed in multiple pages, that is, the fusion calculation is needed. And to solve the problem of dynamic updating of theme pages in different time periods, that is to do incremental extraction; De-reprocessing algorithm: to deal with similar or the same topic repeat problem between sites; (3) structured storage of results: according to the configuration of templates, the relevant information is extracted and stored in a structured form; Design a dynamic extensible information extraction system: according to different needs, dynamically configure the template without changing the code. In this paper, we propose a set of information extraction schemes based on template which can automatically extract all kinds of web pages, and develop the corresponding system IWIES. the practical results prove that. This scheme has better extraction speed, higher accuracy and higher recall than common Web information extraction methods.
【學位授予單位】：電子科技大學
【學位級別】：碩士
【學位授予年份】：2009
【分類號】：TP391.1

【引證文獻】

相關期刊論文 前2條

1 鄭思婷;楊p芑，

本文編號：1422856