詞典與統(tǒng)計相結(jié)合的中文分詞的研究.pdf

1、武漢理工大學(xué)碩士學(xué)位論文詞典與統(tǒng)計相結(jié)合的中文分詞的研究姓名：岳中原申請學(xué)位級別：碩士專業(yè)：計算機應(yīng)用技術(shù)指導(dǎo)教師：胡燕20100501AbstractWiththedevelopmentoftheinformationize，itiseasytoobtainlargeamountofinformationHowevertodealwithmassiveinformationmanuallyisnotpossible，needtelyo

2、nthehelpofcomputerAnddifferentfromWesternlanguages，Chinese，betweenwordsinnoobvioussignssegmentationSo，tomakethecomputercapableofhandlingChinesetext，textmustdochinesewordsegmentationfirstAsthecomplexityofchinesesyntacticn

3、etworkandthethecontinualemergenceofnewwords，ChinesewordsegmentationsystemhasnotachievedsatisfactoryresultsThispaperanalyzestheactualuseofChinesewordsegmentationalgorithmmanykindsofdictionarystructureStudiedthecurrentprob

4、lemsofChinesewordsegmentationInthispaperweusedthecombinationmethodbasedonstatisticalanddictionaryAchieveImprovementsinvariousaspectsFirst，wedividethewholetextintoshortersenteneesaccordingtothePunetuationsinthistextinthes

5、tatistics，throughthestatisticsofresultsofthefragmentationfromChinesewordsegmentationIdentifiedtheunknownwordsinthetextappearedmorethanonceAndaddedtothetemporarydictionaryImprovedthestructureofthedictionaryPutdictionaryin

6、toasinglebasicdictionaryandextendeddictionaryThispaperdescribesaChinesewordsegmentationalgorithmmethodbasedonstatisticalanddictionaryweinereasethenumberofdictionaries，weaddsomesPeeialdictionarieswhichcanbeusedtoeliminate

7、theambiguousnessesandreeognizenewwordsduringthesegmentationProcessbesidesthebasicdictionaryAtthesametimewereconstrutedthedatastructureofbasicdictionaryinthememoryofcomputerbyusingdatastructure”Hashtable“wechoosethefirstt

8、wosinglecharaetersofeverywordinthebasicdictionaryasthekeywordsofthemainandsubHashtables，theremanentwordsarestoredinallarrayaceordingtolengthWiththesedatastructures，wheneverourprogrammeetsaword，theprogramwillbeabletoloeat

9、ethewordstraightlyandquicklyindietionaryAndaddingwordfrequencyinformationintothedictionaryforambiguityresolutionExtendeddictionaryincludequantifiersdictionarynamedictionarytemporarydictionaryworddictionaryDisableandSOonw