教務(wù)數(shù)據(jù)倉(cāng)庫(kù)中數(shù)據(jù)清理方法的研究.pdf

1、華中科技大學(xué)碩士學(xué)位論文教務(wù)數(shù)據(jù)倉(cāng)庫(kù)中數(shù)據(jù)清理方法的研究姓名：曹薇申請(qǐng)學(xué)位級(jí)別：碩士專業(yè)：計(jì)算機(jī)軟件與理論指導(dǎo)教師：劉云生20050401華中科技大學(xué)碩士學(xué)位論文AbstractWiththedevelopmentofinformationtechnology，manyorganizationstodayhaveaccumulatedagreatdealofdataInordertoutilizethemfullytosupportma

2、kingdecisions，theyneedtoconstructdatawarehouseHowever，thereal—worlddataareoftendirtyduetovariousdataqualityproblems，theneedfordatacleaningincreasessignificantlyduetOthe”garbagein，garbageout”principleThestudyisfocusedonth

3、efieldcleaningandtherecordcleaningafterelaboratingtheprincipleofthedatacleaningInthefieldcleaning，inordertodetectdataoutliers，weadoptspecialmethodforeverykindofdata，themainideaistOrestraindatathathavedifferenttypesbymeta

4、datatablesanddetectdataoutliersbycombiningmetadatatableandapplicationprogram；Intherepairofmissingvalues，westudyhowtepairmissingvaluesbyBayesianClassifiedmethodIntherecordcleaning，weimprovethealgorithmtodetectoutliersbase

5、donsimilarcoefficientsum，theimprovedalgorithmcannotonlydetecttheoutliersduetoabnormalmultiple，butalsotheoutliersduetOabnormalfieldvalue，thedetectionprecisionisenhancedfurthermoreBesides，iftherearealotofdataandspellingmis

6、takes，andthewordsaffecteachother，theformermethodsfordetectingapproximatelyduplicaterecordsarenotwellqualified，especiallywhentherearemanyinsertion，deletionmistakesBycombiningandimprovingtheformermethods，weproposeanefficie

7、ntapproachfordetectingapproximatelyduplicaterecords，itcandetectthecommonspellingmistakessuchasinsertion，deletion，substitution，transpositionandwordswitching，comparedwiththeformermethods，theadvantageoftheapproachisitssmall