Conceptual Information Retrieval: A Case Study in Adaptive Partial Parsing

The infonnation revolution is upon us. Whereas the industrial revolution heralded the systematic augmentation of human physical limitations by har­ nessing external energy sources, the infonnation revolution strives to augment human memory and mental processing limitations by harnessing external computational resources. Computers can accumulate. transmit and output much more infonnation and in a more timely fashion than more con­ ventional printed or spoken media. Of greater interest, however, is the computer's ability to process, classify and retrieve infonnation selectively in response to the needs of each human user. One cannot drink from the fire hydrant of infonnation without being immediately flooded with irrelevant text. Recent technological advances such as optical character readers only exacerbate the problem by increasing the volume of electronic text. Just as steam and internal combustion engines brought powerful energy sources under control to yield useful work in the industrial revolution, so must we build computational engines that control and apply the vast infonnation sources that they may yield useful knowledge. Information science is the study of systematic means to control, classify, process and retrieve vast amounts of infonnation in electronic fonn. In par­ ticular, several methodologies have been developed to classify texts manually by annies of human indexers, as illustrated quite clearly at the National Library ofMedicine, and many computational techniques have been developed to search textual data bases automatically, such as full-text keyword searches. In general.