Computer algorithm for automated work group classification from free text: the DREAM technique.

OBJECTIVE: This study developed and tested a computer method to automatically assign subjects to aggregate work groups based on their free text work descriptions. METHODS: The Double Root Extended Automated Matcher (DREAM) algorithm classifies individuals based on pairs of subjects' free text word roots in common with those of standard classification systems and several explicitly defined linkages between term roots and aggregates. RESULTS: DREAM effectively analyzed free text from 5887 participants in a multisite chronic obstructive pulmonary disease prevention study (Lung Health Study). For a test set of 533 cases, DREAMs classifications compared favorably with those of a four-human panel. The humans rated the accuracy of DREAM as good or better in 80% of the test cases. CONCLUSIONS: Automated text interpretation is a promising tool for analyzing large data sets for applications in data mining, research, and surveillance. Work descriptive information is most useful when it can link an individual to aggregate entities that have occupational health relevance. Determining the appropriate group requires considerable expertise. This article describes a new method for making such assignments using a computer algorithm to reduce dependence on the limited number of occupational health experts. In addition, computer algorithms foster consistency of assignments.

Working words: real-life lexicon of North American workers.

OBJECTIVE: This study describes a new computer methodology for analyzing workers' free text work descriptions. METHODS: Computerized lexical analysis was applied to work descriptions of participants in the Lung Health Study, a smoking-cessation study in persons with early chronic obstructive pulmonary disease. Text was parsed and analyzed as single term roots and pairs of roots commonly occurring together. RESULTS: The frequencies of terms reflect the work of a population; our subjects' most frequently used terms included "sale, office, service, business, engine[er], secretary, construct, driv[e], comput[e], teach, truck." Standard classification schemes (NAICS and SOC) and textbooks use terms inconsistent with those of actual workers. Many common empirical terms imply both industry and job information content, although traditional coding schemes separate industry and job title. CONCLUSIONS: Formal analyses of language may facilitate communication, identify translation priorities, and allow automated work coding.