This project will examine whether federal agency rulemaking can be improved with two innovations: a) multi-level deliberation (MLD), in which people discuss rulemakings in small groups that then select members to represent the group in a higher-level group and b) the combination of language technologies into an artificial discussion facilitation agent (DiFA).
The social science herein breaks new ground in the nascent fields of e-rulemaking and democratic deliberation research. The project will advance research on measuring the quality of deliberation and the effects of deliberation and DiFA on individuals and communities.
Research will involve four rulemaking experiments. The first three are subsets of the final one. The final 3X2 experiment crosses MLD, non-MLD deliberation, and non-deliberation with the presence or absence of DiFA. The success of the various conditions of these experiments will be measured using a multi-trait, multi-method approach that will include survey and focus group measures of agency official and participant perceptions and evaluations, a content analysis measure of the cognitive sophistication of rulemaking comments, both human-coded and automated content analyses of the quality of deliberation, measures of the impact of the deliberations on participants (knowledge, trust, citizenship), DiFA usage patterns, and continued participation in our user community.
The project poses computer science challenges of combining several Natural Language Processing technologies (primarily Interactive QA, Dialogue Analysis, and Summarization) into a viable facilitation agent and in applying these technologies in an eclectic, multi-user discussion environment. We expect advances to be made within each component technology. For example, we hope to increase the utility of Dialogue Act tagging across applications and domains by using a set of general discussion tags for tracking and summarizing threads of discussion by combining dialogue structure and content analysis. We will also investigate how general our Question Answer approaches are.