Exploring the role of R&D collaborations and non-patent IP policies in government technology transfer performance: Evidence from U.S. federal agencies (1999-2016).

Around the world, governments make substantial investments in public sector research and development (R&D) entities and activities to generate major scientific and technical advances that may catalyze long-term economic growth. Institutions ranging from the Chinese Academy of Sciences to the French National Centre for Scientific Research to the Helmholtz Association of German Research Centers conduct basic and applied R&D to create commercially valuable knowledge that supports the innovation goals of their respective government sponsors. Globally, the single largest public sector R&D sponsor is the U.S. federal government. In 2019 alone, the U.S. government allocated over $14.9 billion to federally funded research and development centers (FFRDCs), also known as national labs. However, little is known about how federal agencies' utilization of FFRDCs, their modes of R&D collaboration, and their adoption of non-patent intellectual property (IP) policies (copyright protection and materials transfer agreements) affect agency-level performance in technology transfer. In particular, the lack of standardized metrics for quantitatively evaluating government entities' effectiveness in managing innovation is a critical unresolved issue. We address this issue by conducting exploratory empirical analyses of federal agencies' innovation management activities using both supply-side (filing ratio, transfer rate, and licensing success rate) and demand-side (licensing income and portfolio exclusivity) outcome metrics. We find economically significant effects of external R&D collaborations and non-patent IP policies on the technology transfer performance of 10 major federal executive branch agencies (fiscal years 1999-2016). We discuss the scholarly, managerial, and policy implications for ongoing and future evaluations of technology transfer at federal labs. We offer new insights and guidance on how critical differences in federal agencies' interpretation and implementation of their R&D management practices in pursuit of their respective missions affect their technology transfer performance outcomes. We generalize key findings to address the broader innovation processes of public sector R&D entities worldwide.

Topic modeling revisited: New evidence on algorithm performance and quality metrics.

Topic modeling is a popular technique for exploring large document collections. It has proven useful for this task, but its application poses a number of challenges. First, the comparison of available algorithms is anything but simple, as researchers use many different datasets and criteria for their evaluation. A second challenge is the choice of a suitable metric for evaluating the calculated results. The metrics used so far provide a mixed picture, making it difficult to verify the accuracy of topic modeling outputs. Altogether, the choice of an appropriate algorithm and the evaluation of the results remain unresolved issues. Although many studies have reported promising performance by various topic models, prior research has not yet systematically investigated the validity of the outcomes in a comprehensive manner, that is, using more than a small number of the available algorithms and metrics. Consequently, our study has two main objectives. First, we compare all commonly used, non-application-specific topic modeling algorithms and assess their relative performance. The comparison is made against a known clustering and thus enables an unbiased evaluation of results. Our findings show a clear ranking of the algorithms in terms of accuracy. Secondly, we analyze the relationship between existing metrics and the known clustering, and thus objectively determine under what conditions these algorithms may be utilized effectively. This way, we enable readers to gain a deeper understanding of the performance of topic modeling techniques and the interplay of performance and evaluation metrics.