NCI Behavioral Research Measures Project

1. NCI Behavioral Research Measures Project Tom Milke Senior Systems Analyst Westat Seattle Oct 25, 2007

2. Foundations for Enablement of Cyberinfrastructure* Integrative Applications Structured Data Systems, Interoperability, Common Messaging Standards Community Agreement on Vocabulary, Structure of Measures *Courtesy of Christopher Greer, NSF

3. State of Behavioral Science* Behavioral Research Integrative Applications Structured Data Systems, Interoperability, Common Messaging Standards Community Agreement on Vocabulary, Structure of Measures *Courtesy of Christopher Greer, NSF

4. Current Landscape of Measures Public Health Surveillance Local Interventions National Surveillance Data (e.g., NHIS, BRFSS, NHANEs) Routine behavioral surveys (e.g., HINTS) Field based data University Laboratories Medical Research Settings Individually published papers Patient Charts Locally maintained data sets Clinical trials data

5. Measurement Concepts in Behavioral Medicine Measure Construct Alternative Measures with Psychometric Equivalencies* Nicotine Dependence *i.e., measures can be used interchangably to assess the construct.  Usage would vary depending on population, modality, and purpose.

6. BRP Proposal: Create a Database of GRID Compatible Measures Health Behavior Measures Measure Construct http://dccps.cancer.gov/brp/constructs/index.html

7. Purpose of the Measures Database: Create an online resource of commonly used behavioral measures in cancer control Facilitate access by applicants, grantees, and other scientists. Help the Behavioral Research Program (BRP) realize its mission of encouraging rigorous, replicable research applied to behavioral issues in cancer control.

8. Phase I Environmental Scan • A static resource was already available through the BRP web site. • Westat conducted a pilot project to do an environmental scan of comparable resources currently available.

9. Results of Environmental Scan • Identified illustrative web sites that routinely make measures available to their respective communities • Profiled sites on set of criteria • Identified current practices • Made recommendations for how to structure the BRP measures site • Developed comprehensive list of attributes about measures to offer to researchers for identifying measures of interest • Wireframes to show structure and elements for UI

10. Moving Forward • Next step would be to develop a small, testable prototype to illustrate feasibility.  • Ideally, the prototype should follow the principles associated with caBIG tool development.  • The blueprint should be articulated using Unified Modeling Language (UML); • The data should be discoverable on “The Grid”. • The semantic structure of the elements should correspond to NCI’s Enterprise Wide Vocabulary as instantiated in the caDSR.

11. Moving Forward • The measures study team recognized that many population science variables do not currently exist in the caDSR.  • One of the outcomes of the project should be to gain a better understanding of the process needed to register Population Science data elements within the global caDSR or, alternatively, in a DCCPS-managed subset of the caDSR data space.

12. Moving Forward – Questions/considerations • What would it take to incentivize usage and to construct an online community resource?   • Should the process be top-down, a resource-intensive activity that would make BRP a Clearing House for vetted measures; or should it be “bottom-up,” using social technologies to facilitate community data sharing?   • Preference for creating a self-perpetuating resource that is driven and updated by the extramural community.   • Bob Croyle pointed to the success of online knowledge sharing tools such as Wikipedia and PubCan as models.

13. Use Case Health Behavior Measures • Grantee required to use CDE’s • Browses measures based on construct, population, mode, psychometric qualities • Selects measure for use in grant submission, research • Provisions for adding new measures • Community vetting (e.g., usage statistics)

14. Use Case Diagram

15. Domain Model

16. The Measures Class and Its Attributes

17. Ancient Chinese Classification of Animals • The Analytical Language of John Wilkins," by Borges describes the Celestial Emporium of Benevolent Knowledge, in which it is written that animals are divided into: • those that belong to the Emperor, • embalmed ones, • those that are trained, • suckling pigs, • mermaids, • fabulous ones, • stray dogs, • those included in the present classification, • those that tremble as if they were mad, • innumerable ones, • those drawn with a very fine camelhair brush, • others, • those that have just broken a flower vase, • those that from a long way off look like flies.

18. We want to store and search for constructs: Challenge at the vocabulary level

19. Functions/Features Under Consideration • Search Capability (simple, advanced, index) • Resources • Measure comparison tool • Helps (how to evaluate measures, ways to frame searches) • Related links • FAQs • Community/Sharing • Measure submission process • Reviews (read and write) • Notifications • “Most viewed…” measures/constructs • Recommendations • Personalized workspace • Saved search results, Personal reviews, • Personalized notifications

20. Search Measure Author iTunes Model Search Constructs & Measures Researcher Resources - Search Constructs & Measures Resources/ Help and Links Your search yielded the following results: All measuresShow only downloadable measures Share/ Community Compare Measures ∙ My Research Space Target Population Mode of Admin. Number of Items Number of Recommendations Quick Links WISDM-68 xxx xxxxxx xxx xxxxx xxxxx Search Constructs & Measures NDSS xxx xxxxxx xxx xxxxxx xxxxx New! Help CDS - 5 xxx xxxxxx xxx xxxxxx xxxxx Previous 1 23….Next Contact Us Evaluate Measures Share/Community

21. Search Measure Author Compare Measures Researcher Resources Search Constructs & Measures - - Search Constructs & Measures Compare Measures: Select the measures you would like to compare. Share/ Community Resources/ Help and Links Compare Target Population Mode of Admin. Number of Items Number of Recommendations My Research Space WISDM-68 xxxxxx xxx xxx xxxxx xxxxx x Quick Links NDSS x xxx xxxxxx xxx xxxxx xxxxx Search Constructs & Measures CDS - 5 xxx xxxxxx xxx xxxxx xxxxx New! Previous 1 23….Next Help Contact Us Share/Community Evaluate Measures

22. Researcher Resources Select Attributes to Compare Search Constructs & Measures - - Search Constructs & Measures Compare Measures: Add attributes of the measure that you would like to compare: Share/ Community Resources/ Help and Links All Attributes Attributes to Compare Author # of Items Mode of Admin. Target Population Source References Cost Data sharing capability Author Contact Info. # of Recommendations My Research Space << Quick Links >> Search Constructs & Measures Continue New! Help Contact Us Share/Community Evaluate Measures

23. Researcher Resources Compare Measures Search Constructs & Measures - - Search Constructs & Measures Comparison of Measures: Search Share/ Community Resources/ Help and Links Attributes NDSS WISDM-64 CDS-5 My Research Space Cost xxx xxxxxx xxx Data Sharing Capability xxxxxx Quick Links xxx xxx Author Contact Information Search Constructs & Measures xxx xxxxxx xxx Number of Recommendations xxx xxxxxx xxx New! Help Contact Us Share/Community Evaluate Measures

24. TAB Model NCI Header Search My Research All Researcher Researcher Help and Links Constructs and Space Resources Community Measures Dropdown ... [ free text ] Go Search for Search Constructs and Measures : Resources , Helps and Links : simple search , advanced search , NCI links to other sites , tips on how to Index, guide, compare Measures RSS feeds,notifications of new , Left evaluate measures , help files , FAQ Nav . content, tag clouds, guide to search Researcher Community : My Research Space : " sharing , " most viewed , " people who , my recent searches , my reviews , profile liked X liked Y ..." , reviews , site updates , " people like ME liked Y ...“, site updates RSS , tag clouds Dropdown list is " all measures and constructs , " " measures , " and " constructs " -- default is to search all measures and constructs .

25. Discussion

26. Discussion Points • What are the sociotechnical solutions to the “common data element” problem? E.g., • Incentivized data sharing through initiatives? • Specialized caDSR? • Vetting bodies for standards in population science? • Where are targets of opportunity? • Federal surveillance? • Clinical protocols? • Behavioral measures in electronic health records?

27. Discussion Points • What tools are needed to enable transformations in: discovery, visualization, decision support, fusion, and policy planning?

28. Discussion Points • Can Pop Sci SIG create a “blueprint” to guide growth strategically? • Overall vision? • Needed standards investments? • Needed tool investments? • Ethical, social, and legal concerns? • GRID level use case scenarios

29. FINIS • Rick Moser moserr@mail.nih.gov • Brad Hesse hesseb@mail.nih.gov • Jennifer Crafts jennifercrafts@westat.com • Tom Milke tommilke@westat.com