Electronic patient-reported outcome measures using mobile health technology in rheumatology: A scoping review.

OBJECTIVE: This scoping review aims to characterize the current literature on electronic patient-reported outcome measures (ePROMs) in rheumatology and assess the feasibility and utility of ePROMs and mobile health technology in the management of rheumatic disease. INTRODUCTION: Patient-reported outcome measures (PROMs) are commonly used in rheumatology as they are important markers of disease activity and overall function, encourage shared decision-making, and are associated with high rates of patient satisfaction. With the widespread use of mobile devices, there is increasing interest in the use of mobile health technology to collect electronic PROMs (ePROM). INCLUSION CRITERIA: All primary studies that involve the collection of ePROMs using mobile devices by individuals with a rheumatic disease were included. Articles were excluded if ePROMs were measured during clinic appointments. METHODS: A scoping review was performed using Medline, Embase, PsycINFO, and CINAHL with index terms and key words related to "patient-reported outcome measures", "rheumatic diseases", and "mobile health technology". RESULTS: A total of 462 records were identified after duplicates were removed. Of the 70 studies selected for review, 43% were conference proceedings and 57% were journal articles, with the majority published in 2016 or later. Inflammatory arthritis was the most common rheumatic disease studied. Generic ePROMs were used over three times more often than disease-specific ePROMs. A total of 39 (56%) studies directly evaluated the feasibility of ePROMs in clinical practice, 19 (27%) were clinical trials that used ePROMs as study endpoints, 9 (13%) were focus groups or surveys on smartphone application development, and 3 (4%) did not fit into one defined category. CONCLUSION: The use of ePROMs in rheumatology is a growing area of research and shows significant utility in clinical practice, particularly in inflammatory arthritis. Further research is needed to better characterize the feasibility of ePROMs in rheumatology and their impact on patient outcomes.

RAMESES publication standards: meta-narrative reviews.

BACKGROUND: Meta-narrative review is one of an emerging menu of new approaches to qualitative and mixed-method systematic review. A meta-narrative review seeks to illuminate a heterogeneous topic area by highlighting the contrasting and complementary ways in which researchers have studied the same or a similar topic. No previous publication standards exist for the reporting of meta-narrative reviews. This publication standard was developed as part of the RAMESES (Realist And MEta-narrative Evidence Syntheses: Evolving Standards) project. The project's aim is to produce preliminary publication standards for meta-narrative reviews. METHODS: We (a) collated and summarized existing literature on the principles of good practice in meta-narrative reviews; (b) considered the extent to which these principles had been followed by published reviews, thereby identifying how rigor may be lost and how existing methods could be improved; (c) used a three-round online Delphi method with an interdisciplinary panel of national and international experts in evidence synthesis, meta-narrative reviews, policy and/or publishing to produce and iteratively refine a draft set of methodological steps and publication standards; (d) provided real-time support to ongoing meta-narrative reviews and the open-access RAMESES online discussion list so as to capture problems and questions as they arose; and (e) synthesized expert input, evidence review and real-time problem analysis into a definitive set of standards. RESULTS: We identified nine published meta-narrative reviews, provided real-time support to four ongoing reviews and captured questions raised in the RAMESES discussion list. Through analysis and discussion within the project team, we summarized the published literature, and common questions and challenges into briefing materials for the Delphi panel, comprising 33 members. Within three rounds this panel had reached consensus on 20 key publication standards, with an overall response rate of 90%. CONCLUSION: This project used multiple sources to draw together evidence and expertise in meta-narrative reviews. For each item we have included an explanation for why it is important and guidance on how it might be reported. Meta-narrative review is a relatively new method for evidence synthesis and as experience and methodological developments occur, we anticipate that these standards will evolve to reflect further theoretical and methodological developments. We hope that these standards will act as a resource that will contribute to improving the reporting of meta-narrative reviews.

RAMESES publication standards: realist syntheses.

BACKGROUND: There is growing interest in realist synthesis as an alternative systematic review method. This approach offers the potential to expand the knowledge base in policy-relevant areas - for example, by explaining the success, failure or mixed fortunes of complex interventions. No previous publication standards exist for reporting realist syntheses. This standard was developed as part of the RAMESES (Realist And MEta-narrative Evidence Syntheses: Evolving Standards) project. The project's aim is to produce preliminary publication standards for realist systematic reviews. METHODS: We (a) collated and summarized existing literature on the principles of good practice in realist syntheses; (b) considered the extent to which these principles had been followed by published syntheses, thereby identifying how rigor may be lost and how existing methods could be improved; (c) used a three-round online Delphi method with an interdisciplinary panel of national and international experts in evidence synthesis, realist research, policy and/or publishing to produce and iteratively refine a draft set of methodological steps and publication standards; (d) provided real-time support to ongoing realist syntheses and the open-access RAMESES online discussion list so as to capture problems and questions as they arose; and (e) synthesized expert input, evidence syntheses and real-time problem analysis into a definitive set of standards. RESULTS: We identified 35 published realist syntheses, provided real-time support to 9 on-going syntheses and captured questions raised in the RAMESES discussion list. Through analysis and discussion within the project team, we summarized the published literature and common questions and challenges into briefing materials for the Delphi panel, comprising 37 members. Within three rounds this panel had reached consensus on 19 key publication standards, with an overall response rate of 91%. CONCLUSION: This project used multiple sources to develop and draw together evidence and expertise in realist synthesis. For each item we have included an explanation for why it is important and guidance on how it might be reported. Realist synthesis is a relatively new method for evidence synthesis and as experience and methodological developments occur, we anticipate that these standards will evolve to reflect further methodological developments. We hope that these standards will act as a resource that will contribute to improving the reporting of realist syntheses.

RAMESES publication standards: meta-narrative reviews.

BACKGROUND: Meta-narrative review is one of an emerging menu of new approaches to qualitative and mixed-method systematic review. A meta-narrative review seeks to illuminate a heterogeneous topic area by highlighting the contrasting and complementary ways researchers have studied the same or a similar topic. No previous publication standards exist for the reporting of meta-narrative reviews. This publication standard was developed as part of the RAMESES (Realist And MEta-narrative Evidence Syntheses: Evolving Standards) project. The project's aim is to produce preliminary publication standards for meta-narrative reviews. DESIGN: A mixed method study synthesising data between 2011 to 2012 from a literature review, online Delphi panel and feedback from training, workshops and email list. METHODS: We: (a) collated and summarized existing literature on the principles of good practice in meta-narrative reviews; (b) considered the extent to which these principles had been followed by published reviews, thereby identifying how rigor may be lost and how existing methods could be improved; (c) used a three-round online Delphi method with an interdisciplinary panel of national and international experts in evidence synthesis, meta-narrative reviews, policy, and/or publishing to produce and iteratively refine a draft set of methodological steps, and publication standards; (d) provided real-time support to ongoing meta-narrative reviews and the open-access RAMESES online discussion list so as to capture problems and questions as they arose; and (e) synthesized expert input, evidence review, and real-time problem analysis into a definitive set of standards. RESULTS: We identified nine published meta-narrative reviews, provided real-time support to four ongoing reviews, and captured questions raised in the RAMESES discussion list. Through analysis and discussion within the project team, we summarized the published literature, and common questions and challenges into briefing materials for the Delphi panel, comprising 33 members. Within three rounds this panel had reached consensus on 20 key publication standards, with an overall response rate of 90%. CONCLUSIONS: This project used multiple sources to draw together evidence and expertise in meta-narrative reviews. For each item we have included an explanation for why it is important and guidance on how it might be reported. Meta-narrative review is a relatively new method for evidence synthesis and as experience and methodological developments occur, we anticipate that these standards will evolve to reflect further theoretical and methodological developments. We hope that these standards will act as a resource that will contribute to improving the reporting of meta-narrative reviews.

RAMESES publication standards: realist syntheses.

BACKGROUND: There is growing interest in realist synthesis as an alternative systematic review method. This approach offers the potential to expand the knowledge base in policy-relevant areas - for example, by explaining the success, failure or mixed fortunes of complex interventions. No previous publication standards exist for reporting realist syntheses. This standard was developed as part of the RAMESES (Realist And MEta-narrative Evidence Syntheses: Evolving Standards) project. The project's aim is to produce preliminary publication standards for realist systematic reviews. DESIGN: A mixed method study synthesising data between 2011-2012 from a literature review, online Delphi panel and feedback from training, workshops and email list. METHODS: We: (a) collated and summarized existing literature on the principles of good practice in realist syntheses; (b) considered the extent to which these principles had been followed by published syntheses, thereby identifying how rigour may be lost and how existing methods could be improved; (c) used a three-round online Delphi method with an interdisciplinary panel of national and international experts in evidence synthesis, realist research, policy and/or publishing to produce and iteratively refine a draft set of methodological steps and publication standards; (d) provided real-time support to ongoing realist syntheses and the open-access RAMESES online discussion list to capture problems and questions as they arose; and (e) synthesized expert input, evidence syntheses and real-time problem analysis into a definitive set of standards. RESULTS: We identified 35 published realist syntheses, provided real-time support to 9 ongoing syntheses and captured questions raised in the RAMESES discussion list. Through analysis and discussion within the project team, we summarized the published literature and common questions and challenges into briefing materials for the Delphi panel, comprising 37 members. Within 3 rounds this panel had reached consensus on 19 key publication standards, with an overall response rate of 91%. CONCLUSIONS: This project used multiple sources to develop and draw together evidence and expertise in realist synthesis. For each item we have included an explanation for why it is important and guidance on how it might be reported. Realist synthesis is a relatively new method for evidence synthesis and as experience and methodological developments occur, we anticipate that these standards will evolve to reflect further methodological developments. We hope that these standards will act as a resource that will contribute to improving the reporting of realist syntheses.

Rehabilitation for long-term care residents following hip fracture: a survey of reported rehabilitation practices and perceived barriers to delivery of care.

BACKGROUND AND PURPOSE: In Canada, residents of long-term care (LTC) facilities are frequently transferred back to their residential facilities to continue recovery 1 week following hip fractures. Limited evidence is available regarding rehabilitation services in LTC facilities following hip fracture or rehabilitation practices after hip fracture for persons with dementia. We previously performed a systematic review of the impact of rehabilitation after hip fractures on persons with dementia. The purpose of this project was to augment our systematic review findings and determine current reported rehabilitation practices of rehabilitation professionals working in LTC facilities with hip-fractured patients with dementia and examine perceived barriers to the delivery of rehabilitation. METHODS: We performed a survey study of rehabilitation professionals working in LTC facilities in a Canadian province, using a modified Delphi Consensus approach and a Web-based survey format. The survey was constructed on the basis of our systematic literature review and through discussions with clinical experts. A 2-round modified Delphi consensus approach was used to attain consensus among respondents. Examination of agreement and disagreement among respondents in the areas of (1) current practice and (2) perceived barriers to care was undertaken. RESULTS: A total of 42 respondents completed round 1 and 27 respondents completed round 2. Respondents were primarily female and working as physical therapists or occupational therapists. Most worked at facilities that employed at least 1 physical therapist and 1 occupational therapist. Treatment strategies for patients with hip fracture focused on return to independent ambulation and transfers. Although cognition was seen as a barrier to treatment, respondents implemented strategies to mitigate this barrier. Inadequate time and staffing were also seen as barriers to effective treatment. DISCUSSION: Rehabilitation professionals working in LTC facilities reported that the residents with dementia can and should participate in rehabilitation following hip fractures; however, they identified time and staffing constraints in addition to cognitive impairment as barriers. CONCLUSIONS: Current research in LTC facilities is very sparse and little is known about the rehabilitation care provided in these facilities. Further research is needed to determine the most effective rehabilitation interventions following hip fractures for residents of LTC facilities with dementia.

Health information support provided by professional associations in Canada.

BACKGROUND: Healthcare practitioners in Alberta and across Canada have varying levels of access to information resources depending on their institutional and professional affiliations, yet access to current health information is critical for all. OBJECTIVES: To determine what information resources and services are provided by Albertan and Canadian professional health associations to their members. METHODS: Representatives of professional colleges and associations were interviewed regarding information resources and services offered to members and perceptions of their members' information needs. RESULTS: National-level associations are more likely to provide resources than provincial ones. There is a clear distinction between colleges and associations in terms of information offered: colleges provide regulatory information, while associations are responsible for provision of clinical information resources. Only half of the associations interviewed provide members with access to licensed databases, with cost being a major barrier. CONCLUSIONS: There is considerable variation in the number of electronic resources and the levels of information support provided by professional health associations in Alberta and Canada. Access and usage vary among the health professions. National licensing of resources or creation of a portal linking to freely available alternatives are potential options for increasing access and awareness.

Rehabilitation in patients with dementia following hip fracture: a systematic review.

PURPOSE: Best rehabilitation practices after hip fracture for people with dementia have not been established. A systematic review was conducted to determine current evidence for rehabilitation in this population, including residents in continuing care. METHODS: Standardized review methodology was used to search eight databases for literature on hip-fracture rehabilitation for people with dementia. Eligible studies included participants with dementia who had a hip fracture; performed a rehabilitation intervention; and evaluated one or more of function, ambulation, discharge location, or falls. The Newcastle-Ottawa Scale was used to assess validity. RESULTS: A total of 13 studies were included: five randomized controlled trials (RCTs), seven prospective cohort series, and one retrospective cohort study. Average quality ratings for RCTs and cohort studies were good and fair respectively. Participants with mild to moderate dementia receiving rehabilitation showed similar relative gains in function to those without dementia. Only one study examined the effect of rehabilitation among residents in continuing care. CONCLUSIONS: People with mild or moderate dementia may show improved function and ambulation and decreased fall risk after rehabilitation post hip fracture, similar to gains achieved by those without dementia. More research is required to ascertain the effect of rehabilitation in people with moderate to severe dementia, including those residing in continuing-care settings.Purpose: Best rehabilitation practices after hip fracture for people with dementia have not been established. A systematic review was conducted to determine current evidence for rehabilitation in this population, including residents in continuing care. Methods: Standardized review methodology was used to search eight databases for literature on hip-fracture rehabilitation for people with dementia. Eligible studies included participants with dementia who had a hip fracture; performed a rehabilitation intervention; and evaluated one or more of function, ambulation, discharge location, or falls. The Newcastle­Ottawa Scale was used to assess validity. Results: A total of 13 studies were included: five randomized controlled trials (RCTs), seven prospective cohort series, and one retrospective cohort study. Average quality ratings for RCTs and cohort studies were good and fair respectively. Participants with mild to moderate dementia receiving rehabilitation showed similar relative gains in function to those without dementia. Only one study examined the effect of rehabilitation among residents in continuing care. Conclusions: People with mild or moderate dementia may show improved function and ambulation and decreased fall risk after rehabilitation post hip fracture, similar to gains achieved by those without dementia. More research is required to ascertain the effect of rehabilitation in people with moderate to severe dementia, including those residing in continuing-care settings.

RÉSUMÉ Objectif : Il n'y a pas de pratiques exemplaires d'établies pour la réadaptation après une fracture de la hanche chez les personnes aux prises avec la démence. Nous avons procédé à une revue systématique en vue de recueillir les faits cliniques relatifs à la réadaptation chez ce segment de la population, y compris les personnes en soins prolongés. Méthode : Une méthodologie d'examen de documents normalisée a été utilisée pour effectuer une recherche dans huit bases de données afin de recueillir de la documentation sur la réadaptation après une fracture de la hanche chez les personnes souffrant de démence. Les études admissibles traitaient de patients avec démence qui avaient subi une fracture de la hanche; auprès de qui on avait procédé à une intervention en réadaptation; et où au moins une fonction, la marche, le site du congé ou les risques de chutes avaient été évalués. L'échelle de Newcastle­Ottawa a été utilisée aux fins d'évaluation de la validité de ces études. Résultats : Au total, 13 études ont été répertoriées; cinq essais contrôlés randomisés (ECR), sept études de cohorte prospective et une étude de cohorte rétrospective. La qualité moyenne des ECR et des études de cohortes étaient respectivement bonne à moyenne. Les participants avec démence légère à modérée qui recevaient des traitements de réadaptation ont démontré des gains relatifs de fonction similaires à ceux qui ne souffraient pas de démence. Une seule de ces études s'est penchée sur les effets de la réadaptation chez les résidents d'établissements de soins prolongés. Conclusions : Les personnes souffrant de démence légère ou modérée ont démontré une fonction et une ambulation améliorées, de même qu'une réduction des risques de chutes après des soins en réadaptation à la suite d'une fracture de la hanche; ces gains étaient similaires chez les personnes non affectées par la démence. D'autres recherches seront nécessaires pour étudier les effets de la réadaptation chez les personnes souffrant de démence modérée à grave, y compris celles qui résident dans des établissements de soins prolongés.

In search of quality evidence for lifestyle management and glycemic control in children and adolescents with type 2 diabetes: A systematic review.

BACKGROUND: Our purpose was to evaluate the impact of lifestyle behavior modification on glycemic control among children and youth with clinically defined Type 2 Diabetes (T2D). METHODS: We conducted a systematic review of studies (randomized trials, quasi-experimental studies) evaluating lifestyle (diet and/or physical activity) modification and glycemic control (HbA1c). Our data sources included bibliographic databases (EMBASE, CINAHL®, Cochrane Library, Medline®, PASCAL, PsycINFO®, and Sociological Abstracts), manual reference search, and contact with study authors. Two reviewers independently selected studies that included any intervention targeting diet and/or physical activity alone or in combination as a means to reduce HbA1c in children and youth under the age of 18 with T2D. RESULTS: Our search strategy generated 4,572 citations. The majority of citations were not relevant to the study objective. One study met inclusion criteria. In this retrospective study, morbidly obese youth with T2D were treated with a very low carbohydrate diet. This single study received a quality index score of < 11, indicating poor study quality and thus limiting confidence in the study's conclusions. CONCLUSIONS: There is no high quality evidence to suggest lifestyle modification improves either short- or long-term glycemic control in children and youth with T2D. Additional research is clearly warranted to define optimal lifestyle behaviour strategies for young people with T2D.

Effectiveness of corticosteroid treatment in acute pharyngitis: a systematic review of the literature.

OBJECTIVES: The objective was to examine the effectiveness of corticosteroid treatment for the relief of pain associated with acute pharyngitis potentially caused by group A beta-hemolytic Streptococcus (GABHS). METHODS: This was a systematic review of the literature. Data sources used were electronic databases (Cochrane Library, MEDLINE, EMBASE, Biosis Previews, Scopus, and Web of Science), controlled trial registration websites, conference proceedings, study references, experts in the field, and correspondence with authors. Selection criteria consisted of randomized controlled trials (RCTs) in which corticosteroids, alone or in combination with antibiotics, were compared to placebo or any other standard therapy for treatment of acute pharyngitis in adult patients, pediatric patients, or both. Two reviewers independently assessed for relevance, inclusion, and study quality. Weighted mean differences (WMDs) were calculated and are reported with corresponding 95% confidence intervals (CIs). RESULTS: From 272 potentially relevant citations, 10 studies met the inclusion criteria. When compared to placebo, corticosteroids reduced the time to clinically meaningful pain relief (WMD = -4.54 hours; 95% CI = -7.19 to -1.89); however, they provided only a small reduction in pain scores at 24 hours (WMD = -0.90 on a 0-10 visual analog scale; 95% CI = -1.5 to -0.3). Heterogeneity among pooled studies was identified for both outcomes (I(2) = 81 and 74%, respectively); however, the GABHS-positive subgroup receiving corticosteroid treatment did have a significant mean reduction in time to clinically meaningful pain relief of 5.22 hours (95% CI = -7.02 to -3.42; I(2) = 0%). Short-term side effect profiles between corticosteroids and placebo groups were similar. CONCLUSIONS: Corticosteroid administration for acute pharyngitis was associated with a relatively small effect in time to clinically meaningful pain relief (4.5-hour reduction) and in pain relief at 24 hours (0.9-point reduction), with significant heterogeneity in the pooled results. Decision-making should be individualized to determine the risks and benefits; however, corticosteroids should not be used as routine treatment for acute pharyngitis.

Strength of evidence for perioperative use of statins to reduce cardiovascular risk: systematic review of controlled studies.

OBJECTIVE: To determine the strength of evidence underlying recommendations for use of statins during the perioperative period to reduce the risk of cardiovascular events. DESIGN: Systematic review of studies with concurrent control groups. DATA SOURCES: Four electronic databases, the references of identified studies, international experts on perioperative medicine, and the authors of the primary studies. Review methods Two reviewers independently extracted data from studies that reported acute coronary syndromes or mortality in patients receiving or not receiving statins during the perioperative period. MAIN OUTCOME MEASURE: Random effects summary odds ratios for death or acute coronary syndrome during the perioperative period. RESULTS: 18 studies--two randomised trials (n=177), 15 cohort studies (n=799,632), and one case-control study (n=480)--assessed whether statins provide perioperative cardiovascular protection; 12 studies enrolled patients undergoing non-cardiac vascular surgery, four enrolled patients undergoing coronary bypass surgery, and two enrolled patients undergoing various surgical procedures. In the randomised trials the summary odds ratio for death or acute coronary syndrome during the perioperative period with statin use was 0.26 (95% confidence interval 0.07 to 0.99) and the summary odds ratio in the cohort studies was 0.70 (0.57 to 0.87). Although the pooled cohort data provided a statistically significant result, statins were not randomly allocated, results in retrospective studies were larger (odds ratio 0.65, 0.50 to 0.84) than those in the prospective cohorts (0.91, 0.65 to 1.27), and dose, duration, and safety of statin use was not reported. CONCLUSION: The evidence base for routine administration of statins to reduce perioperative cardiovascular risk is inadequate.

Best practices for elderly hip fracture patients. A systematic overview of the evidence.

OBJECTIVES: To determine evidence-based best practices for elderly hip fracture patients from the time of hospital admission to 6 months postfracture. DATA SOURCES: MEDLINE, Cochrane Library, CINAHL, Embase, PEDro, Ageline, NARIC, and CIRRIE databases were searched for potentially eligible articles published between 1985 and 2004. REVIEW METHODS: Two independent reviewers determined studies appropriate for inclusion using standardized selection criteria, extracted data, evaluated internal validity, and then rated studies according to levels of evidence. Only Level 1 or 2 evidence was included in our summary of clinical recommendations. RESULTS: Spinal anesthesia, pressure-relieving mattresses, perioperative antibiotics, and deep vein thromboses prophylaxes had consistent evidence of benefit. Routine preoperative traction was not associated with any benefits and should be abandoned. Types of surgical management, postoperative wound drainage, and even "multidisciplinary" care, lacked sufficient evidence to determine either benefit or harm. There was little evidence to either determine best subacute rehabilitation practices or to direct ongoing medical issues (e.g., nutrition). Studies conducted during the subacute recovery period were heterogeneous in terms of treatment settings, interventions, and outcomes studied and had no clear evidence for best treatment practices. CONCLUSIONS: The evidence for perioperative practices is relatively robust and evidence-based perioperative treatment guidelines can be easily established. Conversely, more evidence is required to better guide the care of elderly patients with hip fracture during the subacute recovery period and convalescence.

Impact of librarians in first-year medical and dental student problem-based learning (PBL) groups: a controlled study.

BACKGROUND: Librarians at the University of Alberta have been involved with teaching undergraduate medical and dental education for several years. After 1 year of increased librarian involvement at the problem-based learning (PBL), small-group level, informal feedback from faculty and students suggested that librarians' participation in PBL groups was beneficial. There was, however, no real evidence to support this claim or justify the high demand on librarians' time. OBJECTIVES: The study aimed to determine whether having a librarian present in the small-group, problem-based learning modules for first-year medical and dental students results in an improved understanding of evidence-based medicine concepts, the nature of medical literature, and information access skills. METHODS: One hundred and sixty-four first-year medical and dental students participated in the study. There were a total of 18 PBL groups, each with approximately nine students and one faculty tutor. Six librarians participated and were assigned randomly to the six intervention groups. Students were given pre- and post-tests at the outset and upon completion of the 6-week course. RESULTS: Post-test scores showed that there was a small positive librarian impact, but final exam scores showed no impact. There was also no difference in attitudes or comfort levels between students who had a librarian in their group and those who did not. CONCLUSIONS: Impact was not sufficient to warrant continued participation of librarians in PBL. In future instruction, librarians at the John W. Scott Health Sciences Library will continue to teach at the larger group level.

Assessing the methodological quality of nonrandomized intervention studies.

In many areas of health care, randomized controlled trials (the best evidence regarding the effectiveness of health care interventions) are lacking and decision-makers have to rely on evidence from nonrandomized studies (NRS). We conducted a Medline search to identify English-language articles describing instruments for assessing the quality of NRS of health care interventions. These instruments varied greatly in scope, in the number and types of items and in developmental rigor. Items commonly included were those related to specification of study questions, allocation method, comparability of groups, and blinding of outcome assessment. We do not support the development of a generic scale to evaluate the methodological quality of nonrandomized intervention studies. Instead, further study should be directed to investigate the degree to which, and the circumstances under which, different methodological characteristics are associated with bias. This information will assist researchers in identifying a priori which methodological characteristics need careful evaluation in particular studies.