Patient information leaflets for placebo-controlled surgical trials: a review of current practice and recommendations for developers.

INTRODUCTION: Informed consent for participation in an RCT is an important ethical and legal requirement. In placebo surgical trials, further issues are raised, and to date, this has not been explored. Patient information leaflets (PILs) are a core component of the informed consent process. This study aimed to investigate the key content of PILs for recently completed placebo-controlled trials of invasive procedures, including surgery, to highlight areas of good practice, identify gaps in information provision for trials of this type and provide recommendations for practice. METHODS: PILs were sought from trials included in a recent systematic review of placebo-controlled trials of invasive procedures, including surgery. Trial characteristics and data on surgical and placebo interventions under evaluation were extracted. Directed content analysis was applied, informed by published regulatory and good practice guidance on PIL content and existing research on placebo-controlled surgical trials. Results were analysed using descriptive statistics and presented as a narrative summary. RESULTS: Of the 62 eligible RCTs, authors of 59 trials were contactable and 14 PILs were received for analysis. At least 50% of all PILs included content on general trial design. Explanations of how the placebo differs or is similar to the surgical intervention (i.e. fidelity) were reported in 6 (43%) of the included PILs. Over half (57%) of the PILs included information on the potential therapeutic benefits of the surgical intervention. One (7%) included information on potential indirect therapeutic benefits from invasive components of the placebo. Five (36%) presented the known risks of the placebo intervention, whilst 8 (57%) presented information on the known risks of the surgical intervention. A range of terms was used across the PILs to describe the placebo component, including 'control', 'mock' and 'sham'. CONCLUSION: Developers of PILs for placebo-controlled surgical trials should carefully consider the use of language (e.g. sham, mock), be explicit about how the placebo differs (or is similar) to the surgical intervention and provide balanced presentations of potential benefits and risks of the surgical intervention separately from the placebo. Further research is required to determine optimal approaches to design and deliver this information for these trials.

Adaptive clinical trials in surgery: A scoping review of methodological and reporting quality.

IMPORTANCE: Adaptive surgical trials are scarce, but adopting these methods may help elevate the quality of surgical research when large-scale RCTs are impractical. OBJECTIVE: Randomized-controlled trials (RCTs) are the gold standard for evidence-based healthcare. Despite an increase in the number of RCTs, the number of surgical trials remains unchanged. Adaptive clinical trials can streamline trial design and time to trial reporting. The advantages identified for ACTs may help to improve the quality of future surgical trials. We present a scoping review of the methodological and reporting quality of adaptive surgical trials. EVIDENCE REVIEW: We performed a search of Ovid, Web of Science, and Cochrane Collaboration for all adaptive surgical RCTs performed from database inception to October 12, 2023. We included any published trials that had at least one surgical arm. All review and abstraction were performed in duplicate. Risk of bias (RoB) was assessed using the RoB 2.0 instrument and reporting quality was evaluated using CONSORT ACE 2020. All results were analyzed using descriptive methods. FINDINGS: Of the 1338 studies identified, six trials met inclusion criteria. Trials were performed in cardiothoracic, oral, orthopedic, and urological surgery. The most common type of adaptive trial was group sequential design with pre-specified interim analyses planned for efficacy, futility, and/or sample size re-estimation. Two trials did use statistical simulations. Our risk of bias evaluation identified a high risk of bias in 50% of included trials. Reporting quality was heterogeneous regarding trial design and outcome assessment and details in relation to randomization and blinding concealment. CONCLUSION AND RELEVANCE: Surgical trialists should consider implementing adaptive components to help improve patient recruitment and reduce trial duration. Reporting of future adaptive trials must adhere to existing CONSORT ACE 2020 guidelines. Future research is needed to optimize standardization of adaptive methods across medicine and surgery.

Examining the application of the IDEAL framework in the reporting and evaluation of innovative invasive procedures: secondary qualitative analysis of a systematic review.

OBJECTIVES: The development of new surgical procedures is fundamental to advancing patient care. The Idea, Developments, Exploration, Assessment and Long-term (IDEAL) framework describes study designs for stages of innovation. It can be difficult to apply due to challenges in defining and identifying innovative procedures. This study examined how the IDEAL framework is operationalised in real-world settings; specifically, the types of innovations evaluated using the framework and how authors justify their choice of IDEAL study design. DESIGN: Secondary qualitative analysis of a systematic review. DATA SOURCES: Citation searches (Web of Science and Scopus) identified studies following the IDEAL framework and citing any of the ten key IDEAL/IDEAL_D papers. ELIGIBILITY CRITERIA: Studies of invasive procedures/devices of any design citing any of the ten key IDEAL/IDEAL_D papers. DATA EXTRACTION AND SYNTHESIS: All relevant text was extracted. Three frameworks were developed, namely: (1) type of innovation under evaluation; (2) terminology used to describe stage of innovation and (3) reported rationale for IDEAL stage. RESULTS: 48 articles were included. 19/48 described entirely new procedures, including those used for the first time in a different clinical context (n=15/48), reported as IDEAL stage 2a (n=8, 53%). Terminology describing stage of innovation was varied, inconsistent and ambiguous and was not defined. Authors justified their choice of IDEAL study design based on limitations in published evidence (n=36) and unknown feasibility and safety (n=32) outcomes. CONCLUSION: Identifying stage of innovation is crucial to inform appropriate study design and governance decisions. Authors' rationale for choice of IDEAL stage related to the existing evidence base or lack of sufficient outcome data for procedures. Stage of innovation was poorly defined with inconsistent descriptions. Further work is needed to develop methods to identify innovation to inform practical application of the IDEAL framework. Defining the concept of innovation in terms of uncertainty, risk and degree of evidence may help to inform decision-making.

REporting quality of PilOt randomised controlled trials in surgery (REPORTS): a methodological survey protocol.

INTRODUCTION: The aim of this methodological review is to evaluate the completeness of reporting of surgical pilot and feasibility randomised trials as per the Consolidated Standards of Reporting Trials (CONSORT) extension to randomised pilot and feasibility trials. Moreover, we aim to assess for the presence of spin reporting and inconsistency between abstract and main text reporting in surgical pilot and feasibility randomised trials. METHODS AND ANALYSIS: A comprehensive, electronic search strategy will be used to identify studies indexed in Medline, Embase and Cochrane Central Register of Controlled Trials (CENTRAL) databases. Studies will be included if they are pilot or feasibility randomised trials of surgical interventions. The primary outcome will be overall CONSORT statement extension to randomised pilot and feasibility trials checklist completeness. This will be defined as trials reporting each of the 40 items in the CONSORT statement extension to randomised pilot and feasibility trials checklist. Secondary outcomes will include the reporting of individual studies as per the CONSORT extension to randomised pilot and feasibility trials, the use of spin reporting strategies, trial factors associated with reporting quality and spin strategy use, and consistency between abstract and main text reporting. Poisson and logistic regressions will be performed to explore the association between trial factors and completeness of reporting as measured by the number of reported CONSORT items. ETHICS AND DISSEMINATION: This is a methodological survey that has been registered a priori on the International Prospective Register for Systematic Reviews (PROSPERO) (CRD42023475512). Local ethics approval is not required. We plan to disseminate study results through peer-reviewed publication and conference presentations.

Quality and safety interventions in surgery.

Medical errors resulting in treatment-related harm have been a challenge for many years, with particularly severe consequences in surgery. Efforts to improve safety should focus on system-based changes to response and rescue pathways, and will require further research and adequate engagement by clinical staff.

Consensus recommendations on how to assess the quality of surgical interventions.

Postoperative complications represent a major public health burden worldwide. Without standardized, clinically relevant and universally applied endpoints, the evaluation of surgical interventions remains ill-defined and inconsistent, opening the door for biased interpretations and hampering patient-centered health care delivery. We conducted a Jury-based consensus conference incorporating the perspectives of different stakeholders, who based their recommendations on the work of nine panels of experts. The recommendations cover the selection of postoperative outcomes from the perspective of patients and other stakeholders, comparison and interpretation of outcomes, consideration of cultural and demographic factors, and strategies to deal with unwarranted outcomes. With the recommendations developed exclusively by the Jury, we provide a framework for surgical outcome assessment and quality improvement after medical interventions, that integrates the main stakeholders' perspectives.

Surgical counting: design of implementation and maintenance of a standardized evidence-based procedure.

OBJECTIVES: to report the implementation and maintenance of an evidence-based Standard Operating Procedure for surgical counting performed at a teaching hospital. METHODS: a report of a project to implement evidence for surgical counting, carried out at a university hospital in December 2017, and the subsequent cycles for better performance of the implemented organizational document and maintenance of better results until March 2022. RESULTS: the report is divided into implementation project presentation and four other cycles after implementation, related to maintenance of improvements. It was possible to prepare a Standard Operating Procedure for Surgical Count, train the nursing team, carry out educational intervention for surgical teams. FINAL CONSIDERATIONS: there was an improvement in complying with the standardized procedure at the first moment and worsening in the period related to the pandemic. New efforts began again, including a self-instructive online course combined with first-time strategies.

Peer evaluation and feedback for invasive medical procedures: a systematic review.

BACKGROUND: There is significant variability in the performance and outcomes of invasive medical procedures such as percutaneous coronary intervention, endoscopy, and bronchoscopy. Peer evaluation is a common mechanism for assessment of clinician performance and care quality, and may be ideally suited for the evaluation of medical procedures. We therefore sought to perform a systematic review to identify and characterize peer evaluation tools for practicing clinicians, assess evidence supporting the validity of peer evaluation, and describe best practices of peer evaluation programs across multiple invasive medical procedures. METHODS: A systematic search of Medline and Embase (through September 7, 2021) was conducted to identify studies of peer evaluation and feedback relating to procedures in the field of internal medicine and related subspecialties. The methodological quality of the studies was assessed. Data were extracted on peer evaluation methods, feedback structures, and the validity and reproducibility of peer evaluations, including inter-observer agreement and associations with other quality measures when available. RESULTS: Of 2,135 retrieved references, 32 studies met inclusion criteria. Of these, 21 were from the field of gastroenterology, 5 from cardiology, 3 from pulmonology, and 3 from interventional radiology. Overall, 22 studies described the development or testing of peer scoring systems and 18 reported inter-observer agreement, which was good or excellent in all but 2 studies. Only 4 studies, all from gastroenterology, tested the association of scoring systems with other quality measures, and no studies tested the impact of peer evaluation on patient outcomes. Best practices included standardized scoring systems, prospective criteria for case selection, and collaborative and non-judgmental review. CONCLUSIONS: Peer evaluation of invasive medical procedures is feasible and generally demonstrates good or excellent inter-observer agreement when performed with structured tools. Our review identifies common elements of successful interventions across specialties. However, there is limited evidence that peer-evaluated performance is linked to other quality measures or that feedback to clinicians improves patient care or outcomes. Additional research is needed to develop and test peer evaluation and feedback interventions.

The Quality of Operative Performance Narrative Feedback: A Retrospective Data Comparison Between End of Rotation Evaluations and Workplace-based Assessments.

OBJECTIVE: To describe the quality of operative performance feedback using evaluation tools commonly used by general surgery residency training programs. SUMMARY OF BACKGROUND DATA: The majority of surgical training programs administer an evaluation through which faculty members may rate and comment on trainee operative performance at the end of the rotation (EOR). Many programs have also implemented the system for improving and measuring procedural learning (SIMPL), a workplace-based assessment tool with which faculty can rate and comment on a trainee's operative performance immediately after a case. It is unknown how the quality of narrative operative performance feedback delivered with these tools compares. METHODS: The authors collected EOR evaluations and SIMPL narrative comments on trainees' operative performance from 3 university-based surgery training programs during the 2016-2017 academic year. Two surgeon raters categorized comments relating to operative skills as being specific or general and as encouraging and/or corrective. Comments were then classified as effective, mediocre, ineffective, or irrelevant. The frequencies with which comments were rated as effective were compared using Chi-square analysis. RESULTS: The authors analyzed a total of 600 comments. 10.7% of EOR and 58.3% of SIMPL operative performance evaluation comments were deemed effective (P < 0.0001). CONCLUSIONS: Evaluators give significantly higher quality operative performance feedback when using workplace-based assessment tools rather than EOR evaluations.

Improving Surgical Quality for Patients With Mental Illnesses: A Narrative Review.

OBJECTIVE: The aim of this study was to identify disparities in care for surgical patients with preexisting mental health diagnoses. SUMMARY BACKGROUND DATA: Mental illness affects approximately 6.7 million Canadians. For them, stigma, comorbid disorders, and sequelae of psychiatric diagnoses can be barriers to equitable health care. The goal of this review is to define inequities in surgical care for patients with preexisting mental illness. METHODS: We searched OVID Medline, Pubmed, EMBASE, and the Cochrane review files using a combination of search terms using a PICO (population, intervention, comparison, outcome) model focusing on surgical care for patients with mental illness. RESULTS: The literature on mental illness in surgical patients focused primarily on preoperative and postoperative disparities in surgical care between patients with and without a diagnosis of mental illness. Preoperatively, patients were 7.5% to 40% less likely to be deemed surgical candidates, were less likely to receive testing, and were more likely to present at later stages of their disease or have delayed surgical care. Similar themes arose in the postoperative period: patients with mental illness were more likely to require ICU admission, were up to 3 times more likely to have a prolonged length of hospital stay, had a 14% to 270% increased likelihood of having postoperative complications, and had significantly higher health care costs. CONCLUSIONS: Surgical patients with preexisting psychiatric diagnoses have a propensity for worse perioperative outcomes compared to patients without reported mental illness. Taking a thorough psychiatric history can potentially help surgical teams address disparities in access to care as well as anticipate and prevent adverse outcomes.

Validity Evidence for Procedure-specific Competence Assessment Tools in General Surgery: A Scoping Review.

OBJECTIVE: This study aims to provide a focused and detailed assessment of the validity evidence supporting procedure-specific operative assessment tools in general surgery. SUMMARY OF BACKGROUND DATA: Competency-based assessment tools should be supported by robust validity evidence to be used reliably for evaluation of operative skills. The contemporary framework of validity relies on five sources of evidence: content, response process, internal structure, relation to other variables, and consequences. METHODS: A systematic search of 8 databases was conducted for studies containing procedure-specific operative assessment tools in general surgery. The validity evidence supporting each tool was assessed and scored in alignment with the contemporary framework of validity. Methodological rigour of studies was assessed with the Medical Education Research Study Quality Instrument. The educational utility of each tool was assessed with the Accreditation Council for Graduate Medical Education framework. RESULTS: There were 28 studies meeting inclusion criteria and 23 unique tools were assessed. Scores for validity evidence varied widely between tools, ranging from 3 - 14 (maximum 15). Medical Education Research Study Quality Instrument scores assessing the quality of study methodology were also variable (8.5-15.5, maximum 16.5). Direct reporting of educational utility criteria was limited. CONCLUSIONS: This study has identified a small group of procedure-specific operative assessment tools in general surgery. Many of these tools have limited validity evidence and have not been studied sufficiently to be used reliably in high-stakes summative assessments. As general surgery transitions to competency-based training, a more robust library of operative assessment tools will be required to support resident education and evaluation.

A Review of PROM Implementation in Surgical Practice.

OBJECTIVE: To synthesize the current state of PROM implementation and collection in routine surgical practice through a review of the literature. SUMMARY OF BACKGROUND DATA: Patient-reported outcomes (PROs) are increasingly relevant in the delivery of high quality, individualized patient care. For surgeons, PROMs can provide valuable insight into changes in patient quality of life before and after surgical interventions. Despite consensus within the surgical community regarding the promise of PROMs, little is known about their real-world implementation. METHODS: The literature search was conducted in MEDLINE and Embase for studies published after 2012. We conducted a scoping review to synthesize the current state of implementation of PROs across all sizes and types of surgical practices. Studies were included if they met the following inclusion criteria: (1) patients ≥18 years 2) routine surgical practice, (3) use of a validated PRO instrument in the peri-operative period to report on general or disease-specific health-related quality of life, (4) primary or secondary outcome was implementation. Two independent reviewers screened 1524 titles and abstracts. FINDINGS: 16 studies were identified that reported on the implementation of PROMs for surgical patients. Sample size ranged from 41 patients in a single-center pilot study to 1324 patients in a study across 17 institutions. PROs were collected pre-operatively in 3 studies, post-operatively in 10, and at unspecified times in 4. The most commonly reported implementation outcomes were fidelity (12) and feasibility (11). Less than half of studies analyzed nonrespondents. All studies concluded that collection of PROMs was successful based on outcomes measured. CONCLUSIONS: The identified studies suggest that implementation metrics including minimum standards of collection pre- and postintervention, reporting for response rates in the context of patient eligibility and analysis of respondents and nonrespondents, in addition to transparency regarding the resources utilized and cost, can facilitate adoption of PROMs in clinical care and accountability for surgical outcomes.

For the People and the Profession.

In 1982 Dean Warren delivered the presidential address "Not for the Profession… For the People" in which he identified substandard surgical residency programs graduating residents who were unable to pass American Board of Surgery exams. Drs. Warren and Shires as members of the independent ACGME began to close the substandard programs in order to improve surgical care for average Americans i.e. "for the people". By 2003 these changes dramatically reduced the failure rate for the ABS exams and trained good surgeons who could operate independently however the residents were on duty for every other or every third night. In 2003 the ACGME mandated duty hour restrictions in order improve resident wellness and improve the training environment for the profession. However, work hour restrictions reduced the time surgical residents spent in the hospital environment primarily when residents had more autonomy and had exposure to emergency cases which degraded readiness for independent practice. Surgical educators in the 2 decades after the work hour restrictions have improved techniques of training so graduates could not only pass the board exams but also be prepared for independent practice. Surgical residency training has improved by both the changes implemented by the independent ACGME in 1981 and by the work hour restrictions mandated in 2003. Five recommendations are made to ensure that Dr Warren's culture of excellence in surgical training continues in an environment that enhances wellbeing of the trainee i.e. "For the People and the Profession".