Introduction

Since its inception, the practice of surgery has involved a process of trying to understand the pathophysiology of surgical disease, introducing an intervention to alter this pathophysiology and observation of the results thereafter (1). However, with time, it has become apparent that the translation of basic physiological principles to patient care does not always lead to improved outcome thereby creating a need for more rigorous methods of evaluating the effects of available treatment options. Modern surgical practice is awash with numerous new interventions and technologies. Most of these are reported to be beneficial to the patient although it is often unclear whether the observed outcomes are due to the intervention itself or due to chance since inherent differences in patients may lead to an observation of effect even where there is none (1, 2). For this reason, it is important that surgeons are equipped with the skills necessary for the identification, evaluation, and utilization of research findings. This review introduces the notion of evidence-based surgery (EBS) with a particular focus on the utilization of systematic reviews by practicing surgeons.

​

Evidence-Based Surgery

Sackett and colleagues define evidence-based medicine (EBM) as the “...conscientious, explicit and judicious use of current best evidence in making decisions about the care of individual patients” (3). This definition encompasses five key and interrelated ideas that are central to the practice of evidence-based surgery (1). One, as surgeons, we must base our decisions on the best available evidence. Second, whatever evidence we seek must be based on the problems we encounter in our practice. Third, for the purpose of reducing bias, identification of best evidence should involve employing epidemiological and biostatistical ways of thinking. Four, evidence that has been identified and critically appraised is only useful if applied to patient care and/or decision making in healthcare. Finally, our performance in the process of generating and utilizing the best evidence must be constantly re-evaluated. Herein lies the value of surgical audit in evidence-based surgery.

​EBS affords us little room (if any) for basing decisions on expert opinion or anecdotal experience. However, the application of best evidence to patient care involves the integration of clinical acumen and incorporation of patient preferences and values in decision making (1, 3, 4). Further, we must remember that EBS is dependent on context. As an example, the advantages of laparoscopic cholecystectomy are well known and documented. However, in low resource settings, the absence of surgeons with skills in laparoscopy as well as lack of the requisite equipment means that only open cholecystectomy can be performed. Further, even if both surgeons and equipment were available, patients may be unable to afford the procedure. Nonetheless, even in such a context, there are best evidence practices regarding open cholecystectomy that should be adhered to.

​The steps involved in applying EBS have been outlined by Rosenberg and Donald (5). First, one must clearly identify and articulate a question arising from surgical practice. Second, there should be a focused search of the literature relevant to the question with subsequent critical appraisal of the identified literature. Finally, there must be an implementation of the new knowledge of the surgical practice. As such, the necessary elements of EBS are the production and dissemination of high-quality evidence on which to base surgical decisions (1).

​Systematic Reviews for Evidence-Based Surgery As has been mentioned above, the current best evidence comes from a systematic search of available research evidence. The process involves posing a focused clinical question, setting out explicit strategies for identification of relevant literature, having explicit strategies for critical appraisal of identified literature and finally summarizing and discussing the results (6). These steps comprise a systematic review. The Center for Evidence-Based Medicine (Oxford University) has devised a system for grading the levels of evidence regarding the prevention of disease, its diagnosis, and prognosis as well as effects of treatment and its harms with systematic reviews providing the highest level of evidence (7). Still, the quality of individual systematic reviews is variable and some authors point to the need for the standardization of their conduct and reporting. It is from this notion that the Preferred Reporting Items for Systematic reviews and Meta-analyses (PRISMA) statement arose (8). Further, the need has also arisen for critical appraisal of systematic reviews themselves so that they may be meaningfully used to formulate clinical guidelines and inform policy. To this end, the Risk of Bias in Systematic Reviews (ROBIS) tool was formulated (9).

​Performing a systematic review is a rigorous and time-consuming venture. In the author’s experience, the process can take anywhere from six months to three years to complete. In addition, one must be adequately trained in systematic review methodology. Clearly, majority of surgeons are unlikely to have the time to do a systematic review. Nonetheless, they should possess the skills necessary for interpreting and applying them topatientcare.Variousorganizationsareinvolvedinthe production of systematic reviews as well as in training of individuals interested in performing systematic reviews. The Cochrane Collaboration maintains a database of systematic reviews of interventions assessed through randomized controlled trials (10). More recently, they have become involved in the production of systematic reviews of diagnostic test accuracy. The collaboration is organized into various regional centers and working groups that deal with specific topics. However, the Cochrane Database of Systematic Reviews (CDSR) may have limited use for surgeons owing to the paucity of high quality surgical RCTs. The Joanna Briggs Institute (JBI) is another organization that maintains a database of systematic reviews and is involved in training reviewers (11). Unlike Cochrane, JBI has a more open view of the evidence and produces reviews even where there are no RCTs. It has collaborating centers in African countries such as South Africa, Uganda, Ethiopia, Ghana, Cameroon, and Kenya.

 

Reading, Interpreting and Utilizing a Systematic Review

The utilization of systematic reviews requires the employment of a structured process of reading and interpreting the review and subsequently applying it to patient care as described by various authors (1, 2, 6). The procedure is outlined in table 1. The first step is to assess the relevance of the clinical question at hand. This should be explicitly stated by the review authors as a precise statement of the population under study, the interventions they were exposed to and the outcomes measured. Together, these constitute the eligibility criteria for inclusion of studies into the review and they help the reader decide whether or not the question posed is sensible. Further, an explicit statement of the eligibility criteria helps to reduce bias when choosing studies for inclusion into the review. These criteria should be broad enough to allow a sufficient number of studies and patients to be studied. This, in turn, will increase the external validity of the results.

​

Click here to view table 1

​

Once the eligibility criteria for inclusion into the review have been assessed, it is necessary to determine whether the search for studies to include was exhaustive. Commonly, bibliographic databases such as Pub Med and Embase are searched but it is prudent to include more databases. Further, reference lists of retrieved articles, personal contact with known experts in the field, abstracts from recent meetings as well as doctoral theses should be used to identify more studies. Often, a meticulous review is one in which the reviewers report the actual search strategy used across databases during their search for studies. A review’s external validity may be threatened by the exclusion of unpublished or grey literature. Since studies with positive results are more likely to be published, there is a likelihood of overestimating treatment effects. This phenomenon is known as publication bias and readers may assess for it by first determining whether the search for relevant papers included unpublished data. Quantitative methods for assessing publication bias include the use of funnel plots.

​Subsequent to performing a detailed search for relevant studies, reviewers must decide on which papers are fit for inclusion, their quality and what data to abstract from them. Since these tasks are prone to error (random and systematic), it is advisable to have them performed by two independent reviewers. Chance corrected agreement between them can then be assessed using the kappa statistic so as to increase the level of confidence in their work. Assessment of the quality of included studies is a key element of the systematic review process and reviewers must report how it was done since studies with less rigorous methodology tend to overestimate the effectiveness of treatments. Standard checklists should be employed when assessing methodological quality.

​Data from methodologically sound studies included in a review must be summarized (or pooled) and results presented in a way that has clinical relevance. Where a quantitative approach is employed, this is termed meta-analysis. However, there are instances where a systematic review may be done without a meta-analysis and a narrative synthesis of results is performed instead. Further, there are also occasions where a meta-analysis alone is performed such as when researchers have access to individual patient data from a number of trials. In such cases, there is often no need to do a systematic literature search with a quality appraisal. However, the clinical question at hand must be focused and sound statistical techniques applied when pooling data.

​The main assumption when pooling data from individual studies is that pooling increases their sensitivity in detecting the effect of an intervention(6). If the magnitude of the effect is similar across the range of patients, interventions and outcomes studied then this assumption holds true. However, the broad eligibility criteria alluded to previously will often expose a number of important differences that challenge this assumption. These differences constitute what is known as heterogeneity. As such, the presentation of results in a systematic review should be done in a manner that aids the reader in assessing heterogeneity and that helps to critique the decision to pool studies in the first place. There are two methods of assessing heterogeneity. For reviews that use forest plots to display results, visual inspection to determine the extent of similarity of point estimates from individual studies may be used. Otherwise, statistical methods involving tests of homogeneity are used. These test the extent to which differences in the results of individual studies are greater than would be expected if the observed differences were due to chance. If there is significant heterogeneity, the reviewers should look for an explanation for it using sensitivity analysis.

​After reviewing the results of a systematic review, the next and most crucial step is to determine whether any reported benefits are worth the costs and potential risks. Such information is key when making recommendations to patients. To do this, the reader should first determine whether all clinically important outcomes were considered. An ideal review should focus on both the benefits and harms of an intervention and should provide a basis for quantifying any expected outcomes. These outcomes must be in turn considered in the context of patient values and preferences (3, 5, 6). For example, a systematic review on outcomes following craniotomy for low grade gliomas may report longer 5 year survival rates compared to no surgery but a higher risk of neurological complications with surgery compared to no surgery. A patient with a low grade glioma may opt for surgery at the expense of neurological complication if they place more value on survival. A similar patient may opt for no surgery because her preference is to live out her life free of any neurological deficits. In such a scenario, the evidence from the systematic review has been applied in consideration of patient values and preference.

 

Conclusion

The practice of EBS mandates that surgeons apply the best available evidence to patient care. Such evidence is best availed through systematic reviews and practicing surgeons must possess skills in reading, interpreting and utilizing systematic reviews for their day to day surgical decisions. Incorporation of patient values and preferences in such decisions is of utmost importance.

 

Conflicts of interest

No conflicts of interest

 

References

 

1. McLeod RS. Evidence-Based Surgery. In: Norton JA, Barie PS, Bollinger RR, et al., eds. Surgery Basic Science and Clinical Evidence. Second ed: Springer; 2008:21-35.

2. Haynes RB, Sackett DL, Guyatt GH, et al. Clinical Epidemiology. How to Do Clinical Practice Research. Third ed: Lippincott Williams & Wilkins; 2006.

3. Evidence Based Medicine Working Group. Evidence-Based Medicine. A New Approach to Teaching the Practice of Medicine. JAMA. 1992; 268(17):2420-5.

4. Sackett DL, Rosenberg WM. The Need for Evidence-based Medicine. J R Soc Med. 1995; 88(11):620-4.

5. Rosenberg W, Donald A. Evidence Based Medicine: An Approach to Clinical Problem-solving. BMJ. 1995; 310(6987):1122-6.

6. Bhandari M, Devereaux PJ, Montori V, et al. Users’ Guide to the Surgical Literature: How to Use a Systematic Literature Review and Meta-analysis. Can J Surg. 2004; 47(1):60-7.

7. Oxford Center for Evidence-Based Medicine-Levels of Evidence 2009 [cited 2016 24th June]. Available from: http://www.cebm.net/oxford-centre-evidence-based-medicine-levels-evidence-march-2009/.

8. Liberati A, Altman DG, Tetzlaff J, et al. The PRISMA Statement for Reporting Systematic Reviews and Meta-analyses of Studies that Evaluate Health Care Interventions: Explanation and Elaboration. J Clin Epidemiol. 2009; 62(10):e1-34.

9. Whiting P, Savovic J, Higgins JP, et al. ROBIS: A New Tool to Assess Risk of Bias in Systematic Reviews was Developed. J Clin Epidemiol. 2016; 69:225-34.

10. The Cochrane Collaboration. Cochrane Library 2016 [cited 2016 24 June]. Available from: http:// www.cochranelibrary.com.

11. Joanna Briggs Institute 2016 [cited 2016 24 June]. Available from: http://joannabriggs.org.

​