Study with the several resources on Docsity
Earn points by helping other students or get them with a premium plan
Prepare for your exams
Study with the several resources on Docsity
Earn points to download
Earn points by helping other students or get them with a premium plan
Community
Ask the community for help and clear up your study doubts
Discover the best universities in your country according to Docsity users
Free resources
Download our free guides on studying techniques, anxiety management strategies, and thesis advice from Docsity tutors
The data extraction and analysis plan for a systematic review of systematic reviews evaluating the clinical effectiveness of Western herbal medicines. It includes the characteristics of included systematic reviews that will be extracted, such as review objectives, study design, year conducted, databases searched, and risk of bias tool used. The decision to re-analyze pooled data from the included reviews is also discussed. A literature search strategy is provided in Appendix B.
What you will learn
Typology: Slides
1 / 49
This page cannot be seen from the preview
Don't miss anything!
Project
Prepared for
NHMRC | Natural Therapies Working Committee Canberra ACT 2601
Research Protocol prepared by Health Technology Analysts Pty Ltd
Jorgensen MA^1 , Allerdice S^1 , Baidya S^1 , Shikhule A^1 , Miles A^1 (^1) Health Technology Analysts, Level 1, 370 Norton Street, Lilyfield, New South Wales 2040 Australia
Please send all correspondence to Health Technology Analysts Pty Ltd mailing address: Level 1, 370 Norton Street, Lilyfield, New South Wales 2040 Australia email: natural.therapies@htanalysts.com.au
This Research Protocol received approval from the National Health and Medical Research Council (NHMRC) Natural Therapies Working Committee on 11 March 2021.
The NHMRC has been engaged by the Department of Health (Department) to update the evidence underpinning the 2015 Review of the Australian Government Rebate on Natural Therapies for Private Health Insurance (2015 Review) (1). The seven natural therapies to be reviewed in the first tranche are naturopathy, Pilates, Rolfing, shiatsu, Tai Chi, Western herbalism and yoga. These therapies are among those excluded from the private health insurance rebate as of 1 April 2019. To support the NHMRC in their evidence review, Health Technology Analysts (HTAnalysts) has been engaged to conduct a systematic review (SR) of the evidence of clinical effectiveness of Western herbal medicines. Eligible reviews submitted from the public, the Natural Therapies Review Expert Advisory Panel (NTREAP) and the Natural Therapies Working Committee (NTWC) will also be included in the evidence evaluation. This Research Protocol has been developed by HTAnalysts in conjunction with the NHMRC, the NTWC, and the NTREAP to provide a framework outlining the methodology that will be used to supplement the 2015 Review of Western herbalism. It is intended that all associated materials will be developed in a robust and transparent manner in accordance with relevant best practice standards (2, 3).
Figure 1 Framework for selecting the SR from which to extract data for any given PICO ............. 24
AMSTAR A MeaSurement Tool to Assess systematic Reviews BRISA Regional Base of Health Technology Assessment Reports of the Americas CI Confidence interval CINAHL Cumulative Index to Nursing and Allied Health Literature COMET Core Outcome Measures in Effectiveness Trials GRADE Grading of Recommendations Assessment, Development and Evaluation HR Hazard ratio MD Mean difference MeSH Medical Subject Headings NHAA National Herbalists Association of Australia NHMRC National Health and Medical Research Council NRSI Nonrandomised study of an intervention NTREAP Natural Therapies Review Expert Advisory Panel NTWC Natural Therapies Working Committee OR Odds ratios PAHO Pan American Health Organization PICO Population, Intervention, Comparator, Outcome PRISMA Preferred Reporting Items for Systematic Reviews and Meta-Analyses RCT Randomised controlled trial RoB Risk of bias RR Risk ratio SD Standard deviation SMD Standardised mean difference SR Systematic review TIDIER Template for Intervention Description and Replication TGA Therapeutic Goods Administration
Western herbalism is the primary form of herbal medicine utilised in Australia (7). A Western herbalist engages in extemporaneous compounding of herbs for therapeutic purposes for individuals under their care (8). Today, the practice of Western herbalism includes a holistic treatment framework that believes in treating individuals within a wider social, emotional, economical, spiritual and cultural framework and, like naturopathy, adherence to the principle of ‘first do no harm’ (9). Western herbalists may practice out of various settings including the home, clinical practices and multimodality centres. A survey of Western herbalists in Australia indicated that most practitioners (97.3%) have access to a herbal dispensary within their clinic (10). Western herbalism is practised for a range of reasons to improve general health and wellbeing, as well as to treat a variety of clinical and preclinical conditions. The current review is not limited to any particular condition or setting (see 3.1.2 Types of participants) and therefore, a concise description of each condition or problem, and the relevant setting, will be provided after conduct of the review.
Western herbalism is a traditional system of plant-based medicine derived primarily from Europe, the United Kingdom and North America (9). While medicinal plants from other herbal traditions, such as Traditional Chinese Medicine and Ayurvedic Medicine can be utilised by Western herbalists, the clinical application of Western herbalism is distinct from these traditions. Western herbal medicine uses plants and plant material to create medicines to help prevent or treat various illnesses. These materials may use some or all parts of a plant such as flowers, roots, stems and rhizomes, fruits and seeds, leaves and bark. WHMs are administered in various preparations including liquid herbal extracts such as tinctures or fluid extracts, oral tablets or capsules, or through topical application, for example, via poultices, creams and pessaries. Most commonly, liquid herbal extracts are prepared using an alcohol solvent, however, glycerol can be used as an alternative, when alcohol-based preparations may not be appropriate (e.g. when prescribing to children). Medicinal herbs can also be extracted in water, and this is commonly referred to as “tea” (9). In Australia, the regulation of herbal medicines differs depending upon the form, preparation and dosage of the herbal medicine. The TGA regulates some medicinal herbal products (tablets, capsules and liquid extracts), including through a list of permissible ingredients for products listed on the Australian Register of Therapeutic Goods. Others, such as raw plant materials (dried or fresh) used in teas, are unregulated beyond the guidelines applied to all food substances (8, 9); with the exception of herbal medicines listed as a scheduled substance on the Australian poisons standard (11). A survey of Western herbalists in Australia indicated that the most common preparation of herbs prescribed was liquid extracts (90%), followed by dried preparations such as teas (4.3%) and tablets and capsules (3.8%) (10). These preparations are usually dispensed as either an individualised mixture of one or multiple herbs or dispensed as proprietary formulae such as premanufactured tablets/capsules. Individual consumers also have access to some premanufactured herbal products through pharmacies, supermarkets and health food stores (10).
In Australia, complementary therapies, including Western herbalism, are most often used in conjunction with conventional medicine and other strategies for maintaining good health and wellness. For this reason, it is important to synthesise the evidence for the effectiveness of WHMs, to enable consumers, health care providers and policy makers to make informed decisions about care. The 2015 Overview identified no SRs containing evidence evaluating Western herbalism as a health service. The review noted that while there is a large body of research on the effects of individual herbal agents and remedies, the study of the real life practice and outcomes of herbalism as a health service is a relatively new area of research that has yet to be addressed in SRs (4, 5). The rationale for conducting this Overview of Reviews is to supplement the evidence and guidance used to inform the 2015 Overview of Western herbalism. That is, to identify studies published since, or not included in, the 2015 review and address the evidence gaps noted. This is to ensure recommendations relating to the use of Western herbalism remain relevant and up to date.
To conduct an Overview of Reviews to evaluate the effectiveness of WHMs in individuals with a described injury, disease, medical condition or preclinical condition. The Overview will compile the evidence from SRs of RCTs and pseudo-RCTs. The intent is to evaluate the evidence representative of the populations and conditions commonly seen by Western herbalists in Australia, the intervention(s) commonly used by the therapist, and outcomes that align with the reasons why patients use Western herbalism and/or practitioners prescribe WHMs.
3.1.1 Types of reviews Study design Eligible SRs are those that examine the effectiveness of eligible WHMs (see Section 3.1.3) compared to control (placebo or no intervention) or another intervention. An Overview of Reviews was selected as the most appropriate methodology, due to the likelihood of sufficient SRs being available to cover a broad range of populations, interventions and outcomes. The aim is to identify and appraise the best available evidence while ensuring the studies included in the review adequately represent the evidence across the full breadth of the PICO (i.e. all eligible comparisons and outcomes for a population or condition). The primary study of interest is a SR of RCTs (and pseudo-RCTs), with or without a meta-analysis. If the method of randomisation of a primary study included within a SRs is not specifically stated, or not considered strictly random, then the study will be considered to be pseudorandomised. Where a SR includes pseudo-RCTs, these will be considered as eligible along with data from RCTs. Reviews that do not report study eligibility criteria or conduct a comprehensive search of the literature (i.e. searching more than one database) will not be included. These reviews do not meet the minimum criteria to be considered ‘systematic’ and may not accurately summarise the body of evidence. Eligible reviews that include a single RCT will be included, as will SRs that include both RCTs and nonrandomised studies of an intervention (NRSIs); however, only evidence from the RCTs (and pseudo-RCTs) will be considered. See Section 3.3.11 for information on how meta-analyses from SRs will be handled where they include ineligible studies (e.g. NRSIs) or where they are missing one or more eligible studies. Additional study designs will not be considered irrespective of whether a SR for an individual herb, herbal combination or condition is identified. This includes individual RCTs or pseudo-RCTs not part of a SR, nonrandomised comparative studies (i.e. nonrandomised experimental trials, cohort studies, case-control studies, interrupted times series), cross-sectional studies and case series with either post-test of pre-test/post-test outcomes. This is because a SR of primary studies (i.e. RCTs or NRSIs) is not feasible given the timeframe and resources. Where a SR is not identified for an eligible population or intervention, this will be noted as an evidence gap. Overviews will not be eligible for inclusion and the search strategy (see Appendix B) is not specifically designed to identify them. However, overviews that are identified in literature searches or are submitted through the Department’s public call for evidence (see Section 3.2.2) will be checked to identify SRs that are eligible. Publication date There are no limitations on publication date, however, SRs published after the literature search date will not be eligible for inclusion. Reviews that are published (or submitted to the Department) after the literature search date will be listed within the ‘ Reviews awaiting classification ’ table of the evaluation report. These SRs will not be subject to a formal evidence evaluation however, a brief statement about the review and the potential impact of its findings on the overall conclusions of the evidence review will be included under the relevant sections of the review (including ‘ Overall completeness and applicability of evidence ’).
Reviews published in languages other than English The literature search, as well as the Department’s call for evidence, will not be limited by language of publication. Databases in languages other than English will not be searched, however, SRs in languages other than English may be identified via the English language databases. For pragmatic reasons, potentially eligible SRs will not undergo full text translation or data extraction, but will be documented via a process outlined in Section 3.3.1 ‘ Reviews published in languages other than English ’. 3.1.2 Types of participants People of any age with any injury, disease, medical condition or preclinical condition are eligible for inclusion. This includes disease prevention in at-risk healthy populations, which is broadly defined as those who are at increased risk of becoming ill or injured based on social, biomedical or behavioural risk factors (16). For the purposes of this review, social risk factors include income, education, employment and social support; biomedical factors include a person’s age, genetic make-up, and health status (such as obesity, high blood pressure, high cholesterol, vitamin deficiency); and behavioural factors include a person’s lifestyle choices (e.g. alcohol consumption, diet, exercise, tobacco and other drug use, etc.). Healthy participants seeking health improvement, such as general wellbeing, fitness, aesthetic improvements, resilience and cognitive or emotional intelligence are not eligible for inclusion; however, a study with eligible and ineligible populations will be included if separate data are available for the eligible population/s. 3.1.3 Types of interventions Interventions This Overview of Reviews aims to examine the effectiveness of core individual and combination herbal preparations used by western herbalists in Australia to treat a described injury, or treat or prevent disease, a medical condition or a preclinical condition. This includes the use of:
symptoms (e.g. pain, joint range of motion, strength, balance and accepted surrogate outcomes such as HbA1C for diabetes, body mass index for weight gain or loss, lung function tests), health related psychological/behavioural outcomes, health related quality of life, self-reported benefits, symptoms and functional ability, medication use or compliance with conventional medicine treatment; and injury or disease specific prevention outcomes (e.g. falls prevention, smoking cessation). Restrictions: Consistent with the terms of reference of NTREAP, personal health care preferences, patient-reported experience measures (PREMS) (e.g. satisfaction with care), safety, quality and economic outcomes are out of scope. Outcome measures and timepoints of interest Any effectiveness outcome anticipated to demonstrate a treatment achieves its intended purpose is eligible for inclusion (18, 19). There are no limitations on time points (e.g. short and long term outcomes) or outcome measure (e.g. objective and subjective measures such as clinical and laboratory assessments and patient-reported outcome measures [PROMS], preferably measured using validated tools, are eligible). As there are a broad range of populations eligible for inclusion in the review, it is not possible to prespecify outcomes. All prespecified outcomes reported in each eligible SR will be listed in the ‘ Characteristics of included reviews ’ tables; however, results will only be extracted for those outcomes identified as critical or important to the Overview. For each identified population, results for a maximum of seven critical (or important) outcomes will be reported in GRADE ‘Summary of Findings’ tables with corresponding evidence statements (see Section 3.3.17). Outcome selection will occur after identification of eligible reviews using a prespecified approach. To avoid introducing bias, outcomes will be prioritised by the NTWC, who will be provided with a list of conditions, outcome domains and outcome measurements (including measurement tools and time points) to prioritise. This list will be derived from the outcomes reported in SRs identified for inclusion in the Overview of Reviews, and, where available, the core outcome set/s for a particular condition (identified by searching COMET). Throughout the outcome prioritisation exercise, the NTWC will remain blinded about the characteristics or results of included SRs (and the included studies within), to prevent knowledge of study or review results, or other characteristics (such as study design) from influencing decision- making. In determining the critical and important outcomes, the NTWC will be guided by GRADE (6) and focus on the relevance and validity of outcome measures. Where appropriate, outcome domains reported using different measurement tools will be grouped and reported accordingly (see Section 3.3.8) Outcomes reported at different timepoints will be grouped and considered as follows: short term, intermediate term, long term or not specified. Determining whether something is considered short, intermediate or long term for a population will be guided by the published evidence, the NTWC and COMET. To avoid unit-of-analysis issues associated with repeated observations, data from a single time point will be selected for each outcome, as determined by the NTWC during outcome prioritisation. Where multiple timepoints are considered critical or important to decision-making (e.g. short- and long-term remission in symptoms) separate outcomes will be specified for each timepoint.
3.2.1 Electronic searches The literature search strategy (see Appendix B) was developed in Ovid (for Embase, MEDLINE, and Emcare) based on the key element of research question (i.e. the intervention). The search is not limited by population or outcome, but rather by study type; with methodological filters for identifying SRs and exclusions for publication types developed and published previously (20). In developing the search strategy, we appraised and adapted the relevant search strategies provided in the 2015 Review; with recent SRs identified in the scoping report and studies suggested by the NTWC also reviewed to identify other potentially relevant concepts. Terms or concepts proven not suitable were removed and other terms added. No date, language or geographic limitations will be applied when conducting the search. Non-English databases will not be searched. The search strategy will be adapted to suit the required syntax for the following electronic bibliographic databases:
To confirm combination herbal preparations are representative of WHM in Australia, a list of potentially relevant SRs will be supplied to a content expert for independent full text screening. The expert will confirm the appropriateness of the herbal combination/s as meeting the WHM eligibility criteria after examination of the SR (or primary studies within). Advice regarding the relevant grouping or subgrouping of the studies for analysis (with regards to the intervention) will also be sought at this time. If a review does not contain the required PICO information for a decision to be made regarding eligibility, the information will be sought from the SR authors through an open-ended request. Author names and included primary studies will be used to identify multiple reports arising from the same review. Eligible studies that are not available in English will be noted and managed as described below under ‘ Reviews published in languages other than English ’. Overviews identified in the literature searches will not be eligible for inclusion in the review, but will be checked for eligible SRs. Evidence provided through the Department’s public call for evidence Potentially relevant SRs identified by the NTWC, NTREAP, and other key stakeholders will be considered for inclusion if they satisfy the eligibility criteria described in Section 3.1 above. All of the submitted literature will be collated, tabulated, and cross-referenced with the evidence identified in the literature search described in Section 3.3.1. In-scope SRs not identified in the literature search will be incorporated into the evidence evaluation. A rationale for exclusion (as noted in Appendix C) will be provided for all SRs and primary studies considered out of scope (documented in a table within the technical report). Overviews provided through the Department’s public call for evidence will be checked for eligible SRs. Potentially relevant RCTs provided through the Department’s public call for evidence (see Section 3.2.2) will be checked to confirm if they have been identified within eligible SRs. If the RCT has been published after the literature search date of an eligible SR, they will be noted and discussed under the relevant section of the Overview. Relevant RCTs that are not be associated with a condition or intervention identified in this Overview will be listed in the technical report; however, these studies will not be subject to a formal evidence evaluation. Reviews published in languages other than English SRs published in languages other than English will undergo title and abstract translation using Google translate (or an equivalent tool). If online translation does not facilitate understanding of the title and abstract, then these reviews will be listed in a table as ‘Reviews unable to be translated or interpreted at the title/abstract stage’. Translated titles and abstracts will be screened to remove irrelevant citations, with articles excluded at title and abstract screen reported in the ‘Results of the search’. Translated titles and abstracts will be reviewed and evaluated against the ‘ Criteria for considering reviews for this Overview ’. Full text translation will not occur to determine eligibility. Reviews assessed as potentially eligible for inclusion in the Overview will be recorded in a ‘Reviews Awaiting Classification’ table. This information will also be reflected in the PRISMA flow diagram. The potential risk of language bias and its implications for the Overview will be discussed in relevant sections of the Evaluation Report (such as ‘ Overall completeness and applicability of evidence ’ and
‘ Agreements and disagreements with other studies or reviews ’). When assessing the extent to which language bias might influence the conclusions, we will also consider whether English-language reviews included in the Overview searched for, and included, studies published in languages other than English. 3.3.2 Data collection process Data from systematic reviews The characteristics of all included SRs will be extracted by one reviewer using a standard pre‐tested data extraction and coding form (see Appendix E). Outcome data will be extracted after agreement has been reached regarding the critical and important outcomes to be appraised (see Section 3.1.4). Pretesting will involve all reviewers data extracting the required information from the same three SRs which will be specifically selected to cover the breadth of the PICO identified for inclusion in the review. The lead reviewer will compare the data extraction forms to ensure the relevant data are extracted consistently between reviewers and as planned, with any necessary revisions made to ensure consistency. All data extraction forms will be checked for completeness and accuracy by the lead reviewer. Where there is uncertainty or disagreement regarding included data, a decision will be made through discussion. Data from included studies Full data extraction (and critical appraisal) of the primary studies included within a SR will not occur. If a return to primary studies is required to check or confirm information, we will note any discrepancies or adjustments made. 3.3.3 Requests for data Eligible SR protocols, SRs published as conference abstracts, and SRs not published in English will be identified for inclusion. Authors will be contacted through an open-ended request for data or further information. If no further data are available, the review will be noted as ‘ Ongoing ’ or ‘ Reviews awaiting classification’ and will not be included in the evidence appraisal. No attempts will be made to obtain or clarify data from authors of published peer-reviewed SRs or primary studies. 3.3.4 Data items The following characteristics of included SRs will be extracted: review objectives, study design (e.g. qualitative review, meta-analysis), year conducted, databases searched, date (and range) of documented search, SR eligibility criteria for participant characteristics (including demographics, comorbidities, etc.), SR eligibility criteria for intervention and comparator characteristics (including number of treatment sessions, program duration, co-interventions), outcomes reported in the SR (including measurement method, timing or severity), the method of synthesis/analysis employed, characteristics of included primary studies (number, study design features), risk of bias tool used to appraise included primary studies and their rating (noting review authors’ comments or concerns), funding sources and the overall conclusion of the SR. The data extraction forms will also note whether the SR searched for and included publications in languages other than English. Included primary studies will also be listed by author, date of publication and eligibility for inclusion in this review. When evaluating the effectiveness of WHM across the same PICO (See Section 3.3.
