A Review of the Quality of Extant Clinical Practice Guidelines in Cancer Therapy-Induced Mucositis


avatar Shahin Salarvand 1 , avatar Simin Hemati 2 , avatar Payman Adibi ORCID 3 , avatar Fariba Taleghani ORCID 4 , *

1 Students Research Committee, Faculty of Nursing and Midwifery, Isfahan University of Medical Sciences, Isfahan, Iran

2 Radiotherapy and Oncology Department, Faculty of Medicine, Isfahan University of Medical Sciences, Isfahan, Iran

3 Faculty of Medicine, Isfahan University of Medical Sciences, Isfahan, Iran

4 Nursing and Midwifery Care Research Center, Faculty of Nursing and Midwifery, Isfahan University of Medical Sciences, Isfahan, Iran

How to Cite: Salarvand S, Hemati S, Adibi P, Taleghani F. A Review of the Quality of Extant Clinical Practice Guidelines in Cancer Therapy-Induced Mucositis. Int J Cancer Manag. 2017;10(10):e11553. doi: 10.5812/ijcm.11553.


International Journal of Cancer Management: 10 (10); e11553
Published Online: October 31, 2017
Article Type: Review Article
Received: March 28, 2017
Revised: June 2, 2017
Accepted: September 16, 2017


Context: Oral mucositis is a common complication in patients with cancer therapy. Oral and dental care should be considered as a vital component of caring before, during and after chemotherapy. This study aimed to assess the methodological quality of existing CPGs about cancer therapy-induced mucositis.

Evidence Acquisition: We conducted a review of extant CPGs for the management of mucositis. After screening the guidelines based on inclusion and exclusion criteria, three CPGs were selected and assessed by 5 appraisers by using AGREE II instrument.

Results: Assessed CPGs gained the highest scores in the clarity of the presentation domain. Scope, purpose (median score, 63.3%), and stakeholder involvement (median score, 54.4%) were found respectively and the lowest scores were in editorial independence domain. The scores in applicability (median score, 30%), rigor of development (median score, 28.8%), and editorial independence (median score, 26.7%) domains were unfavorable. Also, every appraiser scored the overall quality of CPGs.

Conclusions: Quality of mucositis CPGs needs to be improved. In other words, designing high-quality CPGs in this area is necessary.

1. Context

Oral mucositis is a common complication for patients undergoing cancer therapy and has a significant negative effect on a patient’s quality of life (1, 2) as it disrupts eating ability, makes oral hygiene too difficult, and may result in low treatment adherence (3). Severe mucositis is a restricting factor of chemotherapy dose which can have a negative effect on prognosis (1) and bring about increase in overall costs for patient care and health care system (3). For this reason, oral and dental care should be considered as vital components of caring before, during and after chemotherapy (4).

Evidence-based guidelines are considered as powerful instruments to achieve effectiveness and efficient care by professionals, authorities, policy makers, and are valuable instruments in cancer patient care (5). Clinical practice guidelines (CPGs) are systematically developed statements to help people with decision making in the clinical setting (6). But the quality of currently used methods is worrying (7, 8). The quality of the CPGs is defined as: “Ensuring that the biases are likely in developing CPGs process, appropriately have been addressed and recommendations are validated internally and externally and feasible in practice” (6, 9). Given this, CPGs must have special quality criteria that have been defined and validated by AGREE collaboration (5). The aim of AGREE II instrument is providing a framework for assessing the quality of guidelines, presenting a methodological strategy for developing guidelines, promoting that information and how it should be reported in CPGs (6, 9) and helping health care providers to assess a clinical practice guideline before application of its recommendations (8). In doing literature searches, only one study was found on this subject conducted by Potting, et al. which assessed the quality of related CPGs with cancer therapy-induced mucositis, which mentioned documents as guidelines which are lacking effective guideline characteristics (10). This study aimed to assess the methodological quality of existing CPGs about cancer therapy-induced mucositis from different countries and websites.

2. Evidence Acquisition

We conducted a review of published CPGs for the management of cancer therapy-induced mucositis.

2.1. Selection Criteria

Inclusion criteria were: 1, having a clear statement introducing itself as “guideline”; 2, The target population is the adult (above 18 years old); 3, Mucositis induced by cancer therapy; 4, The language of the guideline is English; 5, CPGs were selected which had recommendations including prevention, diagnosis and management for mucositis; 6, CPGs developed by related institutions, communities, associations, cancer care groups, and societies; and 7, Having a guideline structure and characteristics. Included guidelines were original reports based on systematic review and research evidence. We searched whether guidelines were evidence based, whether they reported search strategy, stratified quality of evidence and graded rigor of evidence. If CPGs have been updated, we assessed the last version of the guideline.

Exclusion criteria were: clinical pathways, systematic reviews, protocols, instructional booklets, editorial, narrative reviews, patients’ guides and books.

The present study is a part of a Ph.D. thesis, with the confirmation of the vice-chancellor in research affairs (Code: 395651).

2.2. Search Strategy and Screening of Guidelines

For selecting guidelines, we did a comprehensive search of websites and databases: National comprehensive cancer network (NCCN), the national guideline clearinghouse (NGC), the national institute for health and clinical excellence (NICE), the scottish intercollegiate guidelines network (SIGN), cancer care ontario (CCO), oncology nursing society (ONS), NHSC (http://www.health.gov.au/), guidelines international network (GIN), MEDLINE, Science Direct, Cochrane Library, Scopus, ProQuest, EMBASE, PubMed, Google Scholar, Google, Yahoo, cumulative index to nursing and allied health literature (CINAHL). The mentioned sites were used to find acceptable guidelines published from observation time until April 2017. Also, Persian databases such as SID, Magiran and Iranian medical universities’ websites with keywords Mucositis or oral care and cancer or cancer therapy or chemotherapy and guideline or recommendations and/or combinations of these keywords in the English language were searched.

After finding relevant studies, our research team perused them. Titles, abstracts and full text of the guidelines were assessed and the related ones were screened based on the inclusion and exclusion criteria.

2.3. Appraising Screened CPGs by Applying AGREE II Instrument

After the research team screened guidelines based on the inclusion and exclusion criteria, 5 appraisers assessed screened CPGs by AGREE II instrument independently. The AGREE II instrument updated in 2013 (8) was used for assessing quality of CPGs.

AGREE II instrument contains 6 domains, including “Scope and purpose, stakeholder involvement, rigor of development, clarity of presentation, applicability, editorial independence” (Table 1), and 23 items. At last, in the Overall Assessment domain, an overall score of guideline quality and appraisers’ recommendations for using it are provided. The scoring of this instrument is as follows: a quality score is calculated for each of the 6 domains in the AGREE II instrument. A 7 point scale from 1-strongly disagree (lack of item)- to 7-strongly agree (having expected quality)- was applied to measure the extent that each criterion is provided by the guideline (8, 11, 12). Domain scores were given in percentages. Although they can be compared among guidelines, they are not designed for providing a total score. In other words, scores of the 6 domains are independent and should not be added as a single quality score. The calculated score for each domain is based on this formula: obtained score - minimum possible score/ maximum possible score - minimum possible score (8).

For increasing reliability of assessment according to AGREE II instrument, there were 5 experienced appraisers who scored each guideline independently. For enhancing consistency with existing studies that have used AGREE II instrument and identifying clearly potential useful guidelines in cancer therapy-induced mucositis, overall assessment of AGREE II instrument (3-recommended, 2-recommended with modifications, 1-not recommended) were determined independently by each appraiser. Then, if assigned scores of each domain demonstrated significant variation, we asked appraisers to revise them, but minor differences were ignored. Next, the agreement was achieved by the research team. Finally, the median score was calculated for each item. We practiced according to appraisers’ final scores and AGREE II instrument instruction. Also, we determined satisfactory quality for any guideline that scored above 60% in all six domains, median scores under 30% (unfavorable), median scores between 30– 60% (relatively unfavorable) and above 60% (favorable), as defined by AGREE-II.

3. Results

3.1. Study Selection

After an extensive search, 34 documents were found whose titles, abstracts and full-text the research team assessed according to eligibility criteria. Eventually, 3 CPGs included (Figure 1).

3.2. Characteristic of CPGs

A summary of characteristics of included CPGs is shown in Table 1. All of the CPGs were developed by an evidence-based approach, were updated versions, and focused on prevention, assessment and management for mucositis.

Table 1. Summary and Characteristics of Cancer Therapy-Induced Mucositis Clinical Practice Guidelines
Guideline TitleDate ReleasedCountry or RegionInstituteUpdateType of GuidelineFocus of GuidelineFundingSize of Complete Guideline, pgs
The oral management of oncology patients requiring radiotherapy, chemotherapy and / or bone marrow transplantation1997EnglandThe Royal College of Surgeons of England / The British society for disability and oral health(1) 2004, (2) 2012Evidence based guidelineOral health management of oncology patientsnot disclose58
Mouth care guidance and support in cancer and palliative care2012EnglandThe United Kingdom oral mucositis in cancer care group (UKOMiC)(1) 2015Evidence based guidelineOral health management of oncology patientsnot disclose12
MASCC/ISOO clinical practice guidelines for the management of mucositis secondary to cancer therapy2004U.S.ANational guideline clearing house (NGC)(1) 2014Evidence based guidelineManagement of mucositisBioAlliance pharma and helsinn healthcare, SA.13

CPGs were supported financially by professional organizations, governments and academic societies, except for two CPGs which did not disclose their funding sources. All of the three selected CPGs were developed by medical societies, care groups, and professional organizations.

3.3. Appraising CPGs by Using the AGREE-II Instrument

Five appraisers assessed 3 included CPGs using the AGREE-II instrument; Table 2 shows the summaries of appraising each guideline. Assessed CPGs, gained highest scores in the clarity of presentation domain, scope and purpose (median score, 63.3%), stakeholder involvement (median score, 54.4%) respectively, and the lowest scores in editorial independence domain. The scores in applicability (median score, 30%), rigor of development (median score, 28.8%), and editorial independence (median score, 26.7%) domains were unfavorable. Also, every appraiser scores the overall quality of guidelines, guideline number 3, obtained higher scores for recommending and in all of the domains’ scores than two other guidelines. Overall results of 6 domains of AGREE II instrument in 3 guidelines with descending respectively were clarity of presentation (median score, 92.2%), scope and purpose (median score, 63.3%), stakeholder involvement (median score, 54.4%), applicability (median score, 30%), rigor of development (median score, 28.8%), editorial independence (median score, 26.7%), and overall assessment (63.3%).

Table 2. Individual AGREE II Domain Results for Each Cancer Therapy-Induced Mucositis Guideline and Median Scores of All Domains
Guideline TitleScope and Purpose, %Stakeholder Involvement, %Rigour of Development, %Clarity of Presentation, %Applicability, %Editorial Independence, %Overall Assessment, %Recommendation
The oral management of oncology patients requiring radiotherapy, chemotherapy and / or bone marrow transplantation75.554.428.893.33026.753.36
Mouth care guidance and support in cancer and palliative care63.346.726.385.511.726.763.39
MASCC/ISOO clinical practice guidelines for the management of mucositis secondary to cancer therapy64.462.259.692.251.7958013

In general, amongst all of the AGREE II instrument’s domains, in assessed guidelines, clarity of presentation, scope and purpose domains had favorable scores and other domains had unfavorable and relative favorable scores.

This study aimed to assess the methodological quality of existing CPGs about cancer therapy-induced mucositis from different countries and websites and suggested the quality assessment of existing CPGs of cancer therapy-induced mucositis by AGREE II instrument. Scope and purpose domain are related to the overall aim, the target population, and the specific healthcare questions (items 1 - 3) (8). The present study indicated that the scores of this domain for the 3 assessed guidelines were favorable (median score, 63.3%). In the study of Sabharwal, et al. also scope and purpose domain had the highest score (median score, 95%) in comparison to other domains of AGREE II instrument (13). But in our study, the obtained highest score was in the clarity of presentation (median score, 92.2%). Clarity of presentation is concerned with the language, structure, and framework of the guideline and emphasizes clarity, specificity, and unambiguous recommendations (items 15 - 17) (8). The study of Birken, et al. and Deng, et al. showed that assessed guidelines obtained the highest score in this domain (14, 15). Also, in the study of Sabharwal, et al., another domain taking high scores was clarity of presentation (median score, 92%) as appraisers found guidelines providing clear recommendations were easily identifiable (13).

In stakeholders’ involvement domain, it was assessed whether the professional group is presented, patients’ preference and perspective have been acquired, and intended users of guideline clearly has been defined (items 4 - 6) (16). In the present study, stakeholders involvement domain had a higher score than other domains except for clarity of presentation and scope and purpose domains (median score, 54.4%) but its score was not favorable. The study of Sabharwal, et al. showed stakeholders involvement domain (median score, 83%) obtained a favorable score and after 3 domains: scope and purpose, rigor of development and clarity of presentation, had a higher score than other domains (13). Applicability domain focuses on potential barriers and facilitators for implementing guidelines, strategies for promoting, and implications of resources for using the guideline (items 18 - 21) (8). In the present study, applicability (median score; 30%) took an unfavorable score. In two studies of Sabharwal, et al., Xie, et al. and Deng, et al., this domain obtained the lowest score amongst all domains of the AGREE II instrument and in other studies; applicability domain took weak score among the 6 domains of AGREE II instrument (11, 13, 15-18). The weak scores in applicability domain place emphasis on the necessity of improving the conceptualization in guideline implementation during the development process.

The rigor of development is the core of the methodology of guidelines and continued search processes of evidence, grading, briefing and formulation of recommendations (Items 7 - 14) (16). The present study showed domain of rigor of development (median score, 28.8%) was in unfavorable condition. The findings of the study of Cranney, et al. showed that consistency with the methodological quality of current osteoporosis CPGs was low and practically none of CPGs cover dissemination subjects and a few guidelines were judged as acceptable for applying in their current format (19). Moreover, other studies found that most of the CPGs had serious methodological defects (10, 17).

The domain of editorial independence focuses on the investment of issues and conflict of interest for all of the involvement members (Items 22 - 23) (16). In the present study, we found that assessed guideline had the lowest score in this domain. The median scores were not at a favorable level (< 60%). Two other studies showed that almost all of the assessed CPGs were evaluated as weak in the domain of editorial independence (11, 17). Given that the conflict of interest is the most common source of bias in guideline development (20), in the present study, only one of the CPGs reported the conflict of interest.

4. Conclusions

In the overall quality assessment in the present study, obtaining a score (median score, 63.3%) has had a relative favorable level by appraisers (the lowest level of favorable score) and it necessitates undertaking modification and promoting the quality of CPGs. The Potting’s study showed that it is necessary to promote the methodological quality of guidelines if they were applied in clinical practice (10). The results of Briken, et al. indicated that the quality of survivorship of CPGs was weak (14). In addition, the overall recommendation domain is related to rating overall quality of guidelines and whether the guideline is recommended for use in clinical practice (8). In the present study, guideline (number 3) obtained the highest score for recommendation. Several studies indicated a variation to the recommendation of CPGs (10, 13, 14, 16-18). It was concluded in this study that the quality of cancer therapy-induced mucositis CPGs needs to be improved and designing high-quality CPGs must be considered. Incorporating the process of applying resources and monitoring and standardization of the implementation of the CPGs needs to be improved. Therefore, studies conducted to analyze barriers and facilitators of guideline implementation will be useful.




  • 1.

    Dos Reis PE, Ciol MA, de Melo NS, Figueiredo PT, Leite AF, Manzi Nde M. Chamomile infusion cryotherapy to prevent oral mucositis induced by chemotherapy: a pilot study. Support Care Cancer. 2016;24(10):4393-8. doi: 10.1007/s00520-016-3279-y. [PubMed: 27189615].

  • 2.

    Al Ibraheemi AA, Shamoun S. Incidence and Risk Factors of Oral Mucositis in Patients with Breast Cancer Who Receiving Chemotherapy in Al-Bashir Hospital. Int J Hematol Oncol Stem Cell Res. 2016;10(4):217-23. [PubMed: 27928476].

  • 3.

    Manzi Nde M, Silveira RC, dos Reis PE. Prophylaxis for mucositis induced by ambulatory chemotherapy: systematic review. J Adv Nurs. 2016;72(4):735-46. doi: 10.1111/jan.12867. [PubMed: 26626711].

  • 4.

    Elad S, Raber-Durlacher JE, Brennan MT, Saunders DP, Mank AP, Zadik Y, et al. Basic oral care for hematology-oncology patients and hematopoietic stem cell transplantation recipients: a position paper from the joint task force of the Multinational Association of Supportive Care in Cancer/International Society of Oral Oncology (MASCC/ISOO) and the European Society for Blood and Marrow Transplantation (EBMT). Support Care Cancer. 2015;23(1):223-36. doi: 10.1007/s00520-014-2378-x. [PubMed: 25189149].

  • 5.

    Grol R, Cluzeau FA, Burgers JS. Clinical practice guidelines: towards better quality guidelines and increased international collaboration. Br J Cancer. 2003;89 Suppl 1:S4-8. doi: 10.1038/sj.bjc.6601077. [PubMed: 12915896].

  • 6.

    National Collaborating Centre for Methods and Tools (NCCMT) . Critically appraising practice guidelines: The AGREE II instrument. Hamilton: McMaster University; 2013. Available from: http://www.nccmt.ca/resources/search/100.

  • 7.

    Burgers JS. Towards evidence-based clinical practice: an international survey of 18 clinical guideline programs. Int J Qual Health Care. 2003;15(1):31-45. doi: 10.1093/intqhc/15.1.31.

  • 8.

    Brouwers MC, Kho ME, Browman GP, Burgers JS, Cluzeau F, Feder G, et al. AGREE II: advancing guideline development, reporting and evaluation in health care. CMAJ. 2010;182(18):E839-42. doi: 10.1503/cmaj.090449. [PubMed: 20603348].

  • 9.

    Dans AL, Dans LF. Appraising a tool for guideline appraisal (the AGREE II instrument). J Clin Epidemiol. 2010;63(12):1281-2. doi: 10.1016/j.jclinepi.2010.06.005. [PubMed: 20605571].

  • 10.

    Reames BN, Krell RW, Ponto SN, Wong SL. Critical evaluation of oncology clinical practice guidelines. J Clin Oncol. 2013;31(20):2563-8. doi: 10.1200/JCO.2012.46.8371. [PubMed: 23752105].

  • 11.

    Armstrong JJ, Rodrigues IB, Wasiuta T, MacDermid JC. Quality assessment of osteoporosis clinical practice guidelines for physical activity and safe movement: an AGREE II appraisal. Arch Osteoporos. 2016;11:6. doi: 10.1007/s11657-016-0260-9. [PubMed: 26759266].

  • 12.

    Burls A. AGREE II-improving the quality of clinical care. Lancet. 2010;376(9747):1128-9. doi: 10.1016/S0140-6736(10)61034-3. [PubMed: 20599263].

  • 13.

    Sabharwal S, Patel NK, Gauher S, Holloway I, Athanasiou T. High methodologic quality but poor applicability: assessment of the AAOS guidelines using the AGREE II instrument. Clin Orthop Relat Res. 2014;472(6):1982-8. doi: 10.1007/s11999-014-3530-0. [PubMed: 24566890].

  • 14.

    Birken SA, Ellis SD, Walker JS, DiMartino LD, Check DK, Gerstel AA, et al. Guidelines for the use of survivorship care plans: a systematic quality appraisal using the AGREE II instrument. Implement Sci. 2015;10:63. doi: 10.1186/s13012-015-0254-9. [PubMed: 25935752].

  • 15.

    Deng Y, Luo L, Hu Y, Fang K, Liu J. Clinical practice guidelines for the management of neuropathic pain: a systematic review. BMC Anesthesiol. 2016;16:12. doi: 10.1186/s12871-015-0150-5. [PubMed: 26892406].

  • 16.

    Xie Z, Wang X, Sun L, Liu J, Guo Y, Xu B, et al. Appraisal of clinical practice guidelines on community-acquired pneumonia in children with AGREE II instrument. BMC Pediatr. 2016;16:119. doi: 10.1186/s12887-016-0651-5. [PubMed: 27484087].

  • 17.

    Potting C, Mistiaen P, Poot E, Blijlevens N, Donnelly P, van Achterberg T. A review of quality assessment of the methodology used in guidelines and systematic reviews on oral mucositis. J Clin Nurs. 2009;18(1):3-12. doi: 10.1111/j.1365-2702.2008.02493.x. [PubMed: 19120727].

  • 18.

    Burgers JS, Fervers B, Haugh M, Brouwers M, Browman G, Philip T, et al. International assessment of the quality of clinical practice guidelines in oncology using the Appraisal of Guidelines and Research and Evaluation Instrument. J Clin Oncol. 2004;22(10):2000-7. doi: 10.1200/JCO.2004.06.157. [PubMed: 15143093].

  • 19.

    Cranney A, Waldegger L, Graham ID, Man-Son-Hing M, Byszewski A, Ooi DS. Systematic assessment of the quality of osteoporosis guidelines. BMC Musculoskelet Disord. 2002;3:20. doi: 10.1186/1471-2474-3-20. [PubMed: 12174195].

  • 20.

    Detsky AS. Sources of bias for authors of clinical practice guidelines. CMAJ. 2006;175(9):1033. 1035. doi: 10.1503/cmaj.061181. [PubMed: 17060643].

  • Copyright © 2017, Cancer Research Center (CRC), Shahid Beheshti University of Medical Sciences. This is an open-access article distributed under the terms of the Creative Commons Attribution-NonCommercial 4.0 International License (http://creativecommons.org/licenses/by-nc/4.0/) which permits copy and redistribute the material just in noncommercial usages, provided the original work is properly cited.