Quality assessment of the Saudi initiative for asthma guideline

Fahad Alfhaid; Mohammed Aljulifi; Tahir Ansari; Abdullah Al-Olayan; Mohammed Almansour; Talal Shakhs Alghamdi

doi:10.4103/ijas.ijas_10_18

ORIGINAL ARTICLE

Year : 2018 | Volume : 3 | Issue : 1 | Page : 28-32

Quality assessment of the Saudi initiative for asthma guideline

Fahad Alfhaid¹, Mohammed Aljulifi¹, Tahir Ansari², Abdullah Al-Olayan³, Mohammed Almansour¹, Talal Shakhs Alghamdi¹
¹ Department of Clinical Sciences-Family Medicine, College of Medicine, Majmaah University, Al Majma'ah, Kingdom of Saudi Arabia
² Department of Clinical Sciences-Medicine, College of Medicine, Majmaah University, Al Majma'ah, Kingdom of Saudi Arabia
³ Department of Clinical Sciences-Paediatric Pulmonology, College of Medicine, Majmaah University, Al Majma'ah, Kingdom of Saudi Arabia

Date of Submission	04-Sep-2018
Date of Acceptance	06-Sep-2018
Date of Web Publication	17-Oct-2018

Correspondence Address:
Dr. Abdullah Al-Olayan
Department of Clinical Sciences-Paediatric Pulmonology, College of Medicine, Majmaah University, Al Majma'ah 11952
Kingdom of Saudi Arabia

Source of Support: None, Conflict of Interest: None

Check

DOI: 10.4103/ijas.ijas_10_18

Abstract

Objective: The objective of this study is to assess the quality of Saudi initiative for asthma (SINA) guideline using appraisal of guidelines for research and evaluation (AGREE II) tool.
Design: This study design was methodological evaluation of SINA Guidelines using the AGREE II instrument.
Place and Duration of Study: This study was conducted from September to December 2017 in the college of medicine, Majmaah University, Saudi Arabia.
Methods: The appraisers assessed the quality of SINA guideline. A descriptive statistical analysis through the calculation of the total, standardized-score value by each reviewer, and the value per domain were performed. Six reviewers evaluated the guideline independently; four of them were family physicians, one was Internist, and the other was pediatric pulmonologist.
Results: We found that clarity of presentation for SINA scored the best (64%). The guideline scored average on the domain related to “scope and purpose” and “stakeholder involvement.” The lowest score was in “applicability” and “editorial independence” domain (25%), respectively. However, the overall quality of the SINA guideline was low, and the reviewer suggested the use of the current guideline after modification.
Conclusion: The results of our studies emphasize the need for addressing the domains with low scores including rigor of development, applicability, and editorial independence. However, the reviewers recommend using SINA guideline for asthma management with taking into consideration the modifications that were noted during the appraisal. We also recommend using AGREE II tool to develop and improve the next version of SINA guideline.

Keywords: Appraisal of guidelines for research and evaluation II, asthma, clinical practice guideline, Saudi initiative for asthma guideline

How to cite this article:
Alfhaid F, Aljulifi M, Ansari T, Al-Olayan A, Almansour M, Alghamdi TS. Quality assessment of the Saudi initiative for asthma guideline. Imam J Appl Sci 2018;3:28-32

How to cite this URL:
Alfhaid F, Aljulifi M, Ansari T, Al-Olayan A, Almansour M, Alghamdi TS. Quality assessment of the Saudi initiative for asthma guideline. Imam J Appl Sci [serial online] 2018 [cited 2023 May 28];3:28-32. Available from: https://www.e-ijas.org/text.asp?2018/3/1/28/243621

Introduction

Asthma is among the few common illnesses worldwide. It affects about 334 million people worldwide.^[1] In Saudi Arabia, many studies are assessing the prevalence in different cities. However, there is no national survey conducted to estimate the prevalence. One study concludes that the prevalence is increasing by comparing the prevalence in different cities and different years, 1986 and 1995.^[2] More than two million Saudis are affected by asthma.^[3] Recent research evaluates statics from a national domiciliary audit conducted in Saudi Arabia in 2013 and found that 4.05% of the Saudi population was suffering from asthma.^[4]

Saudi initiative for asthma (SINA) was developed in 2009 to make a national guideline for asthma management. A group of Saudi physician developed SINA guideline, and they are members of the Saudi Thoracic Society. The SINA guideline had three versions: the first one was published in 2009,^[5] then, the second version on 2012,^[6] and then, later in 2016.^[3] The goal of SINA guidelines is to have amended and revised recommendations, which are convenient and easy to understand and simple to utilize by physicians have limited experience in treating asthma, as well as primary care and general practice physicians.^[3],[6]

Clinical practice guidelines (CPGs) have been developed to help health practitioner in making decisions quickly in a scientific way. CPG has been defined as “statements that include recommendations intended to optimize patient care that is appraised by a systematic review of evidence and an assessment of the benefits and harms of alternative care options.”^[7] These guidelines are very helpful in patient care and have an impact on the improved quality outcome of patients. Many studies showed that the methodological quality of different guidelines is not well established and there is a variety of the different guidelines.^[8] Guidelines are needed to be well structured using the specific quality tool, and they should be ideally created within an established and composed guidelines program.^[9]

To appraise the quality of guidelines, one of the commonly used tools is “The Appraisal of Guidelines for Research and Evaluation” AGREE II tool. It was published 1^st time in 2003 by a group of global guideline developers and analysts, the AGREE Collaboration.^[10] Later on, they released the modified version in 2010, AGREE II. It was used by many studies to assess the quality of different guidelines.^{[11],[12],[13]}

To the best of our knowledge, no previous study evaluated the quality of Saudi asthma guideline systematically. Therefore, the objective of this study was to explore the quality of SINA guideline using AGREE II tool.

Methods

The Saudi initiative for bronchial asthma 2016 is the guidelines for the diagnosis and management of asthma in adults and children, was chosen for critical appraisal. This study was conducted from September to December 2017 in the college of medicine, Majmaah University. Six reviewers evaluated the guideline independently, four of them were family physicians, one was Internist, and the other was pediatric pulmonologist with more than 10 years' experience in their relevant specialty, respectively. All the reviewers are familiar with the notions of building and developing the guidelines and methodology of research as well. The reviewer assessed the quality of SINA guideline using the AGREE II instrument.

The AGREE II is both reliable and valid tool.^[14],[15] The AGREE II is a tool composed of six domains involving 23-item succeed by two overall rating items. The 1^st domain is “scope and purpose.” There are three items included in this domain and is covering the aim of the guideline, the specific health questions, and the target population. The 2^nd domain is focused on the “Involvement of stakeholder” in the development of the guideline and whether the developers considered the views of the representative stakeholders. The 3^rd domain concerns the process used to make recommendations. This domain is “rigor of development.” It consists of eight items. The 4^th domain is “clarity and presentation.” The domain oversees the format and structure of the guideline. Three items were included in this domain. The 5^th domain is “applicability.” The applicability focuses on implementation and strategies to improve resources of the guideline. The last domain is about “editorial Independence.” Indeed, this domain is concerned with the building of unbiased recommendations. It composed of two items. We use 7-point Likert scale to score each item of different domain. This scale was ranged between score one for strongly disagree to score seven for strongly agree. Finally, the overall assessment includes two items:First one is focused on the opinion of the appraisal regarding the rating of the overall quality of the guideline. The 2^nd one is whether the guideline would be recommended in practice or not. This item composed of three choices: recommended; recommended with modifications; or not recommended. The score for each domain was calculated using specific following formula: (actual score − minimal possible score)/(maximal possible score − minimum possible score) × 100%.

Finally, the overall judgment of the guideline by reviewers reached consensus according to item assessment and the general impression. A descriptive analysis was calculated by summing up the total score and calculating the value of each domain. There is no minimum score decided by the assessment team. The overall judgment was made by how the SINA match the AGREE II criteria and by comparing to the other appraised guideline in the literature.

Results

The overall quality of the SINA guideline was below average (43%). However, guideline scored highest in the domain related to clarity of presentation. The score range was in between (25%–64%) in different domains. The reviewers found that clarity of presentation domain performed the best (64%) among all other domains. [Figure 1] summarizes the results of the appraisal and the mean score for each domain.

Figure 1: Individual appraisal of guidelines for research and evaluation II domain results for Saudi Initiative for Asthma guideline

Click here to view

Domains-3 related to the “rigor of development” and Domain-5 “applicability” have the substantial influence on the results of the total assessments, while the other domains have a fluctuating influence.^[16] Our study shows disquieting low score (36% and 25%), respectively, in both of the domains. Results are shown in [Figure 2].

Figure 2: Domain-3 and 5 score of Saudi initiative for asthma guideline

Click here to view

SINA scored average on the domain related to “scope and purpose,” and stakeholder involvement (50% and 49%), respectively. Editorial independence (25%) received the lowest scores of all the AGREE II domains. The reviewers recommend using SINA guideline for asthma management with taking into consideration the modifications that were noted and suggested during the appraisal. The results of each domain are presented in [Figure 3].

Figure 3: Different domain score of Saudi initiative for asthma guideline

Click here to view

Discussion

Our study results show that overall assessments of SINA are below average (43%). The domains such as the “rigor of development,” “applicability,” and “editorial independence” also have lower scores. Other similar studies labeled below average quality to that guideline which scoreless then (50%) overall and individual domains, while the rigor of development and applicability have the strongest effect on overall quality of a guideline.^[16],[17] We find these domains have a low score and that might be a reason for overall below average assessment of guideline. The development and revision of guideline need a quality instrument for betterment. The AGREE instrument is universal and can be utilized to guidelines in the different area of medicine, especially those for health promotion, diagnosis, treatment, or interventions.^[18] SINA did not mention which tool is used to build up this guideline.

Our study reports that scores for scope, aim, and target population have 50%. This is close to other studies as only two CPGs scored 60% in this particular domain. The community to whom the guideline is intended to apply is described in a good way. However, the health questions need to be more obvious. These results could be referred to minimal mentioned details in the description of the overall objective, health question, and target-population definition.^[19]

The score for stakeholder involvement is 48.88% in our study. They did not involve patients, nurses, GPs, or methodologist. Patients' participation in decisions regarding their management has shown promising results. Development of a guideline requires an inter-professional team that includes physicians, methodologic experts, policymakers, and patient representatives. In this domain, the target users are mentioned, but it is not easy to find inside the guideline. The domain-2 is reflecting the end users' view by involving them in the development of the guidelines.^[19],[20]

The score for “rigor of development” domain remained 35.83%. The low score of this domain related to the poor performance of guideline while using any guideline appraisal instrument.^[16],[20] The finding is mirroring either the lack of methodologic expertise, which is often linked to inadequate resources or guideline developed by agreement of experts and not by evidence synthesis. The reviewer does not find statement described its procedure for updating guidelines. Involving methodologist may result in the improvement of the quality of guideline.

The score for clarity of presentation domain remained 63.44%. The clarity of presentation domain addresses potential, language, and format, respectively. In this domain, the recommendations were specific and unambiguous with different options for management. Key recommendations were conclusively identifiable, and they were presented. However, the clinical recommendations of SINA are scattered inside the guideline. It is difficult for the readers to identify them easily.

The lowest score of all six domains was in applicability and editorial independence 25%, respectively. The 5^th domain of applicability addresses implementation's different perspectives. Many clinical guidelines have the similar results in applicability domains and its ongoing phenomena even with SINA guideline.^[21] In “applicability” domain, the potential resource implications of applying the recommendations need to be considered to make the guideline more applicable. While in editorial independence domain, the competing interest of guideline development group members should be apparent for the guideline's users. A previous review reached similar results for this domain in different guidelines; they refer that to a separation between development and implementation activities by the guidelines panel.^[21] Editorial independence domain also scored 25%, as with few guidelines scoring above 30% in a study.^[22]

We have few limitations, as a newly established medical college, we have limited number of the reviewer. The reviewer was from different specialties, and they analyzed the guideline in their perspective, they were aware of using AGREE II tool, but still, formal training for using assessment instrument is needed. As we did not find in AGREE II the relative importance in different domains, also the tool does not explain clear guidance for overall assessment of the guideline.

Conclusion

The results of our study emphasize the need for addressing the domains with low scores including rigor of development, applicability, and editorial independence. This guideline has a potential for continuous improvement, especially in rigor for development. Patient participation is needed for enhancing applicability. There is a need to involve specialists from other areas also to review the clinical guidelines for more reliable and valid results.

Financial support and sponsorship

Nil.

Conflicts of interest

There are no conflicts of interest.

References

1.	The Global Asthma Report 2014. Auckland, New Zealand: Global Asthma Network, 2014.
2.	Al Frayh AR, Shakoor Z, Gad El Rab MO, Hasnain SM. Increased prevalence of asthma in Saudi Arabia. Ann Allergy Asthma Immunol 2001;86:292-6.
3.	Al-Moamary MS, Alhaider SA, Idrees MM, Al Ghobain MO, Zeitouni MO, Al-Harbi AS, et al. The Saudi initiative for asthma – 2016 update: Guidelines for the diagnosis and management of asthma in adults and children. Ann Thorac Med 2016;11:3-42. [PUBMED] [Full text]
4.	Moradi-Lakeh M, El Bcheraoui C, Daoud F, Tuffaha M, Kravitz H, Al Saeedi M, et al. Prevalence of asthma in Saudi adults: Findings from a National Household Survey, 2013. BMC Pulm Med 2015;15:77.
5.	Al-Moamary MS, Al-Hajjaj MS, Idrees MM, Zeitouni MO, Alanezi MO, Al-Jahdali HH, et al. The Saudi initiative for asthma. Ann Thorac Med 2009;4:216-33. [PUBMED] [Full text]
6.	Al-Moamary MS, Alhaider SA, Al-Hajjaj MS, Al-Ghobain MO, Idrees MM, Zeitouni MO, et al. The Saudi initiative for asthma – 2012 update: Guidelines for the diagnosis and management of asthma in adults and children. Ann Thorac Med 2012;7:175-204. [Full text]
7.	Graham R, Mancher M, Wolman DM, Greenfield S, Steinberg E. Clinical Practice Guidelines We Can Trust. Washington DC: The National Academies Press; 2011.
8.	Burgers JS, Grol R, Klazinga NS, Mäkelä M, Zaat J; AGREE Collaboration, et al. Towards evidence-based clinical practice: An international survey of 18 clinical guideline programs. Int J Qual Health Care 2003;15:31-45.
9.	Grol R, Cluzeau FA, Burgers JS. Clinical practice guidelines: Towards better quality guidelines and increased international collaboration. Br J Cancer 2003;89 Suppl 1:S4-8.
10.	Brouwers MC, Kho ME, Browman GP, Burgers JS, Cluzeau F, Feder G, et al. AGREE II: Advancing guideline development, reporting and evaluation in health care. CMAJ 2010;182:E839-42.
11.	Radwan M, Akbari Sari A, Rashidian A, Takian A, Abou-Dagga S, Elsous A, et al. Appraising the methodological quality of the clinical practice guideline for diabetes mellitus using the AGREE II instrument: A methodological evaluation. JRSM Open 2017;8:2054270416682673.
12.	Jiang M, Liao LY, Liu XQ, He WQ, Guan WJ, Chen H, et al. Quality assessment of clinical practice guidelines for respiratory diseases in China: A systematic appraisal. Chest 2015;148:759-66.
13.	Sabharwal S, Patel NK, Gauher S, Holloway I, Athanasiou T. High methodologic quality but poor applicability: Assessment of the AAOS guidelines using the AGREE II instrument. Clin Orthop Relat Res 2014;472:1982-8.
14.	Siering U, Eikermann M, Hausner E, Hoffmann-Eßer W, Neugebauer EA. Appraisal tools for clinical practice guidelines: A systematic review. PLoS One 2013;8:e82915.
15.	Semlitsch T, Blank WA, Kopp IB, Siering U, Siebenhofer A. Evaluating guidelines: A review of key quality criteria. Dtsch Arztebl Int 2015;112:471-8.
16.	Hoffmann-Eßer W, Siering U, Neugebauer EA, Brockhaus AC, Lampert U, Eikermann M, et al. Guideline appraisal with AGREE II: Systematic review of the current evidence on how users handle the 2 overall assessments. PLoS One 2017;12:e0174831.
17.	Brosseau L, Rahman P, Poitras S, Toupin-April K, Paterson G, Smith C, et al. Asystematic critical appraisal of non-pharmacological management of rheumatoid arthritis with appraisal of guidelines for research and evaluation II. PLoS One 2014;9:e95369.
18.	Al-Ansary LA, Alkhenizan A. Towards evidence-based clinical practice guidelines in Saudi Arabia. Saudi Med J 2004;25:1555-8.
19.	Acuña-Izcaray A, Sánchez-Angarita E, Plaza V, Rodrigo G, de Oca MM, Gich I, et al. Quality assessment of asthma clinical practice guidelines: A systematic appraisal. Chest 2013;144:390-7.
20.	Cecamore C, Savino A, Salvatore R, Cafarotti A, Pelliccia P, Mohn A, et al. Clinical practice guidelines: What they are, why we need them and how they should be developed through rigorous evaluation. Eur J Pediatr 2011;170:831-6.
21.	Alonso-Coello P, Irfan A, Solà I, Gich I, Delgado-Noguera M, Rigau D, et al. The quality of clinical practice guidelines over the last two decades: A systematic review of guideline appraisal studies. Qual Saf Health Care 2010;19:e58.
22.	Lytras T, Bonovas S, Chronis C, Konstantinidis AK, Kopsachilis F, Papamichail DP, et al. Occupational Asthma guidelines: A systematic quality appraisal using the AGREE II instrument. Occup Environ Med 2014;71:81-6.