Designing and Psychometric Properties of Appraisal Adherence to Care and Treatment Scale in Liver Transplantation Recipients

1. Background

Liver transplantation in patients with end-stage liver disease is a widely accepted successful treatment modality. Since the current technology is unable to replace liver function, this treatment method has been welcomed (1). Annually, more than 30,000 liver transplants are performed worldwide, and a one-year survival rate of 90% has been reported (2). However, the long-term survival outcome of organ recipients has not improved much (3). Malignancies, infections, cardiovascular complications, kidney failure, and metabolic syndrome are the major modifiable causes of death in patients after liver transplantation in the long run (4-6). The effectiveness of different therapies not only depends on the appropriate selection of treatment but also the active cooperation of patients in terms of adherence to the care and treatment process. Adherence to care and treatment is a crucial factor that influence successful organ transplantation. In addition, patients' adherence still remains a challenge for healthcare providers. Therefore, the promotion of adherence by patients and accurate assessment of methods are required to improve survival outcomes. An important consideration after liver transplantation is adherence to care and treatment for achieving desirable long-term results (7, 8). Except for drug use, weight control, monitoring the health condition and infection prevention influence transplantation results. It is necessary to assess adherence to care and treatment in liver transplantation recipients (4, 9, 10).

Direct and indirect methods are commonly used to measure patients’ adherence after organ transplantation. In direct methods, monitoring use of immunosuppressive agents and measurement of serum drug levels and in indirect methods self-report tools have been used in previous studies. In current instruments, patient's adherence is assessed based on how much and how immunosuppressive drugs are consumed (11, 12).

The measurement of adherence has been defined as the assessment of immunosuppressive agents used in the past. The concept of adherence has been defined as correspondence between recipients' behaviors and suggestions provided by the healthcare caregivers, taking medicine, following a suitable diet and a change of lifestyle by the World Health Organization (WHO) (13). The complexity of other aspects of adherence has been challenged since other dimensions require a focus on individuals and different cultural, health and care settings in different societies (14, 15). A comprehensive literature review has shown that designing and assessment of psychometric properties of tools for evaluation of adherence to care and treatment in liver transplantation recipients are required to overcome non-adherence as a caring priority (16). Addressing and evaluating this concept, especially in liver transplantation recipients that have high levels of psychological, behavioral, therapeutic and medical characteristics are of particular importance (17).

Since the meaning and factors influencing any phenomenon are influenced by the socio-cultural context, qualitative research methods are needed to provide a more accurate definition of the study concept (18). In addition, qualitative research is the first step for the development of instruments (19).

2. Objectives

Owing to a lack of care and treatment adherence instrument in liver transplantation recipients', we aimed to design a related validated instrument using the perspective of these patients.

3. Methods

An exploratory sequential mixed-method study (20) was done from May 2017 to late September 2019, in the Clinic of Liver Transplantation related to Tehran University of Medical Sciences.

3.1. Qualitative Phase

A qualitative study using an inductive conventional approach was used to explore concept of adherence to care and treatment based on understandings and experiences of patients with liver transplantation. Therefore, 18 recipients were selected based on the maximum variation using the purposive sampling method. Also, a snowball sampling strategy was used for selecting potential key informants, which included two physicians, one coordinator's nurse, and two spouses of liver transplants recipients from May to November 2017. Sampling was done based on these criteria: (1) those who had at least lived with liver transplantation upper of three-month; (2) had over 18 years old, did not have another disease. The time of living with transplanted liver, age, diseases leading to transplantation, and gender were considered for variation in sampling.

The latest author held the face-to-face interviews in the consultation room of the clinic of liver transplantation. By reviewing the related studies and after performing three pilot interviews, researcher designed a semi-structured interview guide. At first, they were asked to describe their experiences of daily life after liver transplantation, and how they protected transplanted liver or how did you adapt your work and life to the new conditions? Please more explain? In addition, the participants answered probing questions for the pursuit to increase the depth of interviews. The interviews lasted for between 20 and 70 minutes and were discontinued until data saturation was achieved. After 20 interview sessions, data saturation was reached, but three interviews were performed to ensure that no new codes and categories were extracted from data (21). Conventional content analysis was performed by Elo and Kinkas method and software MAXQD 10 was used to manage the data.

3.2. Quantitative Phase: Psychometric Properties of Appraisal Adherence to Care and Treatment Scale

3.2.1. Face Validity

Face validity was carried out using qualitative and quantitative methods. The cognitive interviews with 10 liver transplantation recipients were performed to evaluate qualitative face validity. We need to acquire their perception about items regarding the relevancy, difficulty, and ambiguity of each item. Then related modifications were applied to some items. For quantitative part of face validity, the item impact scores with the formula of Impact score = frequency % × important for each item was calculated and subsequently, the items above 1.5 score were remained (22).

3.2.2. Content Validity

The content validity was the second step done by a panel of experts consisting of 14 experts in the field of Persian literature, questionnaire designer, and liver transplantation specialist. They presented their opinions in terms of wording, grammar, item allocation, scaling, clarity, and Simplicity indices, also their comments were included (23). The quantitative content validity was performed by calculation of the content validity ratio (CVR strict), content validity index [Average of the I-CVIs for all items (S-CVI/Ave) and Item Individual Content Validity Index (I-CVI)] and modified Kappa (K*) (24). The expert panel consisted of 10 physicians and nurses as the members of liver transplantation team, post-operation coordinators, as well as methodologists. In this stage, the level of essentiality of each item was important.

The opinions of specialists were categorized as E, essential; U, useful, but not essential and N, not essential. According to Lawshe's formula when there are 10 experts, the items are considered appropriate, which have a 0.62 or higher CVR value (25). Next, for each item, the I-CVI (item level CVI) was calculated (26). Therefore, the experts determined the relevancy of each item by choosing on a 4-point Likert scale from 1 = not relevant, 2 = somewhat relevant, 3 = quite relevant, and 4 = very relevant. The next step was to compute the S-CVI/Ave. For computing that item, we needed to calculate the I-CVI for each item on the total scale, and the mean of I-CVI for total items. The S-CVI/Ave should be 0.9 as the appropriate level (24). The probability of chance of agreement was calculated by calculating k.

Assessment criteria for kappa was the use of guidelines described by Cicchetti; therefore, if K was > 0.74, it is considered excellent (27-29). A stage item analysis was performed before construct validity. Thirty-six liver transplantation recipients were asked to fill out the scale. The Cronbach’s alpha coefficient for each item and the total scale, and correlations between items and the total scale were calculated (30).

3.2.3. Construct Validity

The exploratory factor analyses (27), confirmatory factor analyses (CFA), and convergent validity were used to assess the scale construct validity (29). The standard criteria to determine the minimum sample size for factor analysis are 5 - 10 times more than the number of items (31). Demographic and clinical characteristics were reported in Table 1. The participants were selected using the convenient sampling method from liver transplantation centers in Shiraz and Tehran. Therefore, 287 recipients filled out Appraisal Adherence to Care and Treatment scale during final data analysis. The CFA was conducted with 200 recipients selected using a random sampling method. Data were entered into the SPSS software and missing values were substituted by the median score of the Likert scale. The Maximum likelihood analysis was performed with Promax rotation. To investigate the appropriateness of the factor analysis model, the number of factors, and sampling adequacy Bartlett’s test of sphericity, the scree plot, the Kaiser-Meyer-Olkin (KMO) test, eigenvalues and determined variance were used. The minimum factor load of 0.3 was considered to maintain items in extracted factors.

3.3. Convergent Validity Evaluation

The congruence between the extracted main factors and the scale was also evaluated. Convergent validity with a correlation between Morisky DE, Green LW, Levine DM (MGL) scale and appraisal adherence to care and treatment scale was calculated (32). The correlation between MGL and appraisal adherence to care and treatment scale should be 0.4 - 0.7 (33).

3.4. Relative Reliability Evaluation

For reliability, the final version of appraisal adherence to care and treatment scale with 20 items was assessed in terms of internal consistency and test–retest. Hence, the alpha Cronbach coefficient was calculated to evaluate this. An alpha Cronbach of 0.7 - 0.8 confirms an appropriate internal consistency (31). To investigate stability of the scale over time was used the test–retest method. Accordingly, 25 liver transplantation recipients completed this scale twice with a 3-week interval. The intraclass correlation coefficient score (ICC) of 0.8 or higher denoted satisfactory stability (34).

3.5. Absolute Reliability Evaluation

Since ICC provides no precise data about the accuracy of scores, as the matter of fact absolute reliability was calculated by estimating standard error of measurement (SEM) [that calculated by SD × √ (1-ICC agreement)]. The minimal detectable change (MDC) was estimated from SEM and was defined as the smallest amount of change that likely shows a true change rather than an intrinsic measurement fault in the score. One advantage of the MDC was that it considered both reliability and responsiveness to changes (35).

The sensitivity of appraisal adherence to care and treatment scale was measured using the hypothesis test. There was a correlation between the time of receiving liver transplantation and adherence to care and treatment scores. Feasibility was assessed based on the time duration of filling out the scale and number of missing items. The ceiling and floor effects were assessed with minimum and maximum response of distribution of scores. The standardization of scores was performed as the total score was changed to a 0 - 100 score using the underneath formula (36).

Transformed score = [(Actual raw score - Lowest possible raw score) / Possible raw score range] × 100

3.6. Statistical Analysis

Data were tested for sampling adequacy using the KMO and Bartlett test. Exploratory factor analysis, descriptive statistical tests, one-way analysis of variance (ANOVA) test, Cronbach’s alpha, intraclass correlation coefficient (27), and Pearson correlation were calculated with the SPSS v.20 software and Lisrel program 8.8.

3.7. Ethical Considerations

Our study was carried out according to the ethics declaration of the revised Helsinki 2001, with the code REC. IR.UMSHA.1396.109. All participants were informed about the aims of the study and the voluntary nature of their participation. Additionally, they were informed that they could withdraw at any time of the study. Then, written informed consent was signed and achieved.

4. Results

4.1. Item Generation: Qualitative Phase

In the first phase, four categories “health-self management”, “health literacy promotion”, “personal and social facilitator”, “commitment to treatment and care” were developed to describe the concept of adherence to care and treatment. They included all behaviors by the recipients to achieve self-management in health, promote health literacy, facilitate personal and social aspects, and remain committed to care and treatment as adherence to care and treatment. An item pool was generated using the categories and their sub-categories based on their operational definitions. The inductive approach was used as all items evaluated adherence. Also, other scales on adherence were reviewed, but no item was added to the item pool. Next, 65 items were entered into the process of psychometric evaluation. The items were scored from “never” to “always” on a 5-point Likert scale.

4.2. Psychometrics Process: Quantitative Phase

For face validity, none of the items were deleted and an impact score for all items had more than 1.5. In the CVR, 21 items were deleted because items acquired less than 0.62. The I-CVI was calculated based on the K* score and one of the items that had a score of less than 0.60 was deleted. The S-CVI/Ave was reported as 0.95, indicating an excellent score. Before performing construct validity, item analysis was conducted and seven items with a corrected item-total correlation of < 0.2 were deleted. The Cronbach’s alpha coefficient for item analysis was 0.856. In construct validity, exploratory factor analysis on 36 items was performed. The result of the KMO test was 0.902, indicating sampling adequacy. Moreover, a significant interrelationship was confirmed between the items by Bartlett’s test (P-value < 0.0001), indicating the suitable factor analysis model. According to the explained variance, four factors were extracted.

The four-factor solution was identified according to the maximum likelihood EFA and consideration of the conceptual meaning. The total variance was reported as 45.622 and factors 1-4 explained 14.920, 11.855, 10.281, and 8.566% of the observed variance after Promax rotation. The cut-off point 0.3 was determined as a minimum factor load in the extracted factors to keep items. Those items that had cross-loadings or were not loaded on any factor were deleted (Figure 1). Four factors and twenty items were constituted. Interpretation and labeling were conducted as follows: health self-management (10 items), conflict management (3 items), self-regulation (4 items), and conscious adherence (3 items). The factors and its factor loadings are shown in Table 2.

Scree plot extracted in exploratory factor analysis.

Figure 1.

Scree plot extracted in exploratory factor analysis.

The CFA also showed reasonable fitness. The convergent validity was also assessed and the Pearson correlation coefficient between the scores of the MGL and appraisal adherence to care and treatment scale was reported as 0.589 (P < 0.01). The indicators of the confirmatory factor analysis model are shown in Table 3.

The Cronbach’s alpha coefficient for the whole appraisal adherence to care and treatment scale was reported as 0.889 and for the factors, it ranged from 0.711 to 0.814, indicating a good internal consistency. The test-retest reliability using the ICC was reported as 0.912 with a range of 0.852 to 0.952. As shown in Table 4, the ICC score was mainly above 0.80, except for the conscious adherence factor (0.728) that reported excellent.

To evaluate the discriminating power of appraisal adherence to care and treatment scale, it was applied to three known groups of liver transplantation recipients living in different urban, rural, and suburban areas. The Suburb had a lower score of adherence to care and treatment. The one-way ANOVA test revealed a statistically significant difference between the groups in terms of adherence to care and treatment (P < 0.002).

The standard deviation of changes in adherence to care and treatment scores was 6.669, and the ICC was 0.912. Therefore, the calculated SEM was 1.978. The minimal detectable change was 5.48 and MDC% was 5.8 known as an excellent agreement. Then minimum important change was reported as 6.70, so responsiveness was confirmed (Table 5).

The sensitivity level was assessed using the theory test. The negative correlation between post-liver transplantation and adherence to care and treatment was statistically significant. Feasibility was assessed by the time duration for filling of appraisal adherence to care and treatment scale, as was 3 - 10 minutes and the numbers of missing items, which was reported very low (0.175%). The maximum distribution of response scores was reported as 7% and no minimum score was achieved. The standardization of scoring was done and the results of standardization for the total scale and four factors are shown in the following:

Standardization score for total of scale = [(Actual score - 20) / 80] × 100

Based on acquired results, scores closer to 100 indicated a higher level of adherence to treatment and care in liver transplantation recipients.

5. Discussion

Adherence to care and treatment influences the transplantation outcome and long-term survival of patients with liver transplantation. This mixed-method study was used to design and assess an instrument to apprise adherence to care and treatment liver transplantation recipients.

The final version of appraisal adherence to care and treatment scale had 20 items with four factors in Iranian patients with liver transplantation. The factors include health self-management, conflict management, self-regulation, and conscious adherence. The health self-management factor included ten items related to self-care behaviors to manage, prevent or promote potential health conditions after liver transplantation. The self-care factor helped maintain and improve patients’ health. The conflict management factor consisted of three items, indicating conditions and situations in which liver transplantation recipients experienced conflicts. Therefore, they should be controlled by the patients to adhere to care and treatment. The combination of four items made the self-regulation factor that was a necessity to create a change in their life and identity. Such a change needed the acceptance of new conditions after liver transplantation. Eventually, the factor of conscious adherence included three items that came from the main category of health literacy promotion. This factor indicated their knowledge and health literacy. The appraisal adherence to care and treatment scale presented all adherence dimensions defined by the WHO. The measurement of the concept of adherence has a behavioral characteristic rather than a medical, attitude, psychological and other modifiable predictors. Therefore, feelings and attitudes of the patients identified in the qualitative phase were separated from adherence explanatory behaviors.

In the quantitative phase, all COSMIN properties were considered in the appraisal adherence to care and treatment scale (37). In content validity, the S-CVI/Ave was reported that was reported in previous similar scales. In addition, an item analysis prior to construction validity was performed in which it was filled out by 36 persons in the target group. It helped identify items that decreased the Cronbach's alpha and led to correcting items with a total correlation below < 0.2.

The reliability of appraisal adherence to care and treatment scale was found appropriate in this sample of Iranian patients with liver transplantation. ICC score suggested that reliability of new scale was appropriate. As well as it has been a correlation between all items, indicating that no conceptual dispersion was seen and the questionnaire’s items measure similar concepts. The confirmatory factor analysis approved a fitting model as four latent factors were confirmed by accepted items in the exploratory factor analysis (Figure 2). The convergent validity was supported based on the positive correlation between the score of our questionnaires and the MGL. However, the correlation coefficient was not very high, confirming that the designed questionnaire assessed a similar concept in the MGL. In contrast, the lack of a high correlation indicated that two questionnaires did not measure exactly the same concept and did not repeat each other.

Confirmatory factor analysis for adherence to care and treatment in liver transplantation recipients [Notes: H, health self-management (10 items); C, conflict management (3 items); S, self- regulation (4 items); and A, conscious adherence (3 items)].

Figure 2.

Confirmatory factor analysis for adherence to care and treatment in liver transplantation recipients [Notes: H, health self-management (10 items); C, conflict management (3 items); S, self- regulation (4 items); and A, conscious adherence (3 items)].

This questionnaire can be used in studies to detect meaningful changes in adherence to care and treatment. Also, it is sensitive enough to detect any small changes in adherence to care and treatment liver transplantation recipients. The standardization of scoring was conducted to facilitate understanding levels of the appraisal adherence to care and treatment scale.

Dobbels (2010) in a systematic review showed three characteristics illustrating the usability of the instrument for the diagnosis of adherence in the patients with transplantation. They included attention to the timing of drug administration, easy completion, and good psychometric properties (38). The appraisal adherence to care and treatment scale was comprehensive, and no ambiguity was reported by the target group. In addition, the short time period of scale completion makes it a user-friendly scale.

5.1. Limitations

The limited access to liver transplantation recipients and lack of time might have affected the cross-validation sampling for confirmatory factor analysis. Also, the use of a nonrandom sample was another limitation of this study.

5.2. Conclusion

The appraisal adherence to care and treatment scale developed in this study assessed adherence to care and treatment among liver transplantation recipients. It had an appropriate psychometric property. This could be the only and specific instrument to evaluate adherence to care and treatment in liver transplantation recipients in the context of clinical practice. It can be used to identify problems and provide a suitable strategy for improving the outcome of liver transplantation.