You are here

Article Text

Article menu

Download PDFPDF

Research ethics
Paper
Are there adverse consequences of quizzing during informed consent for HIV research?
  1. J Sugarman1,2,
  2. A Corneli3,
  3. D Donnell4,
  4. T Y Liu4,
  5. S Rose3,
  6. D Celentano5,
  7. B Jackson6,
  8. A Aramrattana7,
  9. L Wei8,
  10. Y Shao9,
  11. F Liping10,
  12. R Baoling10,
  13. B Dye3,
  14. D Metzger11
  1. 1Berman Institute of Bioethics, Johns Hopkins University, Baltimore, Maryland, USA
  2. 2Department of Medicine, Johns Hopkins School of Medicine, Baltimore, Maryland, USA
  3. 3FHI, Research Triangle Park, Durham, North Carolina, USA
  4. 4SCHARP Vaccine and Infectious Disease Division, Fred Hutchison Cancer Research Center, Seattle, Washington, USA
  5. 5Department of Epidemiology, Johns Hopkins Bloomberg School of Public Health, Baltimore, Maryland USA
  6. 6Department of Pathology, Johns Hopkins School of Medicine, Baltimore, Maryland USA
  7. 7Research Institute for Health Sciences, Chiang Mai University, Thailand
  8. 8Guangxi Centers for Disease Control and Prevention, Nanning, China
  9. 9National Center for AIDS/STD Control and Prevention, Beijing, China
  10. 10Xinjiang Uighur Autonomous Region Centers for Disease Control and Prevention, Urumqi, China
  11. 11Department of Psychiatry, University of Pennsylvania, Philadelphia, Pennsylvania, USA
  1. Correspondence to J Sugarman, Johns Hopkins Berman Institute of Bioethics, 1809 Ashland Avenue, Deering Hall 203, Baltimore, MD 21205, USA; jsugarman{at}jhu.edu

Abstract

Introduction While quizzing during informed consent for research to ensure understanding has become commonplace, it is unclear whether the quizzing itself is problematic for potential participants. In this study, we address this issue in a multinational HIV prevention research trial enrolling injection drug users in China and Thailand.

Methods Enrolment procedures included an informed consent comprehension quiz. An informed consent survey followed.

Results 525 participants completed the informed consent survey (Heng County, China=255, Xinjiang, China=229, Chiang Mai, Thailand=41). Mean age was 33 and mean educational level was 8 yrs. While quizzing was felt to be a good way to determine if a person understands the nature of clinical trial participation (97%) and participants did not generally find the quiz to be problematic, minorities of respondents felt pressured (6%); anxious (5%); bored (5%); minded (5%); and did not find the questions easy (13%). In multivariate analysis, lower educational level was associated with not minding the quizzing (6–10 yrs vs 0–5 yrs: OR=0.27, p=0.03; more than 11 yrs vs 0–5 yrs: OR=0.18, p=0.03). There were also site differences (Heng County vs Xinjiang) in feeling anxious (OR=0.07; p=<0.01), not minding (OR=0.26; p=0.03), being bored (OR=0.25; p=0.01) and not finding the questions easy (OR=0.10; p=<0.01).

Conclusions Quizzing during the informed consent process can be problematic for a minority of participants. These problems may be associated with the setting in which research takes place and educational level. Further research is needed to develop, test and implement alternative methods of ensuring comprehension of informed consent.

Trial Registration clinicaltrials.gov number NCT00270257.

  • Informed consent
  • research ethics
  • ethics
  • attitudes
  • HIV infection and aids
  • substance abusers/users of controlled substances
  • policy guidelines/inst. review boards/review cttes

https://doi.org/10.1136/jme.2011.042358

Statistics from Altmetric.com

    Request Permissions

    If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

    Introduction

    Informed consent is intended to protect the rights and interests of research participants and is a fundamental ethical requirement of clinical research.1–4 Studies on informed consent, however, have consistently shown that study participants often do not fully understand the requirements of the research study in which they are enrolled.5–13 When research poses substantial risk to participants or there may be some question about the capacity of potential participants to provide informed consent, methods to assess potential participants' comprehension prior to study enrolment, such as quizzes, have been incorporated into the informed consent process. Such methods are now commonplace for certain types of clinical research, such as large, multi-site HIV prevention trials14–18 and clinical research involving substance users.19 Yet, participant acceptability and cultural appropriateness of these assessment methods have not been well studied, especially in vulnerable populations who may have had limited formal education, engage in illegal or stigmatised activities (eg, drug use, sex work) or otherwise be unaccustomed to being quizzed.

    Our study intended to address this gap in the literature by assessing participants' opinions about an informed consent comprehension quiz administered as part of a large, multi-site HIV prevention clinical trial and to assess their feelings when completing the quiz.

    Methods

    Our evaluation was embedded within a phase III randomised controlled trial to evaluate the efficacy of different modalities of drug treatment for the prevention of HIV infection and death among opiate dependent injectors. Additional information about the study is available at: http://www.hptn.org/research_studies/hptn058.asp. Study sites included Xinjiang Uighur Autonomous Region Centers for Disease Control and Prevention in Xinjiang, China; Guangxi Zhuang Autonomous Region Centers for Disease Control and Prevention in Guangxi, China; and the Research Institute for Health Sciences in Chiang Mai University in Chiang Mai, Thailand.

    The study was reviewed and approved by the following ethics committees: Chiang Mai University Research Institute for Health Sciences, Thailand; Ministry of Public Health Ethical Review Committee for Research in Human Subjects, Nonthaburi, Thailand; Guangxi Center for Disease Prevention and Control Institutional Review Board (IRB), China; Xinxiang Uighur Autonomous Region Bureau of Health Disease Control and Treatment IRB, China; The Chinese National Center for AIDS/STD Control and Prevention IRB; and Johns Hopkins Medicine IRB #2, USA. Participants provide their informed consent for the informed consent survey (ICS) as part of the enrolment consent process for the trial, although they were given the option to decline participation in the survey when it was offered and still participate in the trial.

    As part of the trial's informed consent and enrolment process, participants must first pass an informed consent comprehension quiz by answering at least nine of 12 questions (75%) correctly within three attempts, prior to signing the consent form. Only participants who passed the informed consent comprehension quiz and gave informed consent to participate in the trial were asked to complete the ICS. Often the ICS was administered immediately following the administration of the comprehension quiz and signing of the consent form, but participants were permitted to complete the ICS at any time during their enrolment study visit. One study staff member administered the informed consent comprehension quiz and a different study staff member administered the ICS to reduce bias associated with socially desirable responses. The ICS was always administered orally.

    Questions on the ICS were informed by educational literature on test taking20 21 and included six questions that asked participants to agree or disagree using a five-point Likert scale (ie, strongly disagree, disagree, neither disagree or agree, agree, strongly agree) on feelings they experienced when answering the informed consent comprehension quiz, such as being anxious or annoyed. Two additional questions assess participants' opinions on whether quizzing is an appropriate method to measure comprehension among potential study participants. Several rounds of field-testing were conducted with study staff and members of the community advisory boards in two sites (Chiang Mai and Heng County) to ensure that the all items and response categories were culturally and linguistically appropriate across the sites. The ICS was translated into the local languages and back-translated to confirm questions on the translated versions and the English version all reflected the same meaning. The final items included in the ICS are listed in table 1 and the ICS instrument is provided in appendix 1 (available online only at http://jme.bmj.com). The Chiang Mai site obtained the necessary regulatory approvals to implement this evaluation at a much later date, and enrolled far fewer participants than the sites in China.

    View this table:
    Table 1

    Informed consent evaluation survey items (English version)

    Data were analysed using standard descriptive and inferential statistics for ordered and unordered variables. Factor analyses were then conducted.

    For each attitude question, the five-point Likert scale was collapsed to positive and negative attitudes (table 1). Logistic regression was used to compute the ORs of a negative attitude for each question for site, gender, age and education. Significant covariates (p value <0.05) were retained in a multivariate logistic regression model.

    Factor analysis was used to define combinations of the attitude questions that had commonality across the participants. Specifically, factor analysis with varimax rotation was applied to the eight attitude questions, after mapping the Likert scale using numeric scores of −2, −1, 0, 1 and 2, with positive attitudes mapped to positive scores. A scree plot test was examined to guide the selection of factors capturing most of the variance in attitude. The resulting selected factors combine the original attitude questions using factor loadings, so questions with high (absolute) factor loadings are interpreted as defining a given factor's characteristics.

    Results

    In all, 587 participants were asked if they were willing to complete the ICS and 525 completed it (41 from Chiang Mai, 255 from Heng County, and 229 from Xinjiang). The response rates for the ICS were 100% in both Chiang Mai (41/41) and Heng County (255/255), and 81% (229/282) in Xinjiang. Overall, the mean age was 33; 7% were female; mean educational level was 8 yrs (table 2). No significant differences in demographic characteristics (age, gender and educational level) were found between those who did and did not complete the ICS in Xinjiang.

    View this table:
    Table 2

    Demographic characteristics of respondents

    While quizzing was felt to be a good way to find out if a person understands the clinical trial (97%) and was not problematic for the majority of participants, minorities of respondents felt pressured (6%); anxious (5%); bored (5%); minded being quizzed (5%); or didn't find the questions easy (13%) (table 3).

    View this table:
    Table 3

    Attitudes about the informed consent questionnaire

    In multivariate analysis, there were site differences (Heng County vs Xinjiang) in feeling anxious (OR=0.07; p=<0.01), not minding the quizzing (OR=0.26; p=0.03), being bored (OR=0.25; p=0.01), and not finding the questions easy (OR=0.10; p=<0.01). Lower educational level was associated with not minding the quizzing (6–10 yrs vs 0–5 yrs: OR=0.27, p=0.03; >11 yrs vs 0–5 yrs: OR=0.18, p=0.03) (table 4).

    View this table:
    Table 4

    Association between attitudes and site, age and education

    The loadings of the first two factors (table 5) define a first factor by attitudes related to emotions about quizzing (high loadings on the first four questions), and the second factor related to the content of the quiz (high loadings on the last two questions).

    View this table:
    Table 5

    Factor analysis

    Discussion

    While quizzing during the informed consent process is aimed at helping to ensure that potential research participants have adequate understanding of the proposed research, and participants in general believe that quizzing is a good way to assess understanding, it can be problematic for some participants.

    In a multivariate analysis, those with more education were more likely to mind being quizzed. Perhaps, those with more education view quizzing in such a situation to be excessive or insulting. Alternatively, they might associate quizzing with stressful experiences in school.

    In addition, in this study the extent to which quizzing is problematic relates at least in part to the setting in which the research takes place. In multivariate analyses, there were significant differences among the two sites in China (which each have substantially more respondents than the site in Thailand). Participants in Heng County compared to those in Xinjiang were more likely to report that they minded answering the quiz and not finding the questions easy; yet they were less likely to be bored or anxious due to quizzing. It is conceivable that these findings are related to a variety of factors such as culture, ethnicity, and the urban (Xinjiang) versus rural (Heng County) location of the sites.

    Of course, these findings should be interpreted with several limitations in mind. First, respondents' reports about quizzing during the informed consent process may be subject to a social response bias, in which respondents are inclined to report that the quizzing was not problematic. Such bias would be understandable since the participants have just given informed consent to participate in the parent trial and may want to please the research team by not ‘complaining’. In order to minimise the likelihood of such a bias, in this study separate study staff members administered the ICS from those who did the quizzing itself. Related, those participants who completed the ICS all passed the quiz, which may have influenced their evaluation of it. Also, the small number of participants in Chiang Mai limits our power to detect anything other than very substantial site differences in attitudes in Thailand compared to the China sites. While the response rate was high in Chiang Mai and Heng county, as many as 20% refused to participate in the ICS in Xinjiang, raising the potential for non-response bias in this site, although no differences were observed in the demographic characteristics among responders and non-responders. Despite these possibilities, this analysis indicates variability in attitudes towards quizzing, suggesting that the instrument was able to overcome these limitations. In addition, although we have data from a large number of participants and the response rate was high, it is important to remember that our data derive from a single parent study taking place in three sites in two countries that includes quizzing about the particularities of that trial. Therefore, it would not be surprising if findings from a similar evaluation would differ if it were to be conducted in other geographic regions and cultures or for another trial. While it is conceivable that quizzing during the informed consent process may be more or less problematic in other instances, such variability alone would be an interesting finding. Moreover, it would be arguably important to identify trials and sites where quizzing would be especially problematic so that alternative means of assessing comprehension might be used.

    Accordingly, further research is needed to assess satisfaction with quizzing during the informed consent process in other studies and in different geographic regions. Fortunately, as we have demonstrated in this study, evaluating attitudes towards quizzing during the informed consent process can be relatively easily implemented in actual clinical trials using standard clinical trial procedures. Others wishing to evaluate the process could elect to use the ICS we developed, perhaps eliminating those items which did not add information based on the factor analysis described above. Regardless, careful attention should be paid to ensure that the items translate and back-translate properly.

    Nevertheless, concerted effort should be directed at ensuring comprehension during the informed consent process. While at first glance quizzing seems to provide a relatively easy means of appearing to fulfil this objective, there remains a surprisingly difficult set of questions which remain regarding quizzing; these include what constitutes adequate understanding, what aspects of the study are important to understand, and what effects will this quizzing have on potential participants, including their attitudes, willingness to participate, trust and adherence to study requirements. These issues warrant rigorous attention, both conceptually and empirically. In the meantime, formative research should be conducted in settings in which quizzing is being proposed to evaluate comprehension during the informed consent process so that alternative means for ensuring comprehension might be developed and implemented in settings where quizzing may be problematic.

    References

    1. ↵
      Council for International Organization of Medical Sciences (CIOMS). International Ethical Guidelines for Biomedical Research Involving Human Subject. Geneva: CIOMS/WHO, 2002.
    2. ↵
      National Commission for the Protection of Human Subjects of Biomedical and Behavioral Research. Belmont Report: Ethical Principles and Guidelines for the Protection of Human Subjects of Research. Washington, DC: U.S. Government Printing Office, 1979.
    3. ↵
      World Medical Association. Declaration of Helsinki: Ethical Principles for Medical Research Involving Human Subjects. Seoul: World Medical Association, 2008.
    4. ↵
      International Conference on Harmonization (ICH). Guideline for Good Clinical Practices. http://www.ich.org/products/guidelines/efficacy/article/efficacy-guidelines.html (accessed 21 May 2011).
    5. ↵
      1. Taiwo OO,
      2. Kass N
      . Post-consent assessment of dental subject's understanding of informed consent in oral health research in Nigeria. BMC Med Ethics 2009;10:11.
      OpenUrlCrossRefPubMed
    6. ↵
      1. Krosin MT,
      2. Klitzman R,
      3. Levin B,
      4. et al
      . Problems in comprehension of informed consent in rural and peri-urban Mali, West Africa. Clin Trials 2006;3:302–13.
      OpenUrl
    7. ↵
      1. Breese PE,
      2. Burman WJ,
      3. Goldberg S,
      4. et al
      . Education level, primary language, and comprehension of the informed consent process. J Empir Res Hum Res Ethics 2007;2:69–79.
      OpenUrlFREE Full Text
    8. ↵
      1. Lynöe N,
      2. Hyder Z,
      3. Chowdhury M,
      4. et al
      . Obtaining informed consent in Bangladesh. N Engl J Med 2001;344:460–1.
      OpenUrlCrossRefPubMedWeb of Science
    9. ↵
      1. Joffe S,
      2. Cook EF,
      3. Cleary PD,
      4. et al
      . Quality of informed consent in cancer clinical trials: a cross-sectional survey. Lancet 2001;358:1772–7.
      OpenUrlCrossRefPubMedWeb of Science
    10. ↵
      1. Daugherty C,
      2. Ratain MJ,
      3. Grochowski E,
      4. et al
      . Perceptions of cancer patients and their physicians involved in phase I trials. J Clin Oncol 1995;13:1062–72.
      OpenUrlAbstract
    11. ↵
      1. Sugarman J,
      2. McCrory DC,
      3. Powell D,
      4. et al
      . Empirical research on informed consent. Hastings Cent Rep 1999;29:S1–42.
      OpenUrlPubMed
    12. ↵
      1. Leach A,
      2. Hilton S,
      3. Greenwood BM,
      4. et al
      . An evaluation of the informed consent procedure used during a trial of a Haemophilus influenzae type B conjugate vaccine undertaken in The Gambia, West Africa. Soc Sci Med 1999;48:139–48.
      OpenUrlCrossRefPubMedWeb of Science
    13. ↵
      1. Riecken HW,
      2. Ravich R
      . Informed consent to biomedical research in Veterans Administration Hospitals. JAMA 1982;248:344–8.
      OpenUrlCrossRefPubMedWeb of Science
    14. ↵
      1. Vallely A,
      2. Lees S,
      3. Shagi C,
      4. et al
      . How informed is consent in vulnerable populations? Experience using a continuous consent process during the MDP301 vaginal microbicide trial in Mwanza, Tanzania. BMC Med Ethics 2010;11:10.
      OpenUrlCrossRefPubMed
    15. ↵
      1. Corneli A,
      2. McKenna K,
      3. Agot K,
      4. et al
      . Enhancing trial participants'understanding of informed consent for a phase 3 clinical trial on pre-exposure prophylaxis for HIV prevention. Abstract #THPE0534. XVIII International AIDS Conference. Vienna, Austria, 18–23 July 2010. http://www.iasociety.org/Default.aspx?pageId=11&abstractId=200738616 (accessed 21 May 2011).
    16. ↵
      1. Coletti A,
      2. Kelly C,
      3. Coumi N,
      4. et al
      . Ongoing informed consent comprehension assessment: results of a pilot conducted in HPTN 035. Abstract #349. Pittsburgh, Pennsylvania: Microbicide, 22–25 May 2010. http://www.mtnstopshiv.org/sites/default/files/attachments/M2010%20Abstracts-11-23-2010_0.pdf (accessed 21 May 2011).
    17. ↵
      MTN-003, The VOICE Study—Vaginal and Oral Interventions to Control the Epidemic. http://www.mtnstopshiv.org/news/studies/mtn003 (accessed 29 Sep 2010).
    18. ↵
      1. Maarschalk S,
      2. Frohlich J,
      3. Ntombela F,
      4. et al
      . Structured tools for assessing for assisting literacy levels and comprehension assessment in the informed consent process; experiences from a microbicide trial in rural kwazulu natal, South Africa. Pittsburgh, Pennsylvania: Microbicide, 22–25 May 2010. Abstract #281. http://www.mtnstopshiv.org/sites/default/files/attachments/M2010%20Abstracts-11-23-2010_0.pdf (accessed 21 May 2011).
    19. ↵
      HPTN 058: A Phase III Randomized Controlled Trial to Evaluate the Efficacy of Drug Treatment in Prevention of HIV Infection and Death Among Opiate Dependent Injectors. http://www.hptn.org/research_studies/hptn058.asp (accessed 29 Sep 2010).
    20. ↵
      1. Fiske DW
      . The subject reacts to tests. Am Psychol 1967;22:287–96.
      OpenUrlCrossRefPubMed
    21. ↵
      1. Lounsbury JW,
      2. Bobrow W,
      3. Jensen JB
      . Attitudes toward employment testing: scale development, correlates, and “known-group” validation. Prof Psychol Res Pract 1989;20:340–9.
      OpenUrlCrossRefWeb of Science

    Footnotes

    • Disclaimer The content is solely the responsibility of the authors and does not necessarily represent the official views of the HPTN, NIAID, or NIMH.

    • Study design: Cross-sectional survey embedded within a clinical trial.

    • Funding National Institute of Allergy and Infectious Diseases (NIAID), the National Institute on Drug Abuse (NIDA) and the National Institute of Mental Health (NIMH) Other Funders: NIH. JS, AC, SR, BD, DD, TYL, DC, BJ, AA, LW, YS, FL, BR and DM had full access to all of the data and have support from the HIV Prevention Trials Network (HPTN) under award numbers U01 AI068619 and U01 AI 069482 for the submitted work. The study funders were not involved in the study design, analysis, or writing of this manuscript.

    • Competing interests None.

    • Ethics approval This study was conducted with the approval of the Chiang Mai University Research Institute for Health Sciences (RIHES), Thailand; Ministry of Public Health Ethical Review Committee for Research in Human Subjects (MOPH), Nonthaburi, Thailand; Guangxi Center for Disease Prevention and Control Institutional Review Board (IRB), China; Xinxiang Uighur Autonomous Region Bureau of Health Disease Control and Treatment IRB, China; The Chinese National Center for AIDS/STD Control and Prevention IRB; and Johns Hopkins Medicine IRB #2, US.

    • Provenance and peer review Not commissioned; externally peer reviewed.

    Other content recommended for you