Background Researchers worry that patients in early-phase research experience unrealistic optimism about benefits and risks of participation. The standard measure of unrealistic optimism is the Comparative Risk/Benefit Assessment (CRBA) questionnaire, which asks people to estimate their chances of an outcome relative to others in similar situations. Such a comparative framework may not be a natural way for research participants to think about their chances.

Objective To examine how people interpret questions measuring unrealistic optimism and how their interpretations are associated with their responses.

Methods Using an early-phase cancer trial vignette, we administered the CRBA to 297 adults from the general public. They estimated their comparative chances of risk and benefit (7-point scale: −3 less likely to +3 more likely), then provided rationales for their estimates.

Results For both CRBA benefit and risk questions, about 50% of respondents chose 0 (the ‘correct’ response of ‘average likelihood’), and 50% chose a non-0 response. Respondents’ rationales for their estimates showed that overall only about 40%–44% gave comparative rationales, indicating that they interpreted the CRBA as intended. 68.7% of respondents who gave the ‘correct’ 0 rating gave comparative rationales, whereas only 11.6% of respondents who gave non-0 ratings did so. A similar trend was seen for chances of risk (p<0.001 for both).

Conclusion Research participants may not understand comparative benefit and risk questions as intended; attributions of unrealistic optimism may require additional evidence that the respondents’ estimates are intended to be comparative.