Rationale: Methods to systematically measure the impact of knowledge resources on health professionals would enhance evaluation of these resources in the real world.
Objective: To propose a new impact assessment method.
Background: We demonstrated the feasibility of combining a 4-level scale with Computerized Ecological Momentary Assessment (CEMA) for efficiently measuring the impact of a knowledge resource.
Method: We critically reviewed the world literature regarding the impact of clinical information-retrieval technology on trainees and doctors, and retained 26 papers for qualitative content analysis.
Findings: Of those, 21 use a nominal scale (yes/no), none systematically measures the impact of searches for information outside of a laboratory setting, and none uses an ordinal scale. The literature supports the proposed levels of impact, and suggests a fifth level.
Conclusion: A new impact assessment method is proposed, which combines a 5-level revised scale and CEMA.