Antipsychotic medication is frequently prescribed for off-label use in individuals with intellectual disability and elderly persons, in particular to reduce challenging behaviour. However, clear evidence for effectiveness is scarce and long-term use of antipsychotics is associated with a wide range of serious side-effects, including movement disorders and metabolic syndrome. Therefore, off-label use of antipsychotics in these populations has been widely debated. Nevertheless, non-pharmacological interventions alone are not always effective and a treatment attempt with an antipsychotic is sometimes inevitable. In this commentary, it is argued that clinicians should focus on the decision-making process in prescribing antipsychotics, rather than focus on discontinuation of these medications.