Theories of visual recognition place different emphasis on the role of non-stimulus factors. Previously, we showed that arbitrary semantic associations influenced visual recognition of novel objects. Here, the neural substrate of this effect was investigated. During a visual task, novel objects associated with arbitrary semantic features produced more activation in frontal and parietal cortex than objects associated with names. Because the task required no semantic retrieval, access to semantics appears to be involuntary. The brain regions involved have been implicated in semantic processing, thus recently acquired semantics activate a similar network to semantics learned over a lifetime.