Investigations of organic lithic micro-residues have, over the last decade, shifted from entirely morphological observations using visible-light microscopy to compositional ones using scanning electron microscopy and Fourier-transform infrared microspectroscopy, providing a seemingly objective chemical basis for residue identifications. Contamination, though, remains a problem that can affect these results. Modern contaminants, accumulated during the post-excavation lives of artifacts, are pervasive, subtle, and even "invisible" (unlisted ingredients in common lab products). Ancient contamination is a second issue. The aim of residue analysis is to recognize residues related to use, but other types of residues can also accumulate on artifacts. Caves are subject to various taphonomic forces and organic inputs, and use-related residues can degrade into secondary compounds. This organic "background noise" must be taken into consideration. Here we show that residue contamination is more pervasive than is often appreciated, as revealed by our studies of Middle Palaeolithic artifacts from two sites: Lusakert Cave 1 in Armenia and Crvena Stijena in Montenegro. First, we explain how artifacts from Lusakert Cave 1, despite being handled following specialized protocols, were tainted by a modern-day contaminant from an unanticipated source: a release agent used inside the zip-top bags that are ubiquitous in the field and lab. Second, we document that, when non-artifact "controls" are studied alongside artifacts from Crvena Stijena, comparisons reveal that organic residues are adhered to both, indicating that they are prevalent throughout the sediments and not necessarily related to use. We provide suggestions for reducing contamination and increasing the reliability of residue studies. Ultimately, we propose that archaeologists working in the field of residue studies must start with the null hypothesis that miniscule organic residues reflect contamination, either ancient or modern, and systematically proceed to rule out all possible contaminants before interpreting them as evidence of an artifact's use in the distant past.