Compared to free chemotherapeutic drugs, nano-sized drug delivery systems exhibit enhanced therapeutic effects and reduced in vivo toxicity. Peptide-based drug delivery systems have garnered significant attention due to the advantageous properties of peptides, including their excellent biocompatibility, diverse side-chain functionalities, and ability to form stable secondary structures. Incorporating stimuli-responsive amino acid residues or specific responsive moieties within their side chains endows these peptide-based drug delivery systems with unique stimuli-responsive characteristics. In this review, we summarize recent advancements and mechanisms in peptide-based nanodrug delivery systems that are capable of responding to one or multiple stimuli as well as conclude with a concise overview of the challenges that lie ahead in this field.