Ultrafast mid-infrared (mid-IR) sources are instrumental in the analysis of material structure and composition. Mid-IR spectroscopy based on nanophotonic waveguides has garnered significant interest due to its compactness, portability, and efficiency. However, conventional on-chip mid-infrared sources have predominantly focused on generation efficiency and spectral bandwidth, often overlooking the stability of carrier-envelope offset (CEO). In this work, we employ self-phase modulation and difference frequency generation to generate an offset-free mid-IR frequency comb within thin-film lithium niobate nanophotonic waveguides on sapphire. Our design and analysis show that output energies in the tens of picojoules across a wavelength range of 3.2-4.72 μm can be achieved in this device. The resulting mid-IR comb, with fceo automatically set at 0, exhibits the stability of CEO, thereby enhancing the accuracy of mid-infrared spectral detection and offering what we believe to be a novel light source for experiments with stringent fceo sensitivity requirements. Our design is characterized by high controllability, flexibility, and integrability, making it conducive to the development of miniaturized, high-precision mid-infrared sources.