Addressing the global malnutrition crisis requires precise and timely diagnostics of plant stresses to enhance the quality and yield of nutrient-rich crops, such as tomatoes. Soft wearable sensors offer a promising approach by continuously monitoring plant physiology. However, challenges remain in identifying direct physiological indicators of plant stresses, hindering the development of accurate diagnostic models for predicting symptom progression. Here, we introduce a machine-learning-powered spectral-dominant multimodal soft wearable system (MapS-Wear) for precise, long-term, and early-stage diagnosis of stresses in tomatoes. MapS-Wear continuously tracks leaf surrounding temperature, humidity, and unique in-situ transmission spectra, which are critical stress-related indicators. The machine learning framework processes these multimodal data to predict gradual stress progression and diagnose nutrient deficiencies in plants over 10 days earlier than conventional computer vision methods. Moreover, MapS-Wears enables portable and large-scale screening of grafted tomato varieties in greenhouses, accelerating the identification of compatible grafting combinations. This demonstration highlights the potential for high-throughput plant phenotyping and yield improvement.