Traditional fluorescence lifetime imaging (FLIM) provides valuable quantitative insights for biomedical and molecular biology research, but often relies on computationally intensive datafitting methods to extract meaningful metrics. To address this limitation, we propose a hardware-efficient deep learning approach using one-dimensional channel attention convolutional neural networks (1D CANNs) to process FLIM data with high efficiency and speed. The 1D CANN offers several advantages, including reduced computational requirements, the ability to train on smaller datasets, and minimal training times. The superior performance of 1D CANNs in fluorescence lifetime fitting has been validated by using raw time-correlated single-photon counting (TCSPC) data. By utilizing an experimental training dataset, we achieved strong consistency between the predicted lifetime maps of dynamic fluorescence imaging and the ground truth. Moreover, we extended the application of 1D CANNs to phosphorescence lifetime imaging (PLIM), achieving a prediction error within 10%. Beyond fluorescence lifetime fitting, we demonstrated the versatility of 1D CANNs by integrating them with FLIRR (Fluorescence-Lifetime Redox Ratio) to diagnose Alzheimer's disease in mouse brain slices. Additionally, we applied 1D CANNs to STED-FLIM imaging, achieving improved spatial resolution. Our findings highlight the broad potential of 1D CANNs in biomedical and photonics applications, demonstrating their robustness across varying photon count conditions.