Single molecule (SM) detection represents the ultimate limit of chemical detection. Over the years, many experimental techniques have emerged with this capacity. Yet, SM detection and imaging methods produce large spectral data sets that benefit from chemometric methods. In particular, surface enhanced Raman scattering spectroscopy (SERS), with extensive applications in biosensing, is demonstrated to be particularly promising because Raman active molecules can be identified without recognition elements and is capable of SM detection. Yet quantification at ultralow analyte concentrations requiring detection of SM events remains an ongoing challenge, with the few existing methods requiring carefully developed calibration curves that must be redeveloped for each analyte molecule. In this work, we demonstrate that a convolutional neural network (CNN) model when applied to bundles of SERS spectra yields a robust, facile method for concentration quantification down to 10 fM using SM detection events. We further demonstrate that transfer learning, the process of reusing the weights of a trained CNN model, greatly reduces the amount of data required to train CNN models on new analyte molecules. These results point the way for unambiguous analysis of large spectral data sets and the use of SERS in important ultra low concentration chemical detection applications such as metabolomic profiling, water quality evaluation, and fundamental research.