Molecular property prediction with limited data in novel chemical domains remains challenging. We introduce an integrated approach combining nonlinear structure-property space identification with uncertainty quantification for reliable predictions. Our framework uniquely integrates kernel principal component analysis, creating a structure-property embedding, two-factor reliability assessment for identifying in-domain molecules, and distance-based uncertainty quantification. Across ecotoxicity datasets, our approach demonstrated error reductions of ~50% for truly in-domain molecules compared to global models. The two-factor reliability assessment effectively distinguishes in-domain from out-of-domain molecules with higher precision than single-method approaches. We establish a strong correlation between structure-property space distance and prediction error, validating our theoretical framework. In a practical bio-lubricant property prediction case, our approach achieved over 20% improvement. This framework transforms molecular property prediction by focusing on relevant domains and providing uncertainty estimates that scale with molecular similarity, addressing critical needs across diverse chemical applications.
