Experimental Benchmark of ADME@NCATS Predictions in Tox21 Compounds
Molecular-graph models for absorption, distribution, metabolism and excretion (ADME)-related properties were compared with measured metabolic stability, solubility and passive permeability in selected Tox21 compounds.
Early compound prioritization when cellular bioactivity data need exposure-related context. This is a laboratory ADME benchmark, not a validated in vivo toxicity predictor.
Model & Design
NCATS used ADME@NCATS molecular-graph models, including directed message-passing neural networks, to predict rat liver microsomal stability, kinetic aqueous solubility, passive gastrointestinal permeability and passive blood–brain barrier permeability.
Experimental Format
The basic ADME comparison covered 180 selected compounds; 80 existing measurements were removed from model training before evaluation and 100 compounds underwent new assays. A separate availability-selected set of 55 acetylcholinesterase-active compounds was tested for passive blood–brain barrier permeability.
Model Components
Component
Tox21 compound sets
Description
A selected 180-compound subset for basic ADME endpoints and a separate, nonrandom 55-compound subset for the blood–brain barrier assay. The sets include drugs and other chemicals.
Component
Laboratory comparators
Description
Rat liver microsomal stability, kinetic aqueous solubility, PAMPA gastrointestinal permeability and PAMPA blood–brain barrier permeability, each compared with its corresponding model output.
Measurements & Endpoints
Endpoint-specific ROC AUC, balanced accuracy, sensitivity and specificity
Structural similarity to training compounds and classification errors by similarity range
Passive permeability in PAMPA, distinct from active transport and in vivo exposure
Analysis
Compare each model with its corresponding experimental classification; remove overlapping measured compounds from training; examine structural similarity and errors across chemical space. Interpret the basic ADME and blood–brain barrier sets separately.
Published Evidence
Study Design
Experimental classification of selected Tox21 compounds for three basic ADME endpoints after training-overlap removal.
Study Sample
180 compounds; 80 with prior NCATS measurements were removed from training and 100 were newly assayed. Endpoint measurements had some missing or inconclusive values.
Comparison
Laboratory rat liver microsomal stability, PAMPA gastrointestinal permeability and aqueous solubility.
Finding
ROC AUC: 0.87 for rat liver microsomal stability, 0.92 for PAMPA gastrointestinal permeability and 0.81 for aqueous solubility. Balanced accuracy was 0.78, 0.81 and 0.75, respectively; solubility overall accuracy was 0.83, sensitivity 0.59 and specificity 0.91. Approximately 40% of selected compounds had maximum Tanimoto similarity at or below 0.4 to relevant training data.
Interpretation & Limits
The selected set is not the complete Tox21 library.
Prediction accuracy can vary by endpoint and represented chemical space.
Rat microsomal stability and artificial-membrane permeability do not measure complete human disposition.
Study Design
Separate comparison of passive blood–brain barrier permeability predictions with PAMPA-BBB measurements.
Study Sample
55 available acetylcholinesterase-active compounds, not a random or fully representative Tox21 sample.
NIH researchers compared ADME@NCATS predictions with laboratory measurements for selected Tox21 compounds. Three basic ADME models reached ROC AUCs of 0.81–0.92; a separate passive blood–brain barrier model reached 0.77. The work supports compound triage, not validated prediction of in vivo toxicity.