LeafSplit on ClinTox: from 0.247 AP to hybrid results above 0.4.
Experiments: 9–23 October 2026
The first Leaf model reached a mean toxicity AP of 0.247. After changes to training, model averaging and a final classifier that could also use the original molecular fingerprints, the hybrid reached 0.474 AP on the original test split. On three additional scaffold splits, hybrid variants averaged 0.410–0.413 AP.
These results describe the Leaf-based pipeline. The biggest gain came from direct access to the fingerprints. The original fingerprint-only logistic baseline was already at 0.472, and later fingerprint-only controls remained competitive. They helped separate improvements to the full system from improvements inside Leaf.
| Stage | Toxicity AP | How the result was measured |
|---|---|---|
| Starting Leaf model | 0.247 | Mean AP of three individual models on the original split |
| Hybrid, stronger Leaf penalty (100) | 0.474 | AP of averaged predictions on the original split, after validation-based penalty selection |
| Hybrid, standard Leaf penalty (1) | 0.413 | Mean of three additional split-level ensemble AP scores |
| Hybrid, stronger Leaf penalty (100) | 0.410 | Mean of the same three additional split-level ensemble AP scores |
ClinTox contains two prediction targets: clinical-trial toxicity and FDA approval status. We trained separate outputs for these labels, except in one experiment that removed FDA from the fine-tuning stage. Our downloaded file contained 1,484 rows. Four SMILES strings could not be parsed, and 19 duplicate rows were removed. That left 1,461 unique molecules.
| Partition | Molecules | Known toxicity labels | Toxic positives |
|---|---|---|---|
| Training | 1,168 | 1,154 | 77 |
| Validation | 146 | 143 | 6 |
| Test | 147 | 145 | 10 |
Molecules were grouped by their Murcko scaffolds—their central structural frameworks—before splitting. Whole groups stayed together. This avoids putting the same scaffold in both training and test.
We first compared Leaf with logistic regression and Random Forest on the same fingerprints. Training subsets contained 292, 583 and 1,168 molecules: approximately 25%, 50% and all available training data. Each setting used seeds 42, 123 and 2026.
| Model | 25% of training data | 50% | 100% |
|---|---|---|---|
| Original Leaf | 0.100 | 0.102 | 0.247 |
| Logistic regression, class-balanced | 0.347 | 0.414 | 0.472 |
| Random Forest | 0.316 | 0.318 | 0.430 |
| Constant-score reference | 0.069 | 0.069 | 0.069 |
The original Leaf produced 11 readouts, but only two fed the task predictions. We added a trainable 11-to-2 layer so every readout could contribute to both outputs. This changed the model from 35 parameters to 57: 33 circuit angles, 22 output weights and two biases.
| Experiment | Ordinary training | Class-balanced training |
|---|---|---|
| Original Leaf, first two readouts | 0.247 | — |
| Trainable layer using all 11 readouts | 0.216 | 0.107 |
| Select weights by validation toxicity AP | 0.182 | 0.105 |
| Train the new layer over a frozen random Leaf | 0.135 | 0.118 |
| Train the new layer over an already trained, frozen Leaf | 0.257 | 0.285 |
| Same frozen Leaf, all 60 epochs without early stopping | 0.257 | 0.283 |
| Unfreeze Leaf and fine-tune with a smaller learning rate | 0.279 | 0.311 |
| Fine-tune with learning-rate drops every 20 epochs | 0.273 | 0.293 |
| Fine-tune toxicity only, without the FDA output | 0.269 | 0.305 |
The useful sequence was to load an already trained Leaf, train its new output layer, then unfreeze the model carefully. During joint fine-tuning, the output layer used a learning rate of 0.01, while Leaf used 0.001 for another 60 epochs. This brought mean AP to 0.279 with ordinary training and 0.311 with class balancing.
Next, we averaged the predicted probabilities from three fine-tuned models. This produces one new set of predictions. Its AP is calculated after averaging; it is not the average of three AP scores.
| Fine-tuned model setting | Mean AP of individual models | AP of averaged predictions |
|---|---|---|
| Ordinary training | 0.279 | 0.312 |
| Class-balanced training | 0.311 | 0.329 |
With all training data, averaging gave another improvement. The balanced ensemble reached 0.329 AP.
The next change produced the largest jump. We gave the final classifier two inputs together: the 2,048 original fingerprint bits and the 11 Leaf readouts. That made 2,059 input features. Leaf stayed frozen while a regularized logistic classifier was fitted with SciPy L-BFGS.
| Final classifier | Ordinary training | Class-balanced training |
|---|---|---|
| Fine-tuned Leaf ensemble | 0.312 | 0.329 |
| Fingerprints + frozen Leaf readouts | 0.460 | 0.435 |
| Matched fingerprint-only control | 0.460 | 0.471 |
| Hybrid after validation-based choice of Leaf penalty | 0.474 | 0.435 |
The ordinary hybrid moved from the Leaf ensemble's 0.312 to 0.460 AP. A stronger penalty on the Leaf-connected weights then brought it to 0.474.
We repeated the comparison with scaffold-split seeds 17, 29 and 43. For each split, 20% of scaffold groups went to test, then 12.5% of the remaining groups went to validation. The test sets contained 231, 265 and 223 molecules.
| Scaffold split seed | Hybrid, Leaf penalty 1 | Hybrid, Leaf penalty 100 | Fingerprint-only |
|---|---|---|---|
| 17 | 0.491 | 0.480 | 0.481 |
| 29 | 0.434 | 0.437 | 0.436 |
| 43 | 0.315 | 0.312 | 0.310 |
| Mean | 0.413 | 0.410 | 0.409 |
Both hybrid variants averaged above 0.4. The standard penalty gave the better mean, while the stronger penalty beat it on only one split. The advantage over fingerprint-only was small: about 0.0043 AP for the standard hybrid.
We looked more directly at Leaf's calculations using the same three scaffold splits with a new one-stage Leaf training setup: fresh models, 60 epochs, no early stopping, output-layer learning rate 0.01 and Leaf learning rate 0.001.
| Change | Matching control AP | Changed model AP |
|---|---|---|
| Squared-amplitude readout, Leaf alone | 0.159080 | 0.095799 |
| Squared-amplitude readout, fingerprint/Leaf hybrid | 0.387848 | 0.406351 |
| Leaf correction added to a frozen fingerprint classifier | 0.409127 | 0.409167 |
| Trainable internal-link angles, Leaf alone | 0.095799 | 0.095343 |
| Trainable internal-link angles, fingerprint/Leaf hybrid | 0.406351 | 0.407867 |
The learned-link hybrid averaged 0.407867 AP, improving on two splits by a mean of about 0.0015 AP. Fingerprint-only was still ahead on average and on two of the three splits.
FDA approval stayed as the second prediction target in most runs. In the latest link comparison, mean FDA AP was 0.989203 for the learned-link hybrid and 0.990370 for fingerprint-only. Those high values need context: FDA-positive labels dominate this dataset. On the original test set, a constant-score model already had FDA AP of 0.938.
The tested system moved from the starting Leaf result of 0.247 AP to hybrid results above 0.4. Better training and model averaging helped. Direct fingerprint access produced the largest gain. Changes inside Leaf also trained correctly, although their extra benefit was small and inconsistent.
The next design uses input-dependent rotation angles, alternating CX directions and a trainable input encoder. These remain planned work: the completed link angles are not input-dependent, and score correction is different from a residual connection between layers.