Sidney Starr
Menu

Teaching a model to pick my next experiment

Benchmarking simple and complex machine-learning models on my experimental data, then using active learning so the model helps choose which experiment to run next.

Status In progress. Comparing ridge regression, random forests and neural networks; active learning is next. · updated Oct 30, 2026

The active-learning loop (schematic)A loop of four steps: run experiments, train models on the results, find where the model is least certain, and choose the experiment that would teach it the most. Then repeat.01Run experimentsin the lab02Train modelsridge · forest · neural net03Find where themodel is least sure04Pick the mostinformative next runrepeat:more learned per run
Fig. 1 · SchematicThe active-learning loop the project is building toward: the model proposes the experiment it would learn the most from.

The short version

Experiments take time. If a model can learn from the experiments I’ve already run, it can suggest which experiment would teach us the most next, so fewer experiments get us further. This project, a collaboration with Yuxin Chen’s group, is building toward exactly that.

TODO(Sidney): confirm how to describe the collaboration (Yuxin Chen's department/affiliation, collaborator names to credit, and your role).

The question being modeled

TODO(Sidney): in one plain sentence, what goes into the model and what it predicts (described without revealing unpublished data).

Step 1: Compare simple and complex models

We’re benchmarking three kinds of model against each other:

  • Ridge regression: a straight-line model with a penalty that stops it from overreacting to noise. Simple, fast and easy to interpret.
  • Random forests: many decision trees voting together. They capture non-linear patterns and still give some sense of which inputs matter.
  • Neural networks: the most flexible, and the most data-hungry.

Why bother comparing? Experimental datasets are small compared with the ones machine learning is famous for. A complex model can memorize a small dataset and look brilliant while learning nothing that holds up. Benchmarking against simple baselines, with a fair evaluation, shows whether the extra complexity is earning its keep.

Step 2: Active learning (next)

Active learning flips the usual order. Instead of collecting all the data and then fitting a model, the model looks at what it’s unsure about and proposes the experiment whose result would be most informative. Run that experiment, add the result, retrain, repeat.

It’s the core idea behind “self-driving” labs, and a natural bridge from my electrochemistry to experiment automation.

Technical depth: methodology

TODO(Sidney): evaluation approach (e.g. cross-validation scheme and why it suits a small experimental dataset), features and preprocessing, how uncertainty will be estimated for the active-learning step, the acquisition strategy under consideration, and tooling (e.g. scikit-learn, PyTorch).

My role

TODO(Sidney): what you do (experiments, data preparation, modeling, analysis) versus what collaborators do.

Results

Not public yet. This page will be updated once results are cleared to share with my PI and Yuxin Chen’s group.

Update log

  • 2026-10-30: Page created. Model comparison under way; active learning planned next.
  • machine learning
  • active learning
  • model benchmarking
  • Python

Results and data from this work will be shared once they’re published or cleared with my research group and Yuxin Chen’s group.