Writing your first grid search

This tutorial assumes you have completed the quick install.

Say we want to find a good step-size for SGD for training a small neural network on MNIST. The following code defines 3 experiments, one for each step-size we will try;

gridsearch.py

from optexp import Experiment, Problem, cli
from optexp.datasets import MNIST
from optexp.metrics import Accuracy, CrossEntropy
from optexp.models import LeNet5
from optexp.optim import SGD

experiments = [
    Experiment(
        problem=Problem(
            dataset=MNIST(),
            model=LeNet5(),
            batch_size=100,
            lossfunc=CrossEntropy(),
            metrics=[  # Monitor both the accuracy and the loss on the training and validation sets
                Accuracy(),
                CrossEntropy(),
            ],
        ),
        group="first_gridsearch",
        optim=SGD(lr=lr),
        steps=10,  # 2 epochs
        eval_every=5,  # Evaluate every epoch
    )
    for lr in [0.1, 0.01, 0.001]
]

if __name__ == "__main__":
    cli(experiments)

To download the datasets, run the following command:

python gridsearch.py prepare

To check the current status of the experiments, run the following command:

python gridsearch.py check
> Out of 3 experiments, 3 still need to run (0.00% complete)

To run the experiments locally and upload the results to wandb, run the following command:

python gridsearch.py run --local

Once the experiments are done, the status of the experiments should read 100% complete:

python gridsearch.py check
> Out of 3 experiments, 3 still need to run (100.00% complete)

You should see the results on wandb (https://app.wandb.ai/your_username/your_project/runs) and are ready to move on to analyzing the results.