Skip to content

Experiment tabs

An experiment contains the following tabs during and after training that you can view to understand the model:

To access one of the above tabs:

  1. After starting an experiment, H2O Hydrogen Torch will take you to the View experiments card; in there, click the experiment (name) you want to view.

  2. Right after, the above mentioned tabs will appear.

Each tab above is explained in turn below.

Charts

H2O Hydrogen Torch offers four different graphs to highlight performance metrics during an experiment, and such graphs can be located on the Charts tab of an experiment. The graphs are as follows:

Learning Rate

The Learning Rate graph displays the current learning rate throughout the training process visualized after each training batch.

Search bar

Train Batch Loss

The Train Batch Loss graph displays the current training loss throughout the training process visualized after each training batch.

Train batch loss

Validation Batch Loss

The Validation Batch Loss graph displays the current validation loss throughout the training process. Loss is calculated as the average of all validation batches. By default, it is calculated and visualized after each training epoch, but this frequency can be adjusted via the Evaluation Epochs setting when creating an experiment.

Validation batch loss

Note

The Evaluation Epochs setting is only available when the Experience Level is set to Expert or Master. To learn more, see Experience levels.

Validation {Metric}

The Validation {Metric} graph displays the current validation metric throughout the training process. In this case, the {Metric} refers to the selected metric for the problem type (e.g., MAE for an image regression experiment). {Metric} is calculated on full validation set predictions. By default, it is calculated and visualized after each training epoch, but this frequency can be adjusted via the Evaluation Epochs setting when creating an experiment.

Validation metric

Note

The Evaluation Epochs setting is only available when the Experience Level is set to Expert or Master. To learn more, see Experience Levels.

Summary

On the Summary tab, H2O Hydrogen Torch displays the following table containing summary metrics about the selected experiment:

Name Description
Name Name of the experiment.
Dataset Dataset used for the experiment.
Problem Type Problem type of the experiment.
Fold The validation fold value.
Seed The random seed value that you selected for model training.
GPU List List of GPUs that H2O Hydrogen Torch used during model training.
Loss Loss refers to the objective function optimized directly during model training.
Metric Metric refers to the selected metric to evaluate model performance.
Epoch The number of epochs used to train the model.
Train Metric Train metric refers to evaluation metric computed on train data. A value for Train metric is available when Calculate Train Metric is turned on when defining the experiment settings.
Val Metric Val Metric refers to the evaluation metric computed on the validation data.
Test Metric Test metric refers to the evaluation metric computed on the test data.

Metrics

On the Metrics tab, H2O Hydrogen Torch displays all available validation metrics applicable to the problem type of the experiment. Computed metrics in this tab are for the validation dataset unless a test dataset is provided (1) or the Calculate Train Metric setting is turned on (2).

  1. If a test dataset is provided, metrics for it will be available.

  2. If the Calculate Train Metric setting is turned on, metrics for the training dataset will be available.

Metrics

Note

  • Best thresholds and their corresponding metric values are shown for threshold-dependant metrics.

  • For text and image classification experiments, confusion matrices are shown. In addition, an interactive threshold slider is displayed in the case of binary or multi-label classification. Moving the slider changes the metric values for threshold-dependant metrics along with the confusion matrices.

Train Data Insights

On the Train Data Insight tab, H2O Hydrogen Torch displays random train visual samples after augmentations.

Note

The Train Data Insights tab is available for image problem-type experiments.

Validation Prediction Insights

On the Validation Prediction Insights tab, H2O Hydrogen Torch visualizes the validation samples of an experiment while providing the loss value of each sample. In particular, visualizations are divided as follows:

  • Random validation samples

    In the Random validation samples tab, H2O Hydrogen Torch visualizes an experiment's best and worst validation samples while proving the loss value for each sample.

  • Best validation samples

    In the Best validation samples tab, H2O Hydrogen Torch visualizes the best validation samples of an experiment while providing the loss value of each sample. Samples in this tab are regarded as the model's smallest significant errors.

  • Worst validation samples

    In the Worst validation samples tab, H2O Hydrogen Torch visualizes the worst validation samples of an experiment while providing the loss value of each sample. Samples in this tab are regarded as the model's most significant errors. Samples in this tab can help identify issues the model might have.

Note

Evaluating the best and the worst performance differs for each problem type and might vary from the selected loss and the selected metric to assess the model.

Validation Interpretation Insights

On the Validation Interpretation Insights tab, H2O Hydrogen Torch visualizes the validation samples for an experiment through Gradient-weighted Class Activation Mapping (Grad-CAM) while providing the loss value of each sample. In particular, H2O Hydrogen Torch divides visualizations as follows:

  • Random validation samples

    In the Random validation samples tab, H2O Hydrogen Torch visualizes the best and worst validation samples through Grad-CAM while providing the loss value for each sample.

  • Best validation samples

    In the Best validation samples tab, H2O Hydrogen Torch visualizes the best validation samples through Grad-CAM while providing the loss value of each sample. Samples in this tab are regarded as the model's smallest significant errors.

  • Worst validation samples

    In the Worst validation samples tab, H2O Hydrogen Torch visualizes the worst validation samples through Grad-CAM while providing the loss value of each sample. Samples in this tab can also be regarded as the model's most significant errors. Samples in this tab can help identify issues the model might have.

Note

The Validation Interpretation Insights tab is only available for image and audio classification and regression problem types:

  • For image problem types, Grad-CAM highlights the areas of an image the model considered the most when generating a prediction (and loss) for the image.

  • For audio problem types, Grad-CAM highlights the areas of a spectrogram the model considers the most when generating a prediction (and loss) for the image.

    Note

    Spectrograms longer than five seconds are transformed to display only the first five seconds of audio.

Logs

On the Logs tab, you can view and download the logs for the experiment in view.

To download the logs of an experiment:

  1. In the H2O Hydrogen Torch navigation menu, click View experiments.
  2. Click the name of the experiment you want to download its logs.
  3. Click Download logs/config.

Config

On the Config tab, you can view the configurations used during the experiment.


Back to top