Pytorch Lightning logs separately for train, validation and test datasets

Question

I am trying to log the loss and auc for all 3 of my datasets - train, validation and test. The datamodule defines the 3 loaders and I finally invoke the model as:

trainer.fit(model,datamodule)
trainer.test(model,datamodule)

This ends up creating 2 different log files - 1 with train and validation metrics, and 1 with test metrics. Is there a way to collect these together? Context is that I am using the mlflow logger and would like to see all 3 sets of metrics on the same chart, but because of this behavior, it posts 2 different sets of metrics with the same run id, creating 2 different graphs.

Matt Pitkin · Accepted Answer · 2025-03-12 09:43:19Z

1

It looks like lightning creates a different runid for each invocation. However, if I noticed if I start a run (in my case by logging hyperparameters):

mlflow_logger = MLFlowLogger(
    experiment_name=MLFLOW_EXPERIMENT_NAME,
    tracking_uri=MLFLOW_TRACKING_URI,
    run_name=MLFLOW_RUN_NAME,
)
mlflow_logger.log_hyperparams(hyperparams)

then lightning reuses the same runId for emitting metrics for fit and test stages.

edited Mar 12, 2025 at 9:43

Matt Pitkin

7,5001 gold badge28 silver badges51 bronze badges

answered Mar 11, 2025 at 23:28

Apurva

1732 silver badges8 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

Pytorch Lightning logs separately for train, validation and test datasets

1 Answer 1

Comments

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Related