Skip to content
This repository was archived by the owner on Aug 25, 2026. It is now read-only.

Support Optax extra arguments in StatefulTrainer - #143

Open
sylvesterkaczmarek wants to merge 2 commits into
google-deepmind:mainfrom
sylvesterkaczmarek:feat/trainer-optimizer-extra-args
Open

sylvesterkaczmarek wants to merge 2 commits into
google-deepmind:mainfrom
sylvesterkaczmarek:feat/trainer-optimizer-extra-args

Conversation

@sylvesterkaczmarek

Copy link
Copy Markdown

Summary

Fixes #126.

StatefulTrainer currently calls optimizer_def.update(grads, state, params) without any way to pass keyword-only extra arguments, which prevents using Optax GradientTransformationExtraArgs transformations that require additional signals.

This change:

  • wraps standard Optax transformations with optax.with_extra_args_support() so existing optimizers keep their current behavior;
  • adds a keyword-only optimizer_extra_args mapping to StatefulTrainer.step();
  • forwards that mapping only to optimizer_def.update(), while the existing **kwargs continue to go only to the loss function.

Testing

Added a regression test using a GradientTransformationExtraArgs update that requires a scale keyword argument and verifies a training step succeeds when the extra argument is supplied.

The existing deterministic MLP training test continues to exercise a normal optax.adam optimizer through the default path.

Signed-off-by: Sylvester Kaczmarek <assistant@SylvesterKaczmarek.com>
Signed-off-by: Sylvester Kaczmarek <assistant@SylvesterKaczmarek.com>
@sylvesterkaczmarek
sylvesterkaczmarek force-pushed the feat/trainer-optimizer-extra-args branch from 7b43ac9 to e2b772e Compare October 1, 2026 19:42
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

1 participant