Conversation
WeightedHuberLoss mirrors WeightedMSELoss / WeightedMAELoss using torch.nn.functional.huber_loss. HuberLoss and WeightedHuberLoss both take an optional delta (default 1.0, matching torch); without it every Huber loss in a graph is stuck at the same transition point. I use different deltas per target (energy vs forces) for outlier-robust MLIP training, so the per-node delta is the part I actually need.
|
It's functional code, but it will be cleaned up a bit if you use the HuberLoss module along with the AutoKw mixin for instantiation. See for example LPReg (or many other AutoKw instances) |
Addresses review: HuberLoss and WeightedHuberLoss are now AutoKw nodes, following the LPReg pattern, with the modules built from torch.nn.HuberLoss and the weighted layer. This removes the wrapped-functional workaround for carrying delta.
|
Good point — switched both nodes to AutoKw with torch.nn.HuberLoss and the weighted module, following the LPReg pattern. Cleaner, thanks. |
| loss_func = staticmethod(torch.nn.functional.l1_loss) | ||
|
|
||
|
|
||
| class WeightedHuberLoss(_WeightedLoss): |
There was a problem hiding this comment.
I don't think you need to define this, I think you just need to import this at the node definition:
https://docs.pytorch.org/docs/2.13/generated/torch.nn.HuberLoss.html
The weighted module now holds torch.nn.HuberLoss(reduction="none") and only adds the same weight normalization the other weighted losses use, so no huber math is defined here. Both nodes take delta through AutoKw; the delta default matches torch.
|
Reworked it — the plain node now builds torch.nn.HuberLoss directly through AutoKw with delta. For the weighted one: delta is a per-term setting (my force term uses 5, energy 1) and the weights are per-sample, so the module takes delta, holds torch.nn.HuberLoss(reduction="none") for the huber part, and only defines the same weight normalization WeightedMSELoss and WeightedMAELoss use. |
This branch has not been deployed
2 participants
Adds WeightedHuberLoss, following the pattern of WeightedMSELoss and WeightedMAELoss. The weighted module holds torch.nn.HuberLoss(reduction="none") for the huber part and only adds the same weight normalization the other weighted losses use. Both HuberLoss and WeightedHuberLoss take delta (default 1.0, matching torch) — delta is a per-term setting, I train energies and forces with different deltas — and the plain HuberLoss node is built from torch.nn.HuberLoss with the AutoKw mixin. With the default arguments nothing changes for existing users.
Tests are in tests/test_losses.py.