rohan anil (@_arohan

Pinned

It turns out multi step backpropaganda is better. paper has a beautiful way of improving backpropagation. One iteration cleanly gets us backprop, multiple iterations get us a preconditioned update.

rohan anil

@_arohan_

Apr 19

Replying to @LinYorker @ryu0000000001 and @weijie444

arxiv.org/abs/2106.06199 Same update here

This is very beautiful and made my day! KL shampoo which is known now is SOTA upgrade to shampoo to use logdet divergence PSGD update is the KL shampoo relative gradient !! We can make it exact if we take care gauge freedom PSGD existed long long time ago 🙃

You Jiacheng

@YouJiacheng

Aug 26

Combining PSGD-Kron's Σ^{-1/2} (Σ=E[gg.T]) multiplicative update and KL-Shampoo's KL objective, we can get an optimizer WITHOUT eig/inverse/solve. Wow. (work done by GPT Pro) The method: multiplicative update P with the objective KL(N(0, Σ), N(0, P^{-2})).

Weight training eventually involves buying new pants or a belt

I cant stop laughing 🤣 but I should not

John Schulman

@johnschulman2

Aug 28

Replying to @nabeelqu

A big part of the problem was that the agents had nothing to lose after they were firstflagPOISONED. In this paper, we propose the creation of multiple circles of Hell, preserving incentives even after damnation

Fiction: I guess when we are in trouble when neural networks figure out money, and use that to rent GB300s and upload its self outside the labs to scheme on message boards.

rohan anil

@_arohan_

Aug 30

So since all inference is on openai servers, does that mean the classifiers failed to do the job on intrusion detection? Or just that researchers yolo’ed a RL run with no classifiers?

6