How can we capture update done with optimizer.step() in an elegant way?
I want to implement a method to monitor in Tensorboard the update-to-data ratio during training with PyTorch, following an idea mentioned in Karpathy’s video. I’ve come up with a solution, but I’m looking for a more elegant and configurable approach.