Relative Content

Tag Archive for optimizationhuggingface

What does optimizer_update_8bit function in Hugginface’s bitsandbytes.functional library in Python actually do with its state1 and state2 parameters?

I am having problems with the use of optimizer_update_8bit in one of my projects. I hope to use this function to perform quantized gradient updates, but I have not found any documentation on the meaning of its state1 and state2 parameters. Do these two parameters provide optimizer_update_8bit with the values of corresponding optimizer states, or do they only provide optimizer_update_8bit with the storage location of optimizer states, while the values of states will be calculated by optimizer_update_8bit and stored in them?