What does optimizer_update_8bit function in Hugginface’s bitsandbytes.functional library in Python actually do with its state1 and state2 parameters?

  Kiến thức lập trình

I am having problems with the use of optimizer_update_8bit in one of my projects. I hope to use this function to perform quantized gradient updates, but I have not found any documentation on the meaning of its state1 and state2 parameters. Do these two parameters provide optimizer_update_8bit with the values of corresponding optimizer states, or do they only provide optimizer_update_8bit with the storage location of optimizer states, while the values of states will be calculated by optimizer_update_8bit and stored in them?

I used the function definition backtracking feature of VSCode IDE to view the implementation of this function, but I could only find calls to some more basic functions, whose implementation seemed to be hidden outside of Python, which did not help solve my problem.

I would be very grateful for any reply that gives a reasonable explanation or provides documentation on this function. Thanks!

Theme wordpress giá rẻ Theme wordpress giá rẻ Thiết kế website

LEAVE A COMMENT