I am trying load Llama-2-13b on multiple GPU’s but isn’t loading, i have 3 GPU’s 24.169 GB each , but unable to load, i have tried using cuda or device_map =’auto’
This is my current code. When I try nvidia-smi in terminal, the GPU is always at 0%.When i remove split options then it works, but then it runs on CPU.
here’s below my try: