Relative Content

Tag Archive for pythontensorflowpytorchcudanvidia

Cuda out of memory even though there is enough memory

I’m trying to setup simple LLM server on Ubuntu 22.04. My GPU is Nvidia GeForce GTX 1050 (2048MB memory). I’m using Cuda Toolkit 12.4. I downloaded pretrained microsoft-phi3-mini model from https://huggingface.co/microsoft/Phi-3-mini-128k-instruct. The download was successful, and authentication is not required (I am logged in anyways). When I run example, provided on huggingface: