You might have to use the gpu_memory_limit and/or lora_on_cpu config possibilities to stay away from functioning away from memory. If you still operate from CUDA memory, you may seek to merge in system RAM https://socialmediainuk.com/story18187798/https-im-token-io-com-an-overview