honestly im so over my current setup its actually driving me crazy at this point. Ive been trying to fine-tune some smaller models for a project Im working on and my 3060 12gb just keeps hitting that stupid out of memory error every single time I try to do anything meaningful. Its like I spend more time debugging memory issues than actually training anything and its just exhausting. I was looking at the 4090 but the prices are just insane right now and I really dont want to spend 2 grand if I can avoid it. My budget is more in the $1300 to $1500 range because I need to get this done by the end of the month for a research paper and I literally dont have time to keep fighting with this old card. Is there anything out there that actually has enough VRAM for serious AI work without breaking the bank? I heard the 4080 Super might be okay but 16GB still feels kinda low for what Im trying to do. Should I just bite the bullet and go for a used 3090 instead for the 24GB or is there some other card Im missing that actually works for deep learning? I just need something that wont crash every 5 minutes...
Honestly, been using a used NVIDIA GeForce RTX 3090 24GB GDDR6X for a year now and Im totally satisfied. Its really the only way to get 24GB VRAM without spending 4090 money right now. No complaints about stability or memory errors anymore. It works well for large batches and fine-tuning. Just make sure your PSU is up to the task tho.
Saw this earlier and it reminds me of my own struggle. I actually tried to make a NVIDIA GeForce RTX 4080 Super 16GB GDDR6X work, but my training runs kept crashing the second I increased the context length. For your budget, a used NVIDIA RTX A5000 24GB GDDR6 is a solid alternative.
Big if true