Sooo, in my experience, I recently bought a NVIDIA GeForce RTX 3090 24GB. Its amazing!! The VRAM is essential for training custom models. Honestly, its the most logical value pick for beginners!

Sooo, I actually went through this exact same struggle last year. I thought I'd be clever and save a ton of cash by picking up a used NVIDIA GeForce RTX 3090 24GB Founder's Edition from a marketplace seller. Honestly, it was kinda a mistake... unfortunately, the card had major thermal issues because the previous owner basically cooked it in a mining rig 24/7. I spent more time fiddling with settings and worrying about it crashing than actually training my models, which was just super frustrating and realy not as good as I expected. I eventually bit the bullet and upgraded to a NVIDIA GeForce RTX 4090 24GB. The speed is actually amazing for LLMs, but man, the price tag still hurts my soul lol. I also looked into enterprise stuff like the NVIDIA RTX A6000 48GB, but the value just isn't there for a personal setup unless you've got corporate money to burn. If you're cautious like me, maybe skip the used market unless you can test the card's stability first. Quick tip: Always prioritize VRAM capacity over raw clock speed for training custom LLMs. If you go for the 4090, definitely pair it with a high-quality PSU like the EVGA SuperNOVA 1000 G6 1000W 80 Plus Gold to avoid random power spikes crashing your long training sessions. Good luck tho!

Any updates on this?

Yeah, I totally see why people look at those older cards for the VRAM, but honestly, as someone who is just getting their feet wet, the whole 'buying used' route seems super risky. I’ve been reading a lot about how these things can fail during long sessions, and that just scares me too much. Tbh, I think you're better off just sticking with brand-new equipment from Nvidia. It’s probably safer to just get any of the current-gen options from a trusted retailer so you actually have a warranty to fall back on (at least that’s my plan). Is it really a good deal if it breaks after a few weeks? I basically just want something that I know is going to work every time I hit start. Does anyone else worry about the stability of those older setups? I'd definitely lean toward the reliability of new hardware instead of trying to hunt for a bargain.