Loading…
0
Tried a tiny local model on my dad's 2009 laptop when everyone said cloud only, and it actually ran fine
Everyone in my dev group told me you need a cloud GPU for anything useful, but I loaded a 3B model in 8GB of RAM last week in Boise and it answered all my notes questions with no internet. Anyone else running small models on old hardware, or am I the only one who thinks the cloud push is overkill for most people?
1 comments
Log in to join the discussion
Log In1 Comment
victor_perry9010d ago
Ran a 3B on my ancient thinkpad with 6GB just to see if it would choke. It didn't, and honestly it handled my grocery lists and dumb questions fine. The cloud thing makes sense for heavy stuff but most people just want a chatbot that works on the couch.
5