When you purchase through links on our site, we may earn an affiliate commission. Here’s how it works .
Now that llama.cpp runs natively on the PS5, fully utilizing the GPU and opening up more RAM to be used, u/AdIllustrious7246 was able to run a 27B parameter model on it at 20 tokens per second, and smaller models like Gemma 4 E4B and Llama 3.2 3B at 61 and 124 tokens per second, respectively. Previously, the large 6.8 GiB LLM would make the system freeze and crash due to the CPU bottleneck, but GPUs are much better at processing LLMs, which is why GPU and RAM prices are insane these days.
Running it on the GPU allowed for much faster token output, and running natively allowed it to use more RAM, up to 11.4 GiB, according to u/AdIllustrious7246. Not only did the developer run the app natively, but they also gave it a dashboard, showing useful data such as live token-per-second generation, GPU and CPU use, power draw, temperatures, and how the memory splits between the model and KV cache. They also made it work with llama.cpp’s web UI, allowing users to chat with the LLM from other local devices using a browser.
Developer uses Claude to enable Xbox Cloud Gaming on jailbroken PS5s
Key considerations
- Investor positioning can change fast
- Volatility remains possible near catalysts
- Macro rates and liquidity can dominate flows
Reference reading
- https://www.tomshardware.com/video-games/playstation/SPONSORED_LINK_URL
- https://www.tomshardware.com/video-games/playstation/jailbroken-ps5s-can-now-run-local-ai-models-at-over-100-tokens-per-second-fork-of-ps5lm-goes-native-and-accesses-gpu-and-more-ram#main
- https://www.tomshardware.com/membership
- Thai cops bust 'audacious' cross-border fiberoptic cable running to Myanmar scam compound
- Modder injects leaked DLSS 5 4x Multipass on 1996 Quake
- RTX 5090 disappears from Nvidia marketplace amid reports of a production halt
- Experiment shows liquid metal TIM doesn’t dry out; it turns solid
- Modder injects leaked DLSS 5 4x Multipass on 1996 Quake
Informational only. No financial advice. Do your own research.