When you purchase through links on our site, we may earn an affiliate commission. Here’s how it works .
Now that llama.cpp runs natively on the PS5, fully utilizing the GPU and opening up more RAM to be used, u/AdIllustrious7246 was able to run a 27B parameter model on it at 20 tokens per second, and smaller models like Gemma 4 E4B and Llama 3.2 3B at 61 and 124 tokens per second, respectively. Previously, the large 6.8 GiB LLM would make the system freeze and crash due to the CPU bottleneck, but GPUs are much better at processing LLMs, which is why GPU and RAM prices are insane these days.
Running it on the GPU allowed for much faster token output, and running natively allowed it to use more RAM, up to 11.4 GiB, according to u/AdIllustrious7246. Not only did the developer run the app natively, but they also gave it a dashboard, showing useful data such as live token-per-second generation, GPU and CPU use, power draw, temperatures, and how the memory splits between the model and KV cache. They also made it work with llama.cpp’s web UI, allowing users to chat with the LLM from other local devices using a browser.
Developer uses Claude to enable Xbox Cloud Gaming on jailbroken PS5s
Key considerations
- Investor positioning can change fast
- Volatility remains possible near catalysts
- Macro rates and liquidity can dominate flows
Reference reading
- https://www.tomshardware.com/video-games/playstation/SPONSORED_LINK_URL
- https://www.tomshardware.com/video-games/playstation/jailbroken-ps5s-can-now-run-local-ai-models-at-over-100-tokens-per-second-fork-of-ps5lm-goes-native-and-accesses-gpu-and-more-ram#main
- https://www.tomshardware.com/membership
- Gigabyte X870 Aorus Infinity Motherboard Review: limited-edition, designed for extreme overclocking, but pricey for X870
- Two killed, one seriously hurt in explosion at Intel’s chip plant in Israel
- All Systems Go: Global PC App Ecosystem Ready for NVIDIA RTX Spark
- Experiment shows liquid metal TIM doesn’t dry out; it turns solid
- ASML struggling to keep up with demand for chipmaking equipment as AI spending continues to boom
Informational only. No financial advice. Do your own research.