When you purchase through links on our site, we may earn an affiliate commission. Here’s how it works .
Now that llama.cpp runs natively on the PS5, fully utilizing the GPU and opening up more RAM to be used, u/AdIllustrious7246 was able to run a 27B parameter model on it at 20 tokens per second, and smaller models like Gemma 4 E4B and Llama 3.2 3B at 61 and 124 tokens per second, respectively. Previously, the large 6.8 GiB LLM would make the system freeze and crash due to the CPU bottleneck, but GPUs are much better at processing LLMs, which is why GPU and RAM prices are insane these days.
Running it on the GPU allowed for much faster token output, and running natively allowed it to use more RAM, up to 11.4 GiB, according to u/AdIllustrious7246. Not only did the developer run the app natively, but they also gave it a dashboard, showing useful data such as live token-per-second generation, GPU and CPU use, power draw, temperatures, and how the memory splits between the model and KV cache. They also made it work with llama.cpp’s web UI, allowing users to chat with the LLM from other local devices using a browser.
Developer uses Claude to enable Xbox Cloud Gaming on jailbroken PS5s
Key considerations
- Investor positioning can change fast
- Volatility remains possible near catalysts
- Macro rates and liquidity can dominate flows
Reference reading
- https://www.tomshardware.com/video-games/playstation/SPONSORED_LINK_URL
- https://www.tomshardware.com/video-games/playstation/jailbroken-ps5s-can-now-run-local-ai-models-at-over-100-tokens-per-second-fork-of-ps5lm-goes-native-and-accesses-gpu-and-more-ram#main
- https://www.tomshardware.com/membership
- Into the Omniverse: How Developers Turn Ideas Into Simulations With Frontier AI Agents
- NVIDIA DGX Spark 64GB Gives Developers More Ways to Build and Scale Local AI
- GFiber and Qualcomm announce Wi-Fi 8 gateway
- TSMC confirms it is considering Terafab chipmaking expansion with Elon Musk
- SK hynix to place custom tiny CPU cores in its SSD controllers to ease Arm load as drive speeds rocket
Informational only. No financial advice. Do your own research.