Nvidia’s memory costs soar 485%, latest AI systems now cost $7.8 million to build — memory now comprises 25% of the total cost, Rubin GPUs a mere $50,000 apiece

Nvidia's memory costs soar 485%, latest AI systems now cost $7.8 million to build — memory now comprises 25% of the total cost, Rubin GPUs a mere $50,000 apiece

Follow Tom's Hardware on Google News , or add us as a preferred source , to get our latest news, analysis, & reviews in your feeds.

Anton Shilov is a contributing writer at Tom\u2019s Hardware. Over the past couple of decades, he has covered everything from CPUs and GPUs to supercomputers and from modern process technologies and latest fab tools to high-tech industry trends. ","collapsible":{"enabled":true,"maxHeight":250,"readMoreText":"Read more","readLessText":"Read less"}}), "https://slice.vanilla.futurecdn.net/13-4-23/js/authorBio.js"); } else { console.error('%c FTE ','background: #9306F9; color: #ffffff','no lazy slice hydration function available'); } Anton Shilov Social Links Navigation Contributing Writer Anton Shilov is a contributing writer at Tom’s Hardware. Over the past couple of decades, he has covered everything from CPUs and GPUs to supercomputers and from modern process technologies and latest fab tools to high-tech industry trends.

ekio Some people will be made fun of in the future, regarding how much they were stupidly paying for compute, allowing 90 percent margins for the mob, I mean from Ngreedia. Reply

alan.campbell99 Data centre endeavours are already debt-laden as it is and this will just add more to the fire, VC/PE money is not an infinite resource. I don't like this timeline. Reply

JamesJones44 alan.campbell99 said: Data centre endeavours are already debt-laden as it is and this will just add more to the fire, VC/PE money is not an infinite resource. I don't like this timeline. Something to watch is a lot of the memory producers have resisted adding capacity. I think if they thought this was going to last they would bring more production online. It will be interesting to see where we are a year from now. Reply

timsSOFTWARE There haven't been any major technological breakthroughs since 2017, with the invention of the transformer, and the "attention is all you need" paper. Everything that has happened since then has been sort of low-value. Even though there are companies that will IPO with trillion-dollar valuations, not having contributed all that much scientifically. Reply

usertests timsSOFTWARE said: There haven't been any major technological breakthroughs since 2017, with the invention of the transformer, and the "attention is all you need" paper. AI says mRNA vaccines and quantum computers (debatable, quantum supremacy when?). I would add drones, specifically the use of low-cost drones in warfare, which have become completely ubiquitous. In computer hardware, most of the action has been in 2.5D/3D packaging, such as the giant interposers being used (mostly for AI chips). GAAFETs and backside power delivery are major on the transistor side. LLMs as we know them were completely worthless in 2017. Now AI-assisted coding is here to stay and they are digging up security vulnerabilities at an alarming rate (also debatable, maybe a pretext for regulating competition out of existence). Reply

Shiznizzle If and when the chinese bring their fabs online will they keep the prices just as high? My guess is they will. I am ok with going outside again and sniffing grass on my bikes just like when i was a kid. What i am not ok with, as are many others it seems, is paying those prices for tech. Us, joe public, not buying expensice stuff will have an effect. What i find interesting is that the demand is not there but everybody is keeping prices sky high. Reply

usertests Shiznizzle said: I am ok with going outside again and sniffing grass on my bikes just like when i was a kid. What i am not ok with, as are many others it seems, is paying those prices for tech. You’re Breathing Plastic: Study Finds 4% of City Air Pollution Is Microplastics Air pollution may be changing people’s bodies, adding fat while draining lean mass Air pollution directly linked to Alzheimer’s risk, scientists say LORD LET THIS MAN SNIFFIZZLE POLLUTION AND PAY FOR TECH Reply

timsSOFTWARE usertests said: AI says mRNA vaccines and quantum computers (debatable, quantum supremacy when?). I would add drones, specifically the use of low-cost drones in warfare, which have become completely ubiquitous. In computer hardware, most of the action has been in 2.5D/3D packaging, such as the giant interposers being used (mostly for AI chips). GAAFETs and backside power delivery are major on the transistor side. LLMs as we know them were completely worthless in 2017. Now AI-assisted coding is here to stay and they are digging up security vulnerabilities at an alarming rate (also debatable, maybe a pretext for regulating competition out of existence). Sorry – I was talking about the field of AI/LLMs specifically. They've gotten bigger, and techniques like RLHF have been employed to improve them incrementally, but they are all still based on the transformer architecture from the 2017 "attention is all you need" paper. AI assistance for coding – like other forms of writing assistance – is very broad/generic. For the companies using AI to churn out massive volumes of code quickly, and deploying that code in production – I don't think the last chapter of that book has been written yet. I don't think that's going to work out for those companies in the end; it's just too hard to maintain and keep a reasonable level of quality, as expected for commercial products. But it has replaced Stack Overflow, etc., as the first place to go when you have a question, can accelerate learning new things by a lot, and can be highly effective when given smaller, well-defined/sandboxed tasks, where the lack of long-term memory/context window limits aren't an issue. Reply

Key considerations

  • Investor positioning can change fast
  • Volatility remains possible near catalysts
  • Macro rates and liquidity can dominate flows

Reference reading

More on this site

Informational only. No financial advice. Do your own research.

Leave a Comment