Somewhat related, why the creators of Zettabyte File System (ZFS) decided to make it 128 bits (writing in 2004):
> Some customers already have datasets on the order of a petabyte, or 2^50 bytes. Thus the 64-bit capacity limit of 2^64 bytes is only 14 doublings away. Moore's Law for storage predicts that capacity will continue to double every 9-12 months, which means we'll start to hit the 64-bit limit in about a decade. Storage systems tend to live for several decades, so it would be foolish to create a new one without anticipating the needs that will surely arise within its projected lifetime.
> Although we'd all like Moore's Law to continue forever, quantum mechanics imposes some fundamental limits on the computation rate and information capacity of any physical device. In particular, it has been shown that 1 kilogram of matter confined to 1 liter of space can perform at most 10^51 operations per second on at most 10^31 bits of information [see Seth Lloyd, "Ultimate physical limits to computation." Nature 406, 1047-1054 (2000)]. A fully-populated 128-bit storage pool would contain 2^128 blocks = 2^137 bytes = 2^140 bits; therefore the minimum mass required to hold the bits would be (2^140 bits) / (10^31 bits/kg) = 136 billion kg.
> To operate at the 10^31 bits/kg limit, however, the entire mass of the computer must be in the form of pure energy. By E=mc^2, the rest energy of 136 billion kg is 1.2x10^28 J. The mass of the oceans is about 1.4x10^21 kg. It takes about 4,000 J to raise the temperature of 1 kg of water by 1 degree Celcius, and thus about 400,000 J to heat 1 kg of water from freezing to boiling. The latent heat of vaporization adds another 2 million J/kg. Thus the energy required to boil the oceans is about 2.4x10^6 J/kg 1.4x10^21 kg = 3.4x10^27 J. Thus, fully populating a 128-bit storage pool would, literally, require more energy than boiling the oceans.*
Single data sets surpassed 2^64 bytes over a decade ago. This creates fun challenges since just the metadata structures can't fit in the RAM of the largest machines we build today.
Virtualization has pushed back the need for a while, but we are going to have to look at pointers larger than 64 bit at some point. It's also not just about the raw size of datasets, but how we get a lot of utility out of various memory mapping tricks, so we consume more address space than the strict minimum required by the dataset. Also if we move up to 128 bit a lot more security mitigations become possible.
By virtualization are you referring to virtual memory? We haven't even been able to mmap() the direct-attached storage on some AWS instances for years due to limitations on virtual memory.
With larger virtual memory addresses there is still the issue that the ratio between storage and physical memory in large systems would be so high that cache replacement algorithms don't work for most applications. You can switch to cache admission for locality at scale (strictly better at the limit albeit much more difficult to implement) but that is effectively segmenting the data model into chunks that won't get close to overflowing 64-bit addressing. 128-bit addresses would be convenient but a lot of space is saved by keeping it 64-bit.
Space considerations aside, 128-bit addresses would open up a lot of pointer tagging possibilities e.g. the security features you allude to.
> By virtualization are you referring to virtual memory?
No, I mean k8s style architecture, where you take physical boxes and slice them into smaller partitions, hence the dataset on each partition is smaller than the raw hardware capability. That reduces the pressure towards the limit.
x86 is a funny example because it supported unaligned access more than many designs. But ignoring that...
Many CPUs, not just x86, have a "physical bits" length that is less than the address size in the ISA. This saves transistor and power budget, since address buses can be smaller. Of course, it means there is a lower maximum RAM config for that design.
The software would still shuffle around the full ISA word/double word/quad word or whatever. In a typical OS, the MMU and page mapping logic would potentially interpret all the bits to map to the more limited physical address range. It didn't mean storing smaller pointers in your software data structures etc.
I'm not an expert, but I think it varies by ISA whether it is defined how the higher address order bits, above the physical range, are handled. Some may allow applications to set them for address tagging, while the CPU ignores those bits. But, others may require them to all be zeroed to get predictable behavior.
> In particular, it has been shown that 1 kilogram of matter confined to 1 liter of space can perform at most 10^51 operations per second on at most 10^31 bits of information
I believe the Bekenstein bound for holographic information on a 1 liter sphere, using space at the Planck scale for encoding, instead of matter, is about 6.7×10^67.
I confess I got that number by taking round trips through multiple models to ensure there was a clear consensus, as my form of "homework", as this is not my area of expertise.
As far as figuring out energy or speed limits for operations over post-Einsteinian twisted space, that will require new physics, so I am just going to wait until I have a 1 liter Planckspace Neo and just measure the draw while it counts to a very big number for a second. (Parallel incrementing with aggregation obviously allowed.)
Point being, there is still a lot of room at the bottom.
Interesting thought. Space can expand faster than the speed of light over significant distances, without breaking the speed limit locally.
But what happens if complex living space begins absorbing all the essentially flat local space around it? Is there a speed limit to space absorption? If space itself is shrinking, due to post-Einsteinian structures/packing, then effective speed limits go away. As traversal distances, and perhaps even the meaning of distance, disappear. So, perhaps not. I call this the "AI Crunch" end-of-the-universe scenario.
That is the computer I want. And I believe that sets a new upper bound for AI maximalism.
Firstly, True Names is an awesome read, and the real origin of cyberpunk. I much prefer it to Neuromancer or Diamond Age.
Secondly, I recently tried to work out what year on the Top500 list[1] I could reasonably be for around US$5000. It's surprisingly difficult to work out mostly because they use 64 bit flops and few other systems quote that number.
Human appetite for more knows no bounds. Imagine what we’d have to do for everyone to have a zettaflop. We won’t have the resources for it. So guys like this one are in competition with normal people who just need a little bit of compute, so that he can feel powerful with a million Claudes. Sad.
I just want to thank the submitter. This is the type of internet that I really miss. A very smart person who's a good writer, proud of their interests and obsessions.
I'm a big believer that humanity's future is in space in a Dyson Swarm. There are simply too many advantages. It's estimated that humanity currently uses ~10^11 Watts of power. About 10^16 Watts of solar energy hits the Earth but the Earth's cross-section is less than a billionth of the Sun's total energy output. A Dyson Swarm would give us access to ~10^25 Watts of power. With our current population that would give every person on Earth living space about equivalent to Africa and access to more energy than our entire civilization currently uses by orders of magnitude.
I bring this up to present an alternate view of the future that a lot of thought has gone into: the Matrioshka Brain. This is basically a Dyson Swarm but the entire thing operates as one giant computer. Some of the heat from inner layers is captured by outer layers for greater efficiency. That's the Matrioshka part.
How much computing power would this be?
It's hard to say but estimates range from 10^40 to 10^50 FLOPS (eg [1]). At 10^45 FLOPS that would give each person on Earth access to roughly 100 trillion zettaflops.
If all LLM advancements stopped today, but compute + energy got to the price where the $30 million zettaflop was possible, I wonder what outcomes would be possible? Would 1000 claudes be able to coordinate in meaningful ways? How much human intervention would be needed?
Fun post, but I find the industry's obsession with compute to be rather vapid, and this is a good example:
> One million Claudes. To be able to search every book in history, solve math problems, write novels, read every comment, watch every reel, iterate over and over on a piece of code until it’s perfect – spend a human year in 10 minutes. 50,000 people working for you, all aligned with you, all answering as one.
We are already near the limits of what we can do if we throw compute at Claude without improving the underlying models, and it is not clear how we can get big improvements on the underlying models at this point. Surely geohot knows this, so I am surprised he thinks that "one million Claudes" would be able to e.g. write a better novel than one hundred Claudes, or even one Claude.
Maybe the bottleneck is shifting from compute to energy and capital ; at some point it stops being a software problem and starts looking like infrastructure power land cooling...
Just feels like the constraint is moving down the stack
I’d say there odd a bit of a flaw in the read 50,000 books part though. The LLM reading that much doesn’t really get you 50k books of value as a person. You’re the bottleneck not the flops
97 comments
> Some customers already have datasets on the order of a petabyte, or 2^50 bytes. Thus the 64-bit capacity limit of 2^64 bytes is only 14 doublings away. Moore's Law for storage predicts that capacity will continue to double every 9-12 months, which means we'll start to hit the 64-bit limit in about a decade. Storage systems tend to live for several decades, so it would be foolish to create a new one without anticipating the needs that will surely arise within its projected lifetime.
* https://web.archive.org/web/20061112032835/http://blogs.sun....
And some math on what that means 'physically':
> Although we'd all like Moore's Law to continue forever, quantum mechanics imposes some fundamental limits on the computation rate and information capacity of any physical device. In particular, it has been shown that 1 kilogram of matter confined to 1 liter of space can perform at most 10^51 operations per second on at most 10^31 bits of information [see Seth Lloyd, "Ultimate physical limits to computation." Nature 406, 1047-1054 (2000)]. A fully-populated 128-bit storage pool would contain 2^128 blocks = 2^137 bytes = 2^140 bits; therefore the minimum mass required to hold the bits would be (2^140 bits) / (10^31 bits/kg) = 136 billion kg.
> To operate at the 10^31 bits/kg limit, however, the entire mass of the computer must be in the form of pure energy. By E=mc^2, the rest energy of 136 billion kg is 1.2x10^28 J. The mass of the oceans is about 1.4x10^21 kg. It takes about 4,000 J to raise the temperature of 1 kg of water by 1 degree Celcius, and thus about 400,000 J to heat 1 kg of water from freezing to boiling. The latent heat of vaporization adds another 2 million J/kg. Thus the energy required to boil the oceans is about 2.4x10^6 J/kg 1.4x10^21 kg = 3.4x10^27 J. Thus, fully populating a 128-bit storage pool would, literally, require more energy than boiling the oceans.*
* Ibid.
With larger virtual memory addresses there is still the issue that the ratio between storage and physical memory in large systems would be so high that cache replacement algorithms don't work for most applications. You can switch to cache admission for locality at scale (strictly better at the limit albeit much more difficult to implement) but that is effectively segmenting the data model into chunks that won't get close to overflowing 64-bit addressing. 128-bit addresses would be convenient but a lot of space is saved by keeping it 64-bit.
Space considerations aside, 128-bit addresses would open up a lot of pointer tagging possibilities e.g. the security features you allude to.
> By virtualization are you referring to virtual memory?
No, I mean k8s style architecture, where you take physical boxes and slice them into smaller partitions, hence the dataset on each partition is smaller than the raw hardware capability. That reduces the pressure towards the limit.
Many CPUs, not just x86, have a "physical bits" length that is less than the address size in the ISA. This saves transistor and power budget, since address buses can be smaller. Of course, it means there is a lower maximum RAM config for that design.
The software would still shuffle around the full ISA word/double word/quad word or whatever. In a typical OS, the MMU and page mapping logic would potentially interpret all the bits to map to the more limited physical address range. It didn't mean storing smaller pointers in your software data structures etc.
I'm not an expert, but I think it varies by ISA whether it is defined how the higher address order bits, above the physical range, are handled. Some may allow applications to set them for address tagging, while the CPU ignores those bits. But, others may require them to all be zeroed to get predictable behavior.
> In particular, it has been shown that 1 kilogram of matter confined to 1 liter of space can perform at most 10^51 operations per second on at most 10^31 bits of information
I believe the Bekenstein bound for holographic information on a 1 liter sphere, using space at the Planck scale for encoding, instead of matter, is about 6.7×10^67.
I confess I got that number by taking round trips through multiple models to ensure there was a clear consensus, as my form of "homework", as this is not my area of expertise.
As far as figuring out energy or speed limits for operations over post-Einsteinian twisted space, that will require new physics, so I am just going to wait until I have a 1 liter Planckspace Neo and just measure the draw while it counts to a very big number for a second. (Parallel incrementing with aggregation obviously allowed.)
Point being, there is still a lot of room at the bottom.
Interesting thought. Space can expand faster than the speed of light over significant distances, without breaking the speed limit locally.
But what happens if complex living space begins absorbing all the essentially flat local space around it? Is there a speed limit to space absorption? If space itself is shrinking, due to post-Einsteinian structures/packing, then effective speed limits go away. As traversal distances, and perhaps even the meaning of distance, disappear. So, perhaps not. I call this the "AI Crunch" end-of-the-universe scenario.
That is the computer I want. And I believe that sets a new upper bound for AI maximalism.
They talk quite a bit about this sort of thing at the end...
Secondly, I recently tried to work out what year on the Top500 list[1] I could reasonably be for around US$5000. It's surprisingly difficult to work out mostly because they use 64 bit flops and few other systems quote that number.
[1] https://top500.org/lists/top500/2025/11/
I bring this up to present an alternate view of the future that a lot of thought has gone into: the Matrioshka Brain. This is basically a Dyson Swarm but the entire thing operates as one giant computer. Some of the heat from inner layers is captured by outer layers for greater efficiency. That's the Matrioshka part.
How much computing power would this be?
It's hard to say but estimates range from 10^40 to 10^50 FLOPS (eg [1]). At 10^45 FLOPS that would give each person on Earth access to roughly 100 trillion zettaflops.
[1]: https://www.reddit.com/r/IsaacArthur/comments/1nzbhxj/matrio...
> One million Claudes. To be able to search every book in history, solve math problems, write novels, read every comment, watch every reel, iterate over and over on a piece of code until it’s perfect – spend a human year in 10 minutes. 50,000 people working for you, all aligned with you, all answering as one.
We are already near the limits of what we can do if we throw compute at Claude without improving the underlying models, and it is not clear how we can get big improvements on the underlying models at this point. Surely geohot knows this, so I am surprised he thinks that "one million Claudes" would be able to e.g. write a better novel than one hundred Claudes, or even one Claude.
I’d say there odd a bit of a flaw in the read 50,000 books part though. The LLM reading that much doesn’t really get you 50k books of value as a person. You’re the bottleneck not the flops
This is all just such crazy coincidence.
Everything is coming together so quickly.