Probably not. "This model is too powerful for the public" can also be interpreted another way, which they've also strongly hinted at - the cost/benefit ratio of the upgrade is negative for the vast majority of all users. Finding vulnerabilities is one of the few cases where it makes sense to use it.
Their writing about the model so far does say this is an issue where, for instance, you can't really use Mythos for interactive coding because it's so slow. You have to give it some work, go home, sleep, come in the next day and then maybe it'll have something for you.
All the AI labs and startups are still losing money hand over fist. Launching Mythos would require it to be priced well above current models, for a much slower product. Would the majority of customers notice the difference in intelligence given the tasks they're setting? If the answer is no, it's not economic to launch.
Really, I'm surprised they've done Mythos. Maybe they just wanted to exploit access to larger contiguous training datacenters than OpenAI, but what these labs need isn't smarter models, it's smaller and cheaper models that users will accept as good enough substitutes (or more advanced model routing, dynamic thinking, etc).
This happened before with GPT-2 being touted as "too dangerous to release"[0] at the time by OpenAI. I don't think that means every model will be safe to release in the future, but nothing I've read about Mythos seems like it's going to be different this time.
Their main motivation of the model being too dangerous is predicated on their discoveries in its ability to find exploits in commonly used software. The idea is that if this were served on a public API, it would massively increase the scale and scope of what malicious actors could do.
I think it's a reasonable choice to make given that Mythos actually does have cyber capabilities on that level. We already have evidence that large-scale scams are being perpetuated using AI models (such as AI video being passed as real, people deepfaking themselves in job interviews).
If you've noticed your new model can be trivially pointed at some open-source codebase with a prompt and harness that amounts to "find as many exploits as possible" and your results are non-trivially substantial and beyond what existing models can do given the same initial parameters, then a gated rollout seems the most reasonable option.
I feel like "this model is too powerful for the general public" was really just the equivalent of responsible disclosure, with the "too powerful" bit just a positive marketing spin like you say.
That is, Mythos will make it much easier to find lurking zero days, so just like responsible disclosure requires a security researcher to notify the software author first and give them some time to patch, giving critical infrastructure folks at least some time to analyze and patch systems seems reasonable to me.
Yup, this whole thing is quite typical for my generations attempts at activism: they always end up as marketing pawns for the very thing they set out to stop.
This whole "this model is too dangerous" ploy originated from (in my opinion severely misguided) activists who wanted to stop or slow AI development down as much as possible, spreading outlandish Doomsday scenarios wherever they could.
These online-first activists have always been a key driver of the success of the very thing they fight. They share the offending thing among themselves, making it go viral in process, and soon baiting these groups is the best marketing imaginable.
There were some rather interesting studies made on the subject around 2011, I particularly remember one made by Swedish jeans brand cheap Monday, but i can't find it now.
> This feels really premature. The announcement was a week ago. The “this model is too powerful for the general public” sounds like marketing to me.
Anthropic was born out of the idea that they feel paternity over humanity. They believe by limiting access they are performing a necessary pillar of security in multiple facets.
I think it's up to the public, and articles like this are part of the public's voice, whether this belief is serious or not and secondarily whether it's okay to even posture this kind of belief since it inherently results in marginalizing the many and rewards an already very successful few.
For me, the seeming majority optimism and acceptance of “mythos’” as yet untold capabilities is betrayed as not real by the fact that one can’t react to it with the same reverence while framing it as a downside without being told “it’s not even out yet”.
“It’s not even out yet” should apply to both situations or neither.
Anthropic marketing is working very well. They are strongly incentivized to say their model is too powerful to release even if it’s not. It’s almost standard practice these days.
> The “this model is too powerful for the general public” sounds like marketing to me
I tend to agree here. Anthropic has built a reputation and now they are in a position where they can claim to have a model way more powerful than it might actually be, and by limiting its access, there won't be an independent way to test it. I'm not denying that it's not smarter than Opus, but probably it's somewhat exaggerated.
There’s a drain clog clearer sold in a jug like all the rest. But they wrap the jug in a thick clear bag. The implication is clear - this stuff is so powerful it’s extra dangerous.
But it is another thing entirely to share access only with enterprise partners such as Crowdstrike, Cisco, and Microsoft, which are known to have massive security incidents regularly.
The stated purpose of Glasswing is to give infra and security orgs the chance to close holes and improve their security. In that context, it seems odd to call for not providing them access with the justification being that they have security breaches sometimes.
Mythos may or may not itself be opened to the public at some point, but I would charitably expect that Anthropic plans that a future model at least as good as Mythos Preview will be, and the limited release for Mythos is intended to make that eventuality safer by having most of the existing holes patched.
> A 16-year-old with no credentials and no capital could just do things. The world of bits offered the freedom to build without being drowned in arbitrary constraints, in a way that didn’t require assembling vast capital or prestige or connections, where your creativity and work could speak for itself, and you had agency.
Some of these concerns are precisely why we are building Nemotron at NVIDIA. We want to make it possible for people to invent and deploy AI in all sorts of new and unforeseen ways.
Nemotron is:
1. Not just open weight, but open data (to the limits of what is feasible), open recipe, open technique
2. In the future built by a coalition of organizations coming together to build great openly developed AI.
Nemotron 3 Super is our most successful model yet. [1]
Ultra is coming soon. And then Nemotron 4.
We can afford to do this because when AI grows, NVIDIA's opportunity also grows.
The problem is Anthropic doesn't have the compute to deploy this model to scale to everyone yet. Dario didn't believe that they needed as much compute, OpenAI is going to have much more compute unlocked this year and especially next year.
In other, underreported news, companies like AirBnB are using open source models. Anthropic and OpenAI have a six months to year advantage over Qwen and other models. We reached the point a while ago where Anthropic models where good enough, and so now, inevitably, we’ve reached the point where open models are good enough. The boasting of models so good they can’t be shared was propaganda to frame the conversation. But for anyone paying attention, what matters is that open models are now good enough.
Most important point in the piece (though I’m not sure if the historical analogy to the grid holds, given local electricity production has been unavailable for the majority of the history of the grid)
> You can generate your own electricity with a solar panel (think local models), but most people would rather pay a utility bill. And the power company doesn’t decide, on the basis of pedigree, who is worthy of electricity. Intelligence should work similarly, where the capabilities you can access scale may scale with vetting and due process, but the presumption should be access. Add safety guardrails to restrict dangerous use; start by making them overly trigger-happy if you must, and calibrate over time. But the default should be to allow entry.
Anthropic should provide a specific service where they attack a businesses infrastructure using this frontier model and then issue a report of all vulnerabilities found. I could imagine it would be quite lucrative.
Much better than hiding it away where it can't help anyone.
You never needed a godzilla or a megatron to get on with your life. But the sellers of those monsters would make every attempt, in connivance with the authorities, to make it a basic necessity to use their services. That's a survival strategy for the monsters. The owners can't keep the monsters in cages for too long, even if the owner is a state actor.
The opportunity from the early days of the American frontier is not typical. Instead, it's the brief burst of unrestrained growth as a better-adapted organization (the US, software companies) expands into, and expands, a niche--cannibalizing the previous occupant (Native Americans, older stagnant companies.) At times growth is so rapid that individuals are able to advance the frontier, but if the field stagnates, individuals will be outcompeted by corporations.
So, opportunity for individuals comes from disruption. Creative destruction is good up to a point, but it results from advancing capabilities. Technological advances compound and accelerate exponentially. Eventually we reach the point where any malcontent can destroy the world by snapping their fingers. At some point we need to place restrictions on the capabilities accessible to individuals. We have reached that point with nuclear weapons, and I think it is sensible to believe that AI is reaching that point as well.
While I don't think Mythos is so powerful that it justifies containment permanently - I wonder how it might work when there is such an AI that can justify containment.
What if this new model can start proving Millenial problems and provide insights in other fields that was not possible before?
My intuition says that a model that is as good will also be equally well aligned -- but it is still highly risky to give it to the general public because all you need is one jailbreak from bad actors.
At that point I think society would change so dramatically that "access to general public" would be a non issue. Rather, time would be spent on making abundance happen - you might think of the political struggles, economics and new ventures.
Its a bit sad that democratised access is not provided because of negative sum possibilities like cybersecurity.
I admit I didn't read the entire post (I honestly think authors really need to come to terms with the fact that we now live in a world of information excess, and pithiness is more important than ever), but I wouldn't feel too bad yet given there was a recent front page HN post about how free, open models could actually catch all the issues Mythos did, it just required a little more orchestration. E.g. see https://aisle.com/blog/ai-cybersecurity-after-mythos-the-jag... for a detailed analysis.
I bought a used 16 GB Intel A770 GPU for $200, and it's capable of running pretty powerful open stable diffusion and large language models.
Sure, I could get more performance out of proprietary models on much more expensive hardware, but there's diminishing returns, and consumer hardware and open models keep getting better.
I don't think the big investments into hosting models will pay off, especially as the base-line capabilities of integrate GPUs become enough to run a good model at home.
I had a similar thought, though not as extreme, the second they started nerfing and filtering models
Their intensions were good, they always are, but the minute you decide to nerf something powerful for someone, it means someone out there has access to the full blown, unnerfed version
Which means there are powerful people out there using AI in ways or for activities in which you will never be allowed to anyway
Maybe not. You stated "Every country will pursue it as far as it can, and given the multipolar world we are back in, and our recent record with treaties and commitments, I do not believe there will be global alignment on risk reduction" and that is true. While Anthropic may hold of releasing Mythos I don't think they will for long. As long as there is someone somewhere in the world that releases a competing model then Anthropic will be forced to release Mythos. This also assumes that its not a marketing tactic from Anthropic in the first place, build suspense before releasing it.
I would like to see more countries capable of producing frontier models. At the moment we have two in the world but many countries are building their own national models and AI infrastructure and may join the race.
Having a multipolar world may actually result in more freedom in gaining access to frontier models.
It's long been conventional wisdom that you shouldn't write your own crypto libraries - leave that to experts. But excellent open source libraries are available, which do get reviewed by experts. And if you're willing to study, maybe you can learn enough about cryptography to become one of the experts?
I'm wondering what other security-sensitive software that might become true of in the era of Mythos-or-better AI's?
There will still be open source projects that anyone could learn enough to contribute to, but maybe starting from scratch and writing your own becomes less feasible if you aren't attracting enough attention to get attention from people with access to the best AI's?
For example, Linux patches are going to get expert reviews, but maybe your homegrown OS won't?
Phenomenal analysis. I am actually going to start working on solutions for this. I am not willing to risk my kids becoming serfs because of misguided leadership at the foundational labs.
I interpreted their "too dangerous to release" comment as a statement about the current situation. If the model is truly as capable as they say, and the security issues so numerous, it makes sense to hold out until the biggest targets have been patched.
It'd only take one company deciding to not worry about safety, to change the calculus back to "we have to release this to stay competitive".
I think we are going to look at the era between 2019-2025 as a very rare blip in the history of public AI access. Regardless of whether fears about Mythos end up being justified, the clear trend is
1. AI models are becoming better and better at causing massively disruptive effects, leaving up larger and larger liabilities, especially as laws and regulations are being passed/proposed which would put the responsibility of some mass disruption/hacking event on the company which serves the model that made it possible
2. The relative advantage of serving an AI model for inference in exchange for money is waning compared to the advantage of using that model internally for purposes which accrue money/power/leverage for that AI company. Why serve a model at 30 dollars/million tokens when you've discovered you can use that model to run a simulated Quant firm with a net profit of 300 dollars/million tokens? Why offer the model to companies so they can find zero-day exploits, when you can find them yourself and sell the discovery to companies which would may millions to avoid this exploit being taken advantage of?
3. Why serve models so another wrapper company like Cursor can make billions off your tokens, and then try to train their own models as fast as possible, trained on your outputs so they aren't dependent on you? The entire AI startup industry and like 90% of YC batches depend on being able to serve frontier models at a profit, mediated through some wrapper, why can't OpenAI/ANthropic, once their models are good enough to handle the ideation/organizational problem, become their own incubator for thousands of AI run startups, running on models way better than the public has access to?
As a consequence, there is less and less incentive over time to offer models as an API to the public.
if it is an actual leap, expect more 90% is good enough open weights to hit the next 6 months. Theres no reason competitors will hold underperforming models if they have to retool.
might as well do more marketing.
That dream was always a lie. But in the past, people could
purchase more in parity. You only need to look at income
versus housing cost in, say, Canada.
Realistically there should not exist any superrich, but
this seems hard to change. That means there needs to be
a different society be given as promise. Other countries
manage that. In the USA they have the orange oligarch who
said a while ago how there is no money for health care
because he has to invade countries and wage war. So much
for the "no more wars" promise.
I doubt that Mythos is just so wonderful and not able to be replicated that we are actually being 'closed off' from frontier models.
For example, the people who Anthropic "trusts" with this "dangerous" model are a handful of fortune 500 companies? Seriously? Those are the people we trust?
We are going to have access to this within 6 months, and if we don't, someone else will offer an equivalent. Anthropic hasn't walked to the edge of the abyss only to be like "let the CEO's handle this!"
mythos has not been demonstrated doing anything dramatically different than other models. so as other comments say: very premature.
but the basic premise (shared among a lot of ai-doomers and ai-shamers), is that the bigs have somehow raped society (by training on everything available). this needs to be challenged: it implies quite a strong model of IP ownership, which is not what appears in law, or in founding documents (which are quite different from current law).
> The Anthropic Mythos announcement is the first time in my life I’ve felt truly poor. Maybe because I grew up on the internet and it was the one permissionless place where you could have leverage and a shot at uncapped exploration and ambition. That is now changing with the gap between models that are publicly available vs those reserved for the already wealthy and pre-established.
The Internet was developed by the US state sector and handed off to the private sector in the 90’s. Then it worked as an open space until it didn’t any more. Predictably driven by corporate interests.
> In 1893, Frederick Jackson Turner argued that much that is distinctive about America was shaped by the existence of free land to the West where anyone could start over, and that this condition infused America with its characteristic liberty, egalitarianism, rejection of feudalistic hierarchy, self-sufficiency, and ambition.
A more asinine comparison could not have been picked.
It's not clear to me if the author talk about European invasion as the colonization pattern behind pretended American frontier, as it was lands that never any human had reached before.
135 comments
Give it a few months and it will be just another model they are selling, but the NEWER model is just too powerful for the public.
Their writing about the model so far does say this is an issue where, for instance, you can't really use Mythos for interactive coding because it's so slow. You have to give it some work, go home, sleep, come in the next day and then maybe it'll have something for you.
All the AI labs and startups are still losing money hand over fist. Launching Mythos would require it to be priced well above current models, for a much slower product. Would the majority of customers notice the difference in intelligence given the tasks they're setting? If the answer is no, it's not economic to launch.
Really, I'm surprised they've done Mythos. Maybe they just wanted to exploit access to larger contiguous training datacenters than OpenAI, but what these labs need isn't smarter models, it's smaller and cheaper models that users will accept as good enough substitutes (or more advanced model routing, dynamic thinking, etc).
One thing to compare to would be what’s been paid for bug bounties in the past.
[0]: https://openai.com/index/better-language-models/
I think it's a reasonable choice to make given that Mythos actually does have cyber capabilities on that level. We already have evidence that large-scale scams are being perpetuated using AI models (such as AI video being passed as real, people deepfaking themselves in job interviews).
If you've noticed your new model can be trivially pointed at some open-source codebase with a prompt and harness that amounts to "find as many exploits as possible" and your results are non-trivially substantial and beyond what existing models can do given the same initial parameters, then a gated rollout seems the most reasonable option.
That is, Mythos will make it much easier to find lurking zero days, so just like responsible disclosure requires a security researcher to notify the software author first and give them some time to patch, giving critical infrastructure folks at least some time to analyze and patch systems seems reasonable to me.
This whole "this model is too dangerous" ploy originated from (in my opinion severely misguided) activists who wanted to stop or slow AI development down as much as possible, spreading outlandish Doomsday scenarios wherever they could.
These online-first activists have always been a key driver of the success of the very thing they fight. They share the offending thing among themselves, making it go viral in process, and soon baiting these groups is the best marketing imaginable.
There were some rather interesting studies made on the subject around 2011, I particularly remember one made by Swedish jeans brand cheap Monday, but i can't find it now.
> This feels really premature. The announcement was a week ago. The “this model is too powerful for the general public” sounds like marketing to me.
Anthropic was born out of the idea that they feel paternity over humanity. They believe by limiting access they are performing a necessary pillar of security in multiple facets.
I think it's up to the public, and articles like this are part of the public's voice, whether this belief is serious or not and secondarily whether it's okay to even posture this kind of belief since it inherently results in marginalizing the many and rewards an already very successful few.
For me, the seeming majority optimism and acceptance of “mythos’” as yet untold capabilities is betrayed as not real by the fact that one can’t react to it with the same reverence while framing it as a downside without being told “it’s not even out yet”.
“It’s not even out yet” should apply to both situations or neither.
>The “this model is too powerful for the general public” sounds like marketing to me.
Is no one else suspicious that they literally called it mythos?
> The “this model is too powerful for the general public” sounds like marketing to me
I tend to agree here. Anthropic has built a reputation and now they are in a position where they can claim to have a model way more powerful than it might actually be, and by limiting its access, there won't be an independent way to test it. I'm not denying that it's not smarter than Opus, but probably it's somewhat exaggerated.
It’s the same stuff inside as all the others.
>
But it is another thing entirely to share access only with enterprise partners such as Crowdstrike, Cisco, and Microsoft, which are known to have massive security incidents regularly.The stated purpose of Glasswing is to give infra and security orgs the chance to close holes and improve their security. In that context, it seems odd to call for not providing them access with the justification being that they have security breaches sometimes.
Mythos may or may not itself be opened to the public at some point, but I would charitably expect that Anthropic plans that a future model at least as good as Mythos Preview will be, and the limited release for Mythos is intended to make that eventuality safer by having most of the existing holes patched.
Nemotron is:
1. Not just open weight, but open data (to the limits of what is feasible), open recipe, open technique
2. In the future built by a coalition of organizations coming together to build great openly developed AI.
Nemotron 3 Super is our most successful model yet. [1]
Ultra is coming soon. And then Nemotron 4.
We can afford to do this because when AI grows, NVIDIA's opportunity also grows.
[1] https://kaitchup.substack.com/p/the-fastest-and-cheapest-120...
> You can generate your own electricity with a solar panel (think local models), but most people would rather pay a utility bill. And the power company doesn’t decide, on the basis of pedigree, who is worthy of electricity. Intelligence should work similarly, where the capabilities you can access scale may scale with vetting and due process, but the presumption should be access. Add safety guardrails to restrict dangerous use; start by making them overly trigger-happy if you must, and calibrate over time. But the default should be to allow entry.
Much better than hiding it away where it can't help anyone.
So, opportunity for individuals comes from disruption. Creative destruction is good up to a point, but it results from advancing capabilities. Technological advances compound and accelerate exponentially. Eventually we reach the point where any malcontent can destroy the world by snapping their fingers. At some point we need to place restrictions on the capabilities accessible to individuals. We have reached that point with nuclear weapons, and I think it is sensible to believe that AI is reaching that point as well.
What if this new model can start proving Millenial problems and provide insights in other fields that was not possible before?
My intuition says that a model that is as good will also be equally well aligned -- but it is still highly risky to give it to the general public because all you need is one jailbreak from bad actors.
At that point I think society would change so dramatically that "access to general public" would be a non issue. Rather, time would be spent on making abundance happen - you might think of the political struggles, economics and new ventures.
Its a bit sad that democratised access is not provided because of negative sum possibilities like cybersecurity.
Sure, I could get more performance out of proprietary models on much more expensive hardware, but there's diminishing returns, and consumer hardware and open models keep getting better.
I don't think the big investments into hosting models will pay off, especially as the base-line capabilities of integrate GPUs become enough to run a good model at home.
Their intensions were good, they always are, but the minute you decide to nerf something powerful for someone, it means someone out there has access to the full blown, unnerfed version
Which means there are powerful people out there using AI in ways or for activities in which you will never be allowed to anyway
So yeah, this is just more of the same
I would like to see more countries capable of producing frontier models. At the moment we have two in the world but many countries are building their own national models and AI infrastructure and may join the race.
Having a multipolar world may actually result in more freedom in gaining access to frontier models.
I'm wondering what other security-sensitive software that might become true of in the era of Mythos-or-better AI's?
There will still be open source projects that anyone could learn enough to contribute to, but maybe starting from scratch and writing your own becomes less feasible if you aren't attracting enough attention to get attention from people with access to the best AI's?
For example, Linux patches are going to get expert reviews, but maybe your homegrown OS won't?
It'd only take one company deciding to not worry about safety, to change the calculus back to "we have to release this to stay competitive".
1. AI models are becoming better and better at causing massively disruptive effects, leaving up larger and larger liabilities, especially as laws and regulations are being passed/proposed which would put the responsibility of some mass disruption/hacking event on the company which serves the model that made it possible
2. The relative advantage of serving an AI model for inference in exchange for money is waning compared to the advantage of using that model internally for purposes which accrue money/power/leverage for that AI company. Why serve a model at 30 dollars/million tokens when you've discovered you can use that model to run a simulated Quant firm with a net profit of 300 dollars/million tokens? Why offer the model to companies so they can find zero-day exploits, when you can find them yourself and sell the discovery to companies which would may millions to avoid this exploit being taken advantage of?
3. Why serve models so another wrapper company like Cursor can make billions off your tokens, and then try to train their own models as fast as possible, trained on your outputs so they aren't dependent on you? The entire AI startup industry and like 90% of YC batches depend on being able to serve frontier models at a profit, mediated through some wrapper, why can't OpenAI/ANthropic, once their models are good enough to handle the ideation/organizational problem, become their own incubator for thousands of AI run startups, running on models way better than the public has access to?
As a consequence, there is less and less incentive over time to offer models as an API to the public.
OMG this generation - we can't separete the outrage from reality anymore.
Meanwhile 3000 people have died arbitrarily in Iran War -while we navel gaze.
> Even though the American dream is nearly dead
That dream was always a lie. But in the past, people could purchase more in parity. You only need to look at income versus housing cost in, say, Canada.
Realistically there should not exist any superrich, but this seems hard to change. That means there needs to be a different society be given as promise. Other countries manage that. In the USA they have the orange oligarch who said a while ago how there is no money for health care because he has to invade countries and wage war. So much for the "no more wars" promise.
For example, the people who Anthropic "trusts" with this "dangerous" model are a handful of fortune 500 companies? Seriously? Those are the people we trust?
We are going to have access to this within 6 months, and if we don't, someone else will offer an equivalent. Anthropic hasn't walked to the edge of the abyss only to be like "let the CEO's handle this!"
It is simply not the edge of the abyss.
> already happening with recursive self improvement
Are any AI labs claiming this?
but the basic premise (shared among a lot of ai-doomers and ai-shamers), is that the bigs have somehow raped society (by training on everything available). this needs to be challenged: it implies quite a strong model of IP ownership, which is not what appears in law, or in founding documents (which are quite different from current law).
> The Anthropic Mythos announcement is the first time in my life I’ve felt truly poor. Maybe because I grew up on the internet and it was the one permissionless place where you could have leverage and a shot at uncapped exploration and ambition. That is now changing with the gap between models that are publicly available vs those reserved for the already wealthy and pre-established.
The Internet was developed by the US state sector and handed off to the private sector in the 90’s. Then it worked as an open space until it didn’t any more. Predictably driven by corporate interests.
> In 1893, Frederick Jackson Turner argued that much that is distinctive about America was shaped by the existence of free land to the West where anyone could start over, and that this condition infused America with its characteristic liberty, egalitarianism, rejection of feudalistic hierarchy, self-sufficiency, and ambition.
A more asinine comparison could not have been picked.