> While I’m certain that this technology is producing some productivity improvements, I’m still genuinely (and frustratingly) unsure just how much of an improvement it is actually creating.
I often wonder how much more productive I'd be if just a fraction the effort and money poured into LLMs was spent on better API documentation and conventional coding tools. A lot of the time, I'm resorting to using an AI because I can't get information on how the current API of some-thing works into my brain fast enough, because the docs are non existent, outdated, or scattered and hard to collate.
This is facts. All of this talk about putting agent skills directly into repos (as Markdown!) is maddening. "Where were LITERALLY ALL OF YOU whenever the topic of docs as code came up?"
This is doubly maddening with NotebookLMs. They are becoming single sources of knowledge for large domains, which is great (except you can't just read the sources, which is very "We will read the Bible to you" energy), but, in the past, this knowledge would've been all over SharePoint, Slack, Google Drive, Confluence, etc.
I've chose to embrace the silver lining where there is now business backing to prioritize all the devx/documentation work because it's easier to quantify the "value" because LLM sessions provide a much larger sample size than inconsistent new hire onboarding (which was also a one-time process, instead of per session).
I do think people are going way overboard with markdown though, and that'll be the new documentation debt. Needs to be relatively high level and pointers, not duplicate details; agents can parse code at scale much faster than humans.
> Where were LITERALLY ALL OF YOU whenever the topic of docs as code came up?
Docs as code is still writing and not coding. Those are simply different skills. As programmers, we find coding to be fun and glamorous and writing to be difficult. Emotionally, it's much easier to finish a piece of code and feel genuinely happy with it (you are proud of your achievement) than it is to write a paragraph of docs and feel genuinely happy with it (you can feel in your bones that it's not good but you don't know how to improve it and you just want it over with). We have not built anywhere near the level of skill for writing than we did for coding when we wrote our own little programs for ourselves and never built a habit of thinking about how other people would interact with our code.
(For me, this is exacerbated by having been more isolated from other people than the average population, partly due to neurodivergence and partly because the hobby was niche at the time, and I assume this is also true of a lot of people currently employed as professional programmers.)
Haha indeed. At work suddenly documentation and APIs are important, but it's all for/behind "skills". Before it was always "sure, that would be nice"...
I do welcome the improvements to doc and APIs this brings though!
Yeah, I joined a project a couple of months ago, felt completely lost.
Last week, a colleague finally added for Claude all the documentation I'd have needed on day one. Meanwhile, I'm addressing issues from the other direction, writing custom linters to make sure that Claude progressively fixes its messes.
I feel like Google search results have gotten tremendously worse over the past 2 years too. It's almost like you have to use AI search to find anything useful now.
Which of course reduces traffic to sites and thus the incentives to create the content you're looking for in the first place :(
> I often wonder how much more productive I'd be if just a fraction the effort and money poured into LLMs was spent on better API documentation and conventional coding tools.
Probably negligible. It's not a problem you can solve by pouring more money in. Evidence: configuration file format. I've never seen programmers who enjoy writing YAML. And pure JSON (without comments) is simply not a format should be written by humans. But as far as I know even in the richest companies these formats are still common. And the bad thing they were supposed to replace, XML config, was popularized by rich companies too...!
Yeah I get this impression too. AI feels like it's papering over overwrought and badly designed frameworks, tech stacks with far too many things in them, and also the decline of people creating or advocating for really expressive languages.
Pragmatic sure, but we're building a tower of chairs here rather than building ladders like a real engineering field.
As someone who does broad activities, it supercharges a lot of things. Having a critical eye is required though. I estimate 40%-60% improvements on basic coding tasks.
And hilariously, the worst offenders are AI frameworks themselves. A couple months ago I was helping a client build out some "agentic" stuff and we switched from OpenAI Agents library to Agno. Agents is messy enough, like making inconsistent use of its own enums etc, but with Agno you can really feel that they are eating their own dog food. Plenty of times I literally could not find the API for some object, and of course their docs page pushes you toward chatting with their goddamn docs chatbot, which barfs up some outdated function signature for you.
> better API documentation and conventional coding tools
Agreed, and it depends on the language I suppose. I'm a C++ developer and when you start working with templates even at a non-casual level, the compiler errors due to either genuine syntactic errors or 'seems correct but the standard doesn't support' can be infuriatingly obtuse. The LLM 'just knows' the standard (kind of, all 2k pages), and can figure out and fix most of those errors far faster than I can. In fact one of my preferred usages is to point Codex at my compiler output and get it to do nothing more than fix template errors.
Kotlin, for example, is much more in your face, in the IDE which does a correctness pass, before you even invoke the compiler (in the traditional sense) and the language spec is considerably leaner with less (no?) UB, unlike C++.
I agree. I think of AI as a search engine on steroids.
But I think it IS the best way to search for information, to be able to put a question in natural language. I'm always amazed just how exactly on-point the answer is.
I mean even the best of docs out there that have a great search bar like the Vue docs still only matches your search term and surfaces relevant topics.
> To what degree did I expand scope because I knew I could do more using the AI?
Someone at work recently termed this “Claude Creep”. It’s so easy to generate things push you towards going further but the reality is that’s you’re setting yourself up for more and more work to get them over the line.
> I’ve had the idea that from a social perspective it’d be regarded like plastic surgery, in that it only looks weird when its over-done, or done badly.
The biggest positive I have seen is not so much in the new tools, but in new ways to convince the higher ups to do sensible things.
We always find that small teams of locals can do much much more than a team with an unlimited number of low cost "developers". Not just because the competence of low cost devs is poor, but also the structure of how you work changes for the worse with a bigger team, for the worse with a distributed team, and for the worse with a skill-diverse team.
Thats before you get into the cultural flaws of favored destinations like India.
So we have been able to argue things like add one local + ai is better than about 20-100 Indians, depending on role and business structure needed to manage low-competence low-trust "developers". So we are planning to completely on-shore in the near future.
The bean counters are happy, and the quality of the work is improving.
All my colleagues that feel super productive pushing code written by LLMs share the same traits: they never really cared much about quality. And nowadays nobody wants to review their code/docs because it’s just painful. Heck, not even they are reviewing their supposedly own stuff.
PRs with dozens of changes the author doesn’t even understand? Won’t touch that. An RFC of 30 pages of which the author didn’t write a single line? Won’t read that
And they write hundreds of MD files (for skills) that never break. No sense of accountability.
I would agree with the utility of Claude and Claude Code. Claude feels like your own executive assistant, sales team and IT department. Combine that with Claude Code and you can build some incredible things. Myself as an example, I used Claude to advise me on starting a business and building a MVP. After a few weeks of refinement I was able to create something I never could have done without Claude. It is a game changer for sure.
> (The) Output was coherent but its ‘style’ was very boring and overtly inoffensive, which was (and still is) a clear limitation of the technology.
The style isn’t a limit of the technology, it’s a limit of the lobotomized models from OpenAI and Anthropic. The open source community has lots of models that are great at creative writing.
The section about being "glazed" into action resonates. Hidden within this concept I think is something profound about human motivation, innuendo and all.
> AI generated prose is at best boring, and at worst genuinely unappealing. I’m continually tempted, because in theory it should work well. The AI has perfect spelling and grammar, has more than enough context to produce article-length content, and can do in seconds what takes me hours.
I have a thesis in mind...that there is something fundamental to the human spirit that relishes a sort of friction that LLMs cannot observe or reproduce on their own.
A big part of the benefit of AI has nothing to do with AI and everything to do with leading point haired bosses around. They won't approve needed refactorings but promise to integrate AI and suddenly budget is no problem, just add an easily removable chatbot afterward and you're golden.
The stupid thing is that instead of using AI to give ourselves 1 hour work days, we’re just cramming more work into the same amount of time we’ve always worked.
Generating AI Content sucks, Consuming AI Content sucks, but combine them in the same loop and it's really addicting. AI Content Prosuming rocks.
Since LLMs, if I see a video I think is interesting, I take the transcript, feed it into an LLM, I summarize it and ask it a couple of questions.
I've turned 12 minute videos back into the 5 phrases news it was based on.
I suppose that when you're the one generating the request, it feels more personal. It is also very interesting that most LLMs respond like a normal person when you talk to them directly, but suddenly adopt the more annoying blogger speech patterns when you tell them 'create content'.
The Gartner hype cycle has 5 phases: tech trigger (6 months - 2 years), peak of inflated expectations (6 months - 2 years ), the slope of enlightenment (2 - 5 years), and the plateau of productivity (5+ years), and the slope of decline (Obsolescence which noone talks about). If we are in fact at the 40th month then we are either approaching the peak of inflated expectations, the slope of enlightenment, or the plateau of productivity. I would say we are probably approaching the peak of inflated expectations. We are constantly hearing the symptoms of the 'This Time is Different' Syndrome from people saying the old rules don’t apply which is the classic sign the peak is approaching. The average financial bubble bursts after 3 years, however the dot-com bubble burst 5 years after peak and the housing bubble took 3-4 years. We are probably in the “bubble mania” phase right now because of all the irrational exuberance. Ride the Lightning!
> For now at least I’m keeping my Claude Pro subscription, but given the persistant rumors of undisclosed rate limiting, and ever improving performance of local LLMs, I can easily imagine that I’ll cancel my subscription before the end of the year.
> I remember the first time I vibe-coded a small project. It was an app that generated placeholder cards for my MTG collection. I prompted the bot (now Claude, not ChatGPT).....
I would be interested what date this was? I am surprised if it's been recent that Claude didn't 1 shot this.
Do you regularly find text content that you know is AI written (but is not marked as such)? Because honestly I don't, and it must exist in decent quantity by now. Or perhaps it's still sparse?
Bro but... you now are having a business is planned by a paid chatbot, they can shutdown anytime or make it more expensive, also it is imposiable to get something new, you are copying for somewhere else, maybe what claude is copying is having a copyrights on it, like a leaked code and etc, also your brain will slowly shutdown from thinking about 'business' so you will hevaly relays on claude in the future :)
My friend is trying to do the same, the Docker stack he made for his SaaS is really amazing, it is following the standards from the ancient age.
The internet as we know it is dead. Websites will seize to exist in 2 years from now. On the fly UIs will be generated. Tailored exactly to your ad profile. Content will be either AI slop that's better than humans can ever make or products that are a perfect fit for your spendable part of your wallet
144 comments
> While I’m certain that this technology is producing some productivity improvements, I’m still genuinely (and frustratingly) unsure just how much of an improvement it is actually creating.
I often wonder how much more productive I'd be if just a fraction the effort and money poured into LLMs was spent on better API documentation and conventional coding tools. A lot of the time, I'm resorting to using an AI because I can't get information on how the current API of some-thing works into my brain fast enough, because the docs are non existent, outdated, or scattered and hard to collate.
This is doubly maddening with NotebookLMs. They are becoming single sources of knowledge for large domains, which is great (except you can't just read the sources, which is very "We will read the Bible to you" energy), but, in the past, this knowledge would've been all over SharePoint, Slack, Google Drive, Confluence, etc.
I do think people are going way overboard with markdown though, and that'll be the new documentation debt. Needs to be relatively high level and pointers, not duplicate details; agents can parse code at scale much faster than humans.
> Where were LITERALLY ALL OF YOU whenever the topic of docs as code came up?
Docs as code is still writing and not coding. Those are simply different skills. As programmers, we find coding to be fun and glamorous and writing to be difficult. Emotionally, it's much easier to finish a piece of code and feel genuinely happy with it (you are proud of your achievement) than it is to write a paragraph of docs and feel genuinely happy with it (you can feel in your bones that it's not good but you don't know how to improve it and you just want it over with). We have not built anywhere near the level of skill for writing than we did for coding when we wrote our own little programs for ourselves and never built a habit of thinking about how other people would interact with our code.
(For me, this is exacerbated by having been more isolated from other people than the average population, partly due to neurodivergence and partly because the hobby was niche at the time, and I assume this is also true of a lot of people currently employed as professional programmers.)
I do welcome the improvements to doc and APIs this brings though!
Last week, a colleague finally added for Claude all the documentation I'd have needed on day one. Meanwhile, I'm addressing issues from the other direction, writing custom linters to make sure that Claude progressively fixes its messes.
Which of course reduces traffic to sites and thus the incentives to create the content you're looking for in the first place :(
> I often wonder how much more productive I'd be if just a fraction the effort and money poured into LLMs was spent on better API documentation and conventional coding tools.
Probably negligible. It's not a problem you can solve by pouring more money in. Evidence: configuration file format. I've never seen programmers who enjoy writing YAML. And pure JSON (without comments) is simply not a format should be written by humans. But as far as I know even in the richest companies these formats are still common. And the bad thing they were supposed to replace, XML config, was popularized by rich companies too...!
Pragmatic sure, but we're building a tower of chairs here rather than building ladders like a real engineering field.
I don't bring huge codebases to it.
I was able to one-shot a parameterized SVG template creator for a laser cutter. Unlikely I could have achieved the same with 40 hours of pure focus.
> better API documentation and conventional coding tools
Agreed, and it depends on the language I suppose. I'm a C++ developer and when you start working with templates even at a non-casual level, the compiler errors due to either genuine syntactic errors or 'seems correct but the standard doesn't support' can be infuriatingly obtuse. The LLM 'just knows' the standard (kind of, all 2k pages), and can figure out and fix most of those errors far faster than I can. In fact one of my preferred usages is to point Codex at my compiler output and get it to do nothing more than fix template errors.
Kotlin, for example, is much more in your face, in the IDE which does a correctness pass, before you even invoke the compiler (in the traditional sense) and the language spec is considerably leaner with less (no?) UB, unlike C++.
But I think it IS the best way to search for information, to be able to put a question in natural language. I'm always amazed just how exactly on-point the answer is.
I mean even the best of docs out there that have a great search bar like the Vue docs still only matches your search term and surfaces relevant topics.
> To what degree did I expand scope because I knew I could do more using the AI?
Someone at work recently termed this “Claude Creep”. It’s so easy to generate things push you towards going further but the reality is that’s you’re setting yourself up for more and more work to get them over the line.
> I’ve had the idea that from a social perspective it’d be regarded like plastic surgery, in that it only looks weird when its over-done, or done badly.
We always find that small teams of locals can do much much more than a team with an unlimited number of low cost "developers". Not just because the competence of low cost devs is poor, but also the structure of how you work changes for the worse with a bigger team, for the worse with a distributed team, and for the worse with a skill-diverse team.
Thats before you get into the cultural flaws of favored destinations like India.
So we have been able to argue things like add one local + ai is better than about 20-100 Indians, depending on role and business structure needed to manage low-competence low-trust "developers". So we are planning to completely on-shore in the near future.
The bean counters are happy, and the quality of the work is improving.
And they write hundreds of MD files (for skills) that never break. No sense of accountability.
> (The) Output was coherent but its ‘style’ was very boring and overtly inoffensive, which was (and still is) a clear limitation of the technology.
The style isn’t a limit of the technology, it’s a limit of the lobotomized models from OpenAI and Anthropic. The open source community has lots of models that are great at creative writing.
The section about being "glazed" into action resonates. Hidden within this concept I think is something profound about human motivation, innuendo and all.
> AI generated prose is at best boring, and at worst genuinely unappealing. I’m continually tempted, because in theory it should work well. The AI has perfect spelling and grammar, has more than enough context to produce article-length content, and can do in seconds what takes me hours.
I have a thesis in mind...that there is something fundamental to the human spirit that relishes a sort of friction that LLMs cannot observe or reproduce on their own.
I think we'll find that for most AI stuff.
Since LLMs, if I see a video I think is interesting, I take the transcript, feed it into an LLM, I summarize it and ask it a couple of questions. I've turned 12 minute videos back into the 5 phrases news it was based on. I suppose that when you're the one generating the request, it feels more personal. It is also very interesting that most LLMs respond like a normal person when you talk to them directly, but suddenly adopt the more annoying blogger speech patterns when you tell them 'create content'.
Marketers present a list of potential problems
The smallest success stories are marketed as indicators of future success, but to verify this, one must wait patiently for the future to arrive
> For now at least I’m keeping my Claude Pro subscription, but given the persistant rumors of undisclosed rate limiting, and ever improving performance of local LLMs, I can easily imagine that I’ll cancel my subscription before the end of the year.
> I remember the first time I vibe-coded a small project. It was an app that generated placeholder cards for my MTG collection. I prompted the bot (now Claude, not ChatGPT).....
I would be interested what date this was? I am surprised if it's been recent that Claude didn't 1 shot this.
a system that can allocate all the atoms / energy better than all of mankind won't eternally exist to coddle hairless apes
My friend is trying to do the same, the Docker stack he made for his SaaS is really amazing, it is following the standards from the ancient age.
Some people push Claude and Claude Code at work, weights are closed, even Claude Code is completely closed and proprietary.
If terms, which Anthropic controls, ever change, all work and time spent on these products will be for naught.
No, I don’t think any skills or knowledge is acquired by the use of these tools can be reliably translated to another tool / model.
Hence I firmly think it’s utter nonsense rubbish to engage with that, and even Qwen3.5-Coder-Next plus OpenCode spits out working apps.
Really, do better product users.
> 40 months
Not counting from 1971s DARPA? Sorry I'm allegric when LLMs being called AI like nothing existed before it.