Most of this discourse feels like some kind of religious ritual built on the foundation of authority bias. Where is the evidence that skills improves performance over any other methodology outside of the fact of its nascent popularity?
I do agree with Jacques Ellul in Technological Society that technique precedes science, and that's certainly the case with LLMs; however, this whole industry waves off rigorous validation in favor of personal anecdotes ("it feels more productive to me!" "they didn't study after Opus 4.5 was released").
The difference I'm noticing is in that with a proper skill you can skip the process of LLM wandering about and trying to guess how to interact with an API or else.
so they basically just save you time, even if they are 50% efficient of what it COULD be
Thinking about what manzanarama mentioned regarding checking skills into a repo, that's exactly how I look at it. To me, these skills are just specialized configuration. We've been versioning configuration for decades because it's critical to reproducibility and maintainability.
The "cognitive load" problem latand6 raised for reviewing every skill is real. That's where you integrate security and quality gates – treat these skills like any other software artifact. You wouldn't manually review every line of every dependency, so automate the validation here too.
The distribution problem is harder than it looks because it's actually a composition problem in disguise. A single skill is trivially shareable — zip it, gist it, whatever.
But in practice you end up with skills that depend on other skills, or a skill that assumes specific instructions are already loaded, conflicting skills, versioning and supply chain issues - and suddenly you need dependency resolution.
I've built a package-manager approach for this (APM - github.com/microsoft/apm) and the thing that surprised me most was how quickly even small teams end up with config sprawl - and how much a manifest that travels with the project helps.
The "too small for a repo" thing is real, but one pattern that works is having a monorepo per dev team or org with all skills and building jointly over there.
Yeah, I've built my own skill-package manager as well btw! Then it clicked and I hyperfocused for a whole week and vibecoded a skill marketplace haha. Because the selling of a skill seems to sound like a new idea.
going to write a show HN about that maybe
Idk I just never used skills. I have an architecture md (kind of like a skill in that it’s static? It always reads this first), and todos for every single ticket the AI goes through and does it. I have like 90% plus success.
Yeah, that's the default way to approach it, but the cognitive load is still a problem. Manually reviewing every single skill I might need just for one task is tedious. GitHub stars won't give you any useful signal. I actually started building a product that solves this.
13 comments
I do agree with Jacques Ellul in Technological Society that technique precedes science, and that's certainly the case with LLMs; however, this whole industry waves off rigorous validation in favor of personal anecdotes ("it feels more productive to me!" "they didn't study after Opus 4.5 was released").
so they basically just save you time, even if they are 50% efficient of what it COULD be
The "cognitive load" problem latand6 raised for reviewing every skill is real. That's where you integrate security and quality gates – treat these skills like any other software artifact. You wouldn't manually review every line of every dependency, so automate the validation here too.
But in practice you end up with skills that depend on other skills, or a skill that assumes specific instructions are already loaded, conflicting skills, versioning and supply chain issues - and suddenly you need dependency resolution.
I've built a package-manager approach for this (APM - github.com/microsoft/apm) and the thing that surprised me most was how quickly even small teams end up with config sprawl - and how much a manifest that travels with the project helps.
The "too small for a repo" thing is real, but one pattern that works is having a monorepo per dev team or org with all skills and building jointly over there.
That’s what I do for each ticket.
Idk I just never used skills. I have an architecture md (kind of like a skill in that it’s static? It always reads this first), and todos for every single ticket the AI goes through and does it. I have like 90% plus success.