This "screenshot -> refine loop" is a great strategy and I have built it into my 3D Modeling product as well[0], but had to disable it because it would often quadruple the costs and the product is already expensive.
I am on standby to enable it though, just need a price to drop a bit more!
My late maternal grandfather was Slovenian, so I enjoyed your project's backstory. I've mucked around with ChatGPT and OpenSCAD so can identify with that also. Great concept and best of luck!
> Instead of expecting it to understand my requests, I almost always build tooling first to give us a shared language to discuss the project.
This is probably the key. I’ve found this to be true in general. Building simple tools that the model can use help frame the problem in a very useful way.
>I still occasionally hand write code in NeoVim on the bits I care the most about (CSS, design and early architecture like API patterns)
I find it amazing how people's opinions differ here. This is the first stuff I'd trust to Claude and co. because it is very much in-distribution for training data. Now if I had sensitive backend code or a framework/language/library that is pretty new or updated frequently, I'd be much more cautious about trusting LLMs or at least I would want to understand every bit of the code.
Just yesterday I used Claude to great effect in FreeCAD to model a church tower. The tower has a square base and an octagonal top, but connecting the two by creating a loft using the GUI in FreeCAD results in a wrong and ugly abomination.
Claude understood the problem and produced elegant Python code that worked perfectly the first time.
So I continued and described the other features of the tower to Claude, who coded them.
It's sometimes difficult to properly describe what you want in English, and Claude does a lot of thinking, and sometimes goes deep into a wrong direction of which it won't come out easily; but in the end the result is almost perfect.
My mind-blown moment was when I was doing work like this and Claude started positioning the camera itself to get better looks at areas it wanted to improve.
Great article. I've been trying to achieve something similar with a Revit. It's an old CAD application for Windows which means there's a few additional hurdles in exposing a cli interface that allows the LLM to drive it. However, once that is done, the loop of "write code, take a screenshot, repeat" works pretty well.
43 comments
I am on standby to enable it though, just need a price to drop a bit more!
[0]: https://grandpacad.com
> Instead of expecting it to understand my requests, I almost always build tooling first to give us a shared language to discuss the project.
This is probably the key. I’ve found this to be true in general. Building simple tools that the model can use help frame the problem in a very useful way.
>I still occasionally hand write code in NeoVim on the bits I care the most about (CSS, design and early architecture like API patterns)
I find it amazing how people's opinions differ here. This is the first stuff I'd trust to Claude and co. because it is very much in-distribution for training data. Now if I had sensitive backend code or a framework/language/library that is pretty new or updated frequently, I'd be much more cautious about trusting LLMs or at least I would want to understand every bit of the code.
Claude understood the problem and produced elegant Python code that worked perfectly the first time.
So I continued and described the other features of the tower to Claude, who coded them.
It's sometimes difficult to properly describe what you want in English, and Claude does a lot of thinking, and sometimes goes deep into a wrong direction of which it won't come out easily; but in the end the result is almost perfect.