> You did not write a single line of SQL. You did not set up an attribution model. You asked a question, in English, and got a table.
But nobody bothered to check if it was correct. It might seem correct, but I've been burned by queries exactly like these many, many times. What can often happen is that you end up with multiplied rows, and the answer isn't "let's just add a DISTINCT somewhere".
The answer is to look at the base table and the joins. You're joining customers to two (implied) one-to-many tables, charges and email_events. If there are multiple charges rows per customer, or an email can match multiple email_events rows, it can lead to a Cartesian multiplication of the rows since any combination of matches from the base table to the joined tables will be included.
If that's the case, the transactions and revenue values are likely to be inflated, and therefore the pretty pictures you passed along to your boss are wrong.
Same way you do today; you trust whoever wrote the query.
I do not sell a wrapper on top of some LLM; you can absolutely write your SQL directly. There is an engine, there are iceberg tables. You can just live your best life doing your own SQL by hand.
Now if you couldnt do it before and you have a sensible understanding, you can likely do a bit more with the CLI tooling. And if you know a lot more, you can still do that. The queries are not hidden, or abstracted, If you need them they will be saved - transparently in SQL.
So I dont know what is the answer to the question "how do people do things they don't know how to do" ?
> So I dont know what is the answer to the question "how do people do things they don't know how to do" ?
The statue quo had been to learn SQL or ask a human you trust to check their own work, which hopefully you can reuse.
Now it's ask AIs that are intentionally a bit random, and less likely to (or incapable of) check(ing) their work. Perhaps without seeing the SQL at all, requiring to trust it for every interaction. And in a culture that moves so fast that there is no checking by any(one|thing).
I think the author should be introduced to (or reminded of) the tale of the average from the US Air Force [1]. Social reality is high-dimensional and the "normal" thing is actually to be average in some dimensions, but strongly non-average in many others. So a "perfectly average" family would paradoxically be an outlier themselves.
I think this is important, because if his hypothesis is right, then LLMs behave differently here: They really are average in all dimensions. They are the pilots the Air Force thought they had before Daniels made the study.
So if he is right, we'd be changing from a mostly-non-average to a mostly-average society, which would really be a massive change - and probably not a good one IMO.
At any tech company with leveling guidelines that I have seen, promotions above mid level have never been based on “I codez real gud”. It’s always been based on scope, impact and dealing with ambiguity. It’s stated differently in different companies.
No one has ever differentiated themselves based on how good of a ticket taker they are. Coding especially on the enterprise dev side where most developers work has been being commoditized since 2016 at least and compensation has stagnated since then and hasn’t come near keeping up with inflation.
In 2016, a good solid full stack, mobile or web developer working in the enterprise could make $135K working in a second tier city. That’s $185K inflation adjusted today. Those same companies aren’t paying $185K for the same position.
My one anecdote is that the same company I worked for back then making $125K and some of my coworkers were making $135K just posted a position on LinkedIn with the same requirements (SQL Server + C#) offering $145K fully remote.
Tbh I dont really agree with your statements.
Especially with working with data, intention is key.
By using an llm, by definition, you are loosing intention.
And Thai puts you in a position where you have to 1) think of exactly what you look for. 2) able to understand what the llm generated.
You might say it "still less work" and that's true, perhaps, only for the first few times. After a while you _learn_ how to do it, and understand how to _think_ with the language of your data.
With LLMs, you never get this benefit, and also loose your ability to judge the LLM's output properly.
But again, that might be enough on your case, or, you simply don't _know_.
The real risk is not that "LLM SQL" is wrong, it is in fact that a man who asked for it can't recognize when it is wrong. Writing a query and knowing when JSON silently doubled your rows are different skills and LLMs are able to give you only one of them (I'm not sure if they really can)
Average is only a tombstone of someone having failed to do better. And settling for average means pulling down.
When it comes to bs dashboard where "average is all you need", maybe the "better than average" result would be asking yourself if it's even worth doing in the first place?
how do you know those queries are actually correct without domain knowledge?
Do you know enough about JOINs and how they work to be able to break those big queries down and figure out whether they are doing exactly what you're asking for in English?
I always find it a bit weird to see posts on the front page where all the comments disagree with the central premise of the article. In this case the post is an ad advocating for executing code you didn't write and handing the results to your manager.
It makes me wonder if Hacker News has a silent majority of people who would actually use AI in this way without wanting to admit it, and a vocal minority of people who wouldn't.
> But this is a pain, first because, if you do anything that is not selling a product online that people can buy right when they click a button, it is a drag to create those attribution models effectively: is it last click, first click, weighted attribution... who knows. Nobody knows. Everybody gives up and just adds it to a dashboard and pretends it makes sense.
Yes, thinking about your data and how to check it is so annoying. Much better to do something average, see if the result puts you in a good light, and share that insight into your company's working with ~~everyone on the internet~~ your boss.
Rarely have I seen "we help you create meaningless slop more easily" advertised so explicitly. Or is this also average?
Why average? I've always taken pride in my work and developed things that went beyond the expectations of the management and of the final users. Now I'm using LLMs a lot and I've been able to do much more than I used to- I find them great coworkers, technically very knowledgeable, patient and fast. I provide the big picture, keep an eye on the architectural soundness and code quality, and design the features. The LLM does the rest. The results are way above average.
analytics/data engineer here. The approach described here falls apart on most datasets I've seen, because the source data has folds that are almost always out of context of the data itself. Even for a typical simple question a founder might have, like "what's the revenue for product X last month". Perhaps some orders don't have a Stripe record associated, and we receive money through a separate invoicing process. Perhaps there's a high revenue breakage rate between when a purchase is originally placed and when the payment goes through, and so a naive query for point-in-time revenue will almost certainly over-count revenue. The SQL the agent generate might not even yield directionally correct answers.
And that's when the agent even manages to construct a reasonable naive query. I've seen even Opus 4.6 ignore a is_demo column in the schema it was given when asked to construct a query for the number of active users.
Where I've seen text-to-SQL work well enough is when you're pointing it at data that's already been well-modeled for analytics such that the naive query a LLM will construct is correct by default. The data is either structured as a wide table such that no joins are necessary, or all the joins are 1:1 fact <-> dimension joins. All metrics are additive and so can be aggregated without asterisks. Columns follow a consistent naming convention, using the business domain terms a user would use in their prompt to the agent.
But that's a much thinner niche that what rawquery is proposing. You can't get around the analytics engineering effort involved in constructing a quality analytics dataset; the LLM will be a best a fuzzy fronted to your data warehouse, coextent with your BI tool.
Note: I do see value in value in rawquery's CLI-first approach to accessing data. In the right hands agents are very helpful at rapidly exploring datasets and validating assumptions on source data; but all the cloud data warehouse products I've interacted are all somewhat fiddly to access locally.
It's a post claiming average AI is useful... by a for-profit "data platform with a CLI that LLM agents can use directly". What are they going to do? Criticize the whole industry they are selling to?
This tracks. Tasks that used to be a day or two of grunt work are now an hour with Claude.
And there is a lot of that type of work to do if you're trying to grow a business. But, something in there should be trying to be exceptional or else you have no moat. Claude will probably not be able to breeze through that part with the same amount of ease...
i fail to understand how text2sql on quite simple data sources is anything to write home about 3 years after it came onto the market? can someone elaborate?
where it gets interesting is when you have a custom system that your LLM surely never saw (custom ERP) that has 50 sometime cryptic tables, unclear look up tables and unexplained flags.
something no text2sql solution solved for us.
we built a second mcp that lets the agent look up business logic (generated from source code) and then does better queries. that i think is something i never read in a blog post about a text2sql solution.
Dunno mate. Recently wanted to extend life of my old average laptop and need 4GB of RAM. I mean how much 4GB of RAM can cost? checks online... 55EUR, shipping in 2 weeks...
This is not only average. This is actual magic.
So let's be real: the SQL is average. The joins are average. The chart is average. And that took us less than 5 minutes and that was amazing, that is the entire point.
You did not need a data engineer to model your HubSpot data, or a meeting to agree on whether it should be last-click or first-click or linear or time-decay or whatever.
You needed a query, written fast, on data you already own. Your LLM wrote it. You confirmed it made sense. Your manager got a link.
Honestly, average is clearly magic; prove me wrong.
I'll give it a go. This is generated slop, and the poor, factory-made quality of the writing undercuts every aspect of the argument.
102 comments
> You did not write a single line of SQL. You did not set up an attribution model. You asked a question, in English, and got a table.
But nobody bothered to check if it was correct. It might seem correct, but I've been burned by queries exactly like these many, many times. What can often happen is that you end up with multiplied rows, and the answer isn't "let's just add a DISTINCT somewhere".
The answer is to look at the base table and the joins. You're joining customers to two (implied) one-to-many tables, charges and email_events. If there are multiple charges rows per customer, or an email can match multiple email_events rows, it can lead to a Cartesian multiplication of the rows since any combination of matches from the base table to the joined tables will be included.
If that's the case, the transactions and revenue values are likely to be inflated, and therefore the pretty pictures you passed along to your boss are wrong.
Further reading, and a terrific resource:
https://kb.databasedesignbook.com/posts/sql-joins/#understan...
I do not sell a wrapper on top of some LLM; you can absolutely write your SQL directly. There is an engine, there are iceberg tables. You can just live your best life doing your own SQL by hand.
Now if you couldnt do it before and you have a sensible understanding, you can likely do a bit more with the CLI tooling. And if you know a lot more, you can still do that. The queries are not hidden, or abstracted, If you need them they will be saved - transparently in SQL.
So I dont know what is the answer to the question "how do people do things they don't know how to do" ?
> So I dont know what is the answer to the question "how do people do things they don't know how to do" ?
The statue quo had been to learn SQL or ask a human you trust to check their own work, which hopefully you can reuse.
Now it's ask AIs that are intentionally a bit random, and less likely to (or incapable of) check(ing) their work. Perhaps without seeing the SQL at all, requiring to trust it for every interaction. And in a culture that moves so fast that there is no checking by any(one|thing).
Modern models are quite capable at surfacing and validating their assumptions and checking correctness of solutions.
Oversight helps you build confidence in the solutions. Is it perfect, no.. but way better then most engineers I also ask to check things.
I can write that script faster than I can write the text asking the AI to write the script as SQL is concise and my IDE has auto-complete.
I think this is important, because if his hypothesis is right, then LLMs behave differently here: They really are average in all dimensions. They are the pilots the Air Force thought they had before Daniels made the study.
So if he is right, we'd be changing from a mostly-non-average to a mostly-average society, which would really be a massive change - and probably not a good one IMO.
[1] https://noblestatman.com/uploads/6/6/7/3/66731677/cockpit.fl...
Why didn’t the boss ask the AI for the charts to begin with?
Everyone’s income is going to be below average, because they got fired.
No one has ever differentiated themselves based on how good of a ticket taker they are. Coding especially on the enterprise dev side where most developers work has been being commoditized since 2016 at least and compensation has stagnated since then and hasn’t come near keeping up with inflation.
In 2016, a good solid full stack, mobile or web developer working in the enterprise could make $135K working in a second tier city. That’s $185K inflation adjusted today. Those same companies aren’t paying $185K for the same position.
My one anecdote is that the same company I worked for back then making $125K and some of my coworkers were making $135K just posted a position on LinkedIn with the same requirements (SQL Server + C#) offering $145K fully remote.
You might say it "still less work" and that's true, perhaps, only for the first few times. After a while you _learn_ how to do it, and understand how to _think_ with the language of your data. With LLMs, you never get this benefit, and also loose your ability to judge the LLM's output properly.
But again, that might be enough on your case, or, you simply don't _know_.
When it comes to bs dashboard where "average is all you need", maybe the "better than average" result would be asking yourself if it's even worth doing in the first place?
Do you know enough about JOINs and how they work to be able to break those big queries down and figure out whether they are doing exactly what you're asking for in English?
It makes me wonder if Hacker News has a silent majority of people who would actually use AI in this way without wanting to admit it, and a vocal minority of people who wouldn't.
> ninety percent of everything is crud
https://en.wikipedia.org/wiki/Sturgeon%27s_law
> But this is a pain, first because, if you do anything that is not selling a product online that people can buy right when they click a button, it is a drag to create those attribution models effectively: is it last click, first click, weighted attribution... who knows. Nobody knows. Everybody gives up and just adds it to a dashboard and pretends it makes sense.
Yes, thinking about your data and how to check it is so annoying. Much better to do something average, see if the result puts you in a good light, and share that insight into your company's working with ~~everyone on the internet~~ your boss.
Rarely have I seen "we help you create meaningless slop more easily" advertised so explicitly. Or is this also average?
And that's when the agent even manages to construct a reasonable naive query. I've seen even Opus 4.6 ignore a
is_democolumn in the schema it was given when asked to construct a query for the number of active users.Where I've seen text-to-SQL work well enough is when you're pointing it at data that's already been well-modeled for analytics such that the naive query a LLM will construct is correct by default. The data is either structured as a wide table such that no joins are necessary, or all the joins are 1:1 fact <-> dimension joins. All metrics are additive and so can be aggregated without asterisks. Columns follow a consistent naming convention, using the business domain terms a user would use in their prompt to the agent.
But that's a much thinner niche that what rawquery is proposing. You can't get around the analytics engineering effort involved in constructing a quality analytics dataset; the LLM will be a best a fuzzy fronted to your data warehouse, coextent with your BI tool.
Note: I do see value in value in rawquery's CLI-first approach to accessing data. In the right hands agents are very helpful at rapidly exploring datasets and validating assumptions on source data; but all the cloud data warehouse products I've interacted are all somewhat fiddly to access locally.
It's a post claiming average AI is useful... by a for-profit "data platform with a CLI that LLM agents can use directly". What are they going to do? Criticize the whole industry they are selling to?
And there is a lot of that type of work to do if you're trying to grow a business. But, something in there should be trying to be exceptional or else you have no moat. Claude will probably not be able to breeze through that part with the same amount of ease...
where it gets interesting is when you have a custom system that your LLM surely never saw (custom ERP) that has 50 sometime cryptic tables, unclear look up tables and unexplained flags.
something no text2sql solution solved for us.
we built a second mcp that lets the agent look up business logic (generated from source code) and then does better queries. that i think is something i never read in a blog post about a text2sql solution.
It is like nails on a chalkboard.