There was a saying that before learning postgres in depth, the db is just a dumb store of data for devs, once you spend time to learn the tools it provides though, most applications just look like a very thin layer on top of the sql.
There is so much more to rdbms (especially pg) than just joins - common table expressions, window functions, various views, let alone all the extensibility - extensions, custom types, even enums.
All of that can enable writing performant, type safe and very compact applications.
I am yet to see libs that embrace the elegance of it all - I’ve attempted this once - https://github.com/ivank/potygen but didn’t get much traction. I’m now just waiting for someone more determined to pick up those ideas - a client lib that exposes the type safety and intellisence at compile time and allows you to easily compose those sql queries.
I think this project has some ways to go to reach that though, but thankfully it is a step in the right direction.
If you're into JS there was an query builder posted a few days back that was fairly closely mapped to SQL, it in turn was inspired by Linq (and EFCore behind it).
I didn't look forward to working with C# initially but Linq has been a fresh air since your statements more or less map 1:1 to SQL and probably quite overlooked because of "Microsoft"(and that Linq with old EF could have nasty surprises).
What people don't know/realize is that because Linq expressions in C# are left "uncompiled" they can be passed to the SQL layers and converted to idiomatic SQL, so you have all the typesafety of C# and regular C# code but get SQL code that is executed on the server (there is some minor impedance mismatch but it's minor enough and mainly with strings).
We use jOOQ, and love it. But sqlc also makes nice trade offs. I see it does Kotlin generation: if this lib was around when we picked jOOQ, I's certainly had considered it.
Differences:
* jOOQ is an eDSL with an optional schema-to-classes generator
* you write jOOQ queries in Java (or Kotlin as we do)
* there's quite a bit of type-safety added when using the generator: the schema needs to be match the queries you write or you get compile errors
* jOOQ queries are built are run time adding a little overhead that sqlc does not
* writing jOOQ is very close writing SQL (a very thin abstraction), sqlc is "just SQL" it seems
That's a really nice project indeed. We looked at it I remember, when we pciked jOOQ, but it was too new to bet the farm on back then. It progressed really nicely.
Building on top of IntelliJ's parser and 'PSI' stack as a headless engine sounds slightly mad at first but seems to provide a lot of leverage, as demonstrated by the number of dialects the team is able to support via mixins. By contrast sqlc appears to require a lot of code to support each additional dialect.
Whenever I write a backend, it's a thin layer on top of the RDBMS like you said. I don't know if a lib or framework could help with this. It's more about designing the schema well, avoiding excessive tooling (ORMs, query builders, etc), not trying to abstract away the DB, and writing ample integration tests.
If you get that stuff out of the way, you can focus on the real problems like design, xact isolation, and performance, for which there's tons of conflicting advice rather than an agreed-upon approach. And then there's sharding. It's hard enough already.
Personally I haven't found the need for type safety in code, or even the code-SQL boundary. The DB tables have types, as does my OpenAPI or Protobuf or whatever API spec. That's basically everything already. If something slips past my tests, it's because the tests are bad, and stronger typing wouldn't have helped.
Ah, we have `openapi-generator` already (look it up) and it works well enough. I plan on starting to use OpenAPI only for types and be able to generate structs and various other types in several languages -- fingers crossed.
You may be interested in https://docs.postgrest.org/en/v12/ an automatic way of creating a REST API from the database. It even uses the database to do authentication which is very rarely seen in production (CREATE ROLE, GRANT SELECT, etc). And you create functions in the database to customize things including implementing custom RPC methods when the standard REST is insufficient.
Many devs however would consider creating functions and stored procedures a bridge too far to cross.
I totally agree and think it's because polyglot programming still kinda sucks. Either you use a universally frustrating ORM or dataframe library or you pass your db strings that even your most integrated IDEs can't really integrate into the rest of your code.
If we figured out how to do type inference and go-to-definition and all the other nice LSP stuff across language boundaries in a good way, I'd hope the things you mention would all get a lot more widely used.
> Unfortunately, plrust extension is responsible for many gigabytes of artifacts in the image. Typically, if you need Rust, this is not a big problem as both development machines and servers can handle this just fine.
Lmao. At least they provide a "slim" flavor without it.
Current image builds are to be phased out soon(ish). They were a way to get something into people's hands. And yes, plrust/rust are definitely space hogs :)
You missed the ole' reliable, temp tables. I always end up using them heavily for non-trivial analytic queries since it gives you so much control over query execution.
In terms of actually using SQL more effectively, I think the ideal is just a small utility to use reflection and map a ResultSet (or equivalent) into a strongly typed object using reflection.
Even inserts can get tricky because there are so many different knobs you can tune. It's pretty rare that people just need to insert into a single table with no additional selects beforehand which means there is room to play around with preemptive locking, isolation level, etc.
It doesn't seem to be doing what I meant. I mean typing compile time and most importantly while using the IDE. Without having to generate types manually or importing them from a generated file.
I have probably misunderstood the documentation, the built-in sql tagged template function seems to be providing static type safety using conditional types magic. But, yeah, maybe I am too optimistic about this. Well, too bad then.
There is so much more to rdbms (especially pg) than just joins - common table expressions, window functions, various views, let alone all the extensibility - extensions, custom types, even enums.
All of that can enable writing performant, type safe and very compact applications.
I am yet to see libs that embrace the elegance of it all - I’ve attempted this once - https://github.com/ivank/potygen but didn’t get much traction. I’m now just waiting for someone more determined to pick up those ideas - a client lib that exposes the type safety and intellisence at compile time and allows you to easily compose those sql queries.
I think this project has some ways to go to reach that though, but thankfully it is a step in the right direction.