Looks like it might fix three big issues I had with Cue:
1. The only way to use it is to run their Go CLI app to convert the Cue into JSON and then load that. That sucks. I want native support. Jsonnet does this a lot better (https://jsonnet.org/ref/bindings.html), and PKL at least supports 4 languages. Cue only supports Go directly. Not good.
2. Cue has a super fancy type system, but as far as I could figure out there's no way to actually take advantage of this in an IDE, which is like 60% of the benefits of fancy type systems. In a Cue document you can't say "this is the schema". XML had that decades ago (and it has awesome IDE integration with Red Hat's XML extension for VSCode). Even JSON can sort of do it via `$schema`. The docs are a bit scant but it looks like this supports it too. The fact that Cue doesn't sucks.
3. Cue is pretty much only a fancy type system. It's a really elegant and nice type system, but that's it. It doesn't even have functions. So it isn't going to help with a lot of the things that Jsonnet and PKL help with.
This is not really in the same area as Cue. It's a way more direct competitor to Jsonnet and looks better, based on my brief skim.
My only concern with these sorts of things is that they're basically a whole new programming language, but without many of the features you'd want from a real programming language. It's in an uncanny valley.
Note that CUE has comprehensions, which are morally (but not syntactically) functions (actually closures). They are a way to transform values (which can be types) into other values.
We are also adding real function types to CUE. At least in the beginning these functions will be written in other languages than CUE itself, however.
While we are very principled when it comes to language design, we are also very responsive to finding solutions to user's problems, and we welcome any sort of feedback, especially if it's backed by specific use cases and experiences.
As mentioned in another comment, support for languages other than Go is coming.
That is indeed the only reference I could find (and maybe Cue is inspired by this paper?), but they don't seem to explain why they use such language. They talk about "unjustified reasoning" and laws, but the connection to ethics seems questionable. But I've not read the whole paper.
A moral equivalence class is an axiomatic equivalence class that when quotienting some mathematical structure the model in question might lose some of its soundness or adequacy properties, but one that nevertheless might be useful for other meta-mathematic reason.
This use of the word is so common in certain math/category theory/compsci communities that I was not aware it was unconventional in any way. It has nothing to do with ethics.
I guess the moral of the story is that the moral of the story can be lost if people don't understand the story.
Love this comment and I agree with basically everything. What are you using for configuration these days?
I've fallen back to YAML because at least its already used for a lot of tools, and has comments, jsonschema support in VSCode giving IDE features, language library support, yamllint, and yq for formatting/querying/mass-updating from the CLI
Yeah I actually haven't found a great answer yet. Here's everything I've tried and why it sucks:
* JSON. No comments. Deal-breaker
* JSONC. No unique file extension so its difficult to distinguish from JSON. Poor library support due to library authors drinking the "comments are bad" koolaid.
* JSON5. This would be an excellent option IMO except that library and IDE support is not great.
* JSON6. This just complicates JSON5 for minimal benefits. Pointless.
* Cue. As described.
* Jsonnet. Pretty good option tbh! However I couldn't get the Rust library to work. It's a low level parser, seems like you can't just plug it into Serde, which is what 99% of people really want. Also I ran into the "uncanny valley" effect where you can do some things but not all. So it tricks you into writing some programmatic config (e.g. with string manipulation) but then you find you can't do that string manipulation.
* Dhall. Weird syntax (backslash to declare functions. I've also heard it is slow. Didn't try this too much.
* YAML. Obviously YAML is the worst option. However I did realise you can use it as basically JSON5 except with a different comment character, which is not too bad.
* Starlark. Actually I haven't tried this yet but it looks promising.
So yeah I have no idea at the moment.
I wonder if it would be worth defining a "YAML JSON5" format, that's basically YAML-compatible JSON5.
Ha I forgot about that. TOML is pretty awful too. It's fine as long as you only need 1 level of nesting. As soon as you need to go deeper than that you end up with [[weird syntax]] that is very not obvious. I would say it's less obvious than YAML and YAML is already pretty unintuitive.
Looks like it has better IDE integration. Still, I am going to stick with cue because of what you mentioned and also because it is a community project. Apple has very few actively maintained open source projects and sometimes such projects are difficult to contribute to or have wavering support for the open source side. It is great having corporate backing behind something like swift that needs a massive amount of work, but for cue, I am happy with steady improvements meeting the needs of a wide community once I can figure out a good IDE integration.
I really like CUE, but for most use cases I have I would want to embed it in an application, and Go is the only language with support.
For it to gain more adoption it really needs a rewrite in a low-level language (C/Rust), so it can be exposed in various languages through an extension/FFI.
Making CUE available as a library for other languages is one of our top priorities. Sadly, I can't provide an ETA at this time, all I can say is that I am personally working on this.
Getting feedback from the community about what other languages they'd want supported first would be of massive help, however.
This is honestly my only complaint about Cue after having it used it for a few weeks after looking at everything else in the space (KCL, Jsonnet, Dhall, etc.). Cue is incredible imo and the commenter above talking about not being able to define the schema vs. the data is sort of missing the point - Cue makes them the same thing in a way that really understands the whole role of config langauges and IMO is way better for it. When you start to really understand what a config language _should_ do, Cue is the only option, and most attempts to dismiss it seem like hand-waves in order to push some other preference.
However it only being in Go and not implemented with some C ABI is a major downside for adoption, especially when their documentation itself for implementing the core CLI functionality in a go program (to then compile to a DLL for use in non-Go land) is pretty sparse.
Thank you for your comment. We're working hard to add support for other languages. I agree that exposing a Go library as a C shared object is non-trivial and rough around the edges. We are committed to polishing these edges.
I've been somewhat surprised that CUE bills itself as "tooling friendly" and doesn't yet have a language server- the number one bit of tooling most devs use for a particular language.
Tooling friendly can mean different things to different people. Similarly, different groups of people have different priorities.
It has always been clear that LSP was high priority, but we have many other high priority work that also needs to be done. Most of the work that we do is driven by feedback and demand from the community.
Additionally, we want to do the LSP right instead of quickly hacking something together. That requires more work than one might think.
While CUE has not reached 1.0 yet, people definitely use CUE in production and we work hard not to break any of their code. I can assure you LSP is missing simply because we had other things to tackle first and not because the language is unstable in a colloquial sense.
That’s what I’m seeing as well. Curious to try it out to see how its expressiveness compares to Cue. Looks like it’s Turing-complete as opposed to Cue, which is a plus… but that comes with downsides.
One thing I like to see is the direction of “declare types and validations in a single place, integrate with any language”.
My daily codebase atm has types declarations in typescript, cue, pydantic and for our database… and it’s driving me bonkers seeing as most types are already declared in Cue to start with. I played a little with packages meant to translate them i.e. Cue -> TS, but nothing worth the effort.
IMO it would be a big upside for Cue to handle these as first class language features.
There are three (maybe more?) ways things can be Turing-incomplete:
1. You are limited to N evaluation/reduction steps.
2. The language doesn't include primitives like recursion or loops.
3. You can have recursion or loops, but the language makes you somehow prove that your program will terminate.
I think (1) would be fine, but I don't know any configuration languages that use this approach.
(2) is restrictive/annoying whenever you want to implement any logic in the config language library. Eg. a tool uses a homegrown data format BAML and you need to convert JSON to BAML in the config. Now either you have to write and manually call a preprocessor, or you need to use a patched version of the <config language evaluator> that will have JSON->BAML as a built-in, or you must implement JSON->BAML without loops or recursion. For a more realistic example, imagine that a certain config string has to be HTML-escaped and the config language doesn't provide a built-in for that purpose.
(3) -- you don't want it. There are languages (like Agda) that let you prove things like "this program terminates", but writing those proofs can be harder than writing the program itself.
I think the C preprocessor is an interesting example of (2), because the metaprogramming community has converged on an extremely clever paradigm to circumvent the lack of recursion: continuation machines. By defining a linear number of “continuation evaluation” macros, you can generate an exponential number of “recursive” macro expansions, which trivially scales to the point that it could take until the heat death of a universe for an arbitrary program to terminate, but a program can choose to terminate at any time. The Chaos-pp and Order-pp projects are good implementations of this!
I think 2) seems incorrect. What you can’t have is unbounded loops and recursion. Bounded loops are perfectly fine and I don’t tend to need unbounded ones when programming (with exceptions being infinite loops for handling asynchronous events, which a configuration language doesn’t need to do).
Recursion is trickier. I think banning it or simply limiting stack depth seems fairly reasonable? In fact I’m pretty sure most Turing-complete languages have a stack depth limit, so unbounded recursion is not allowed for those either. I don’t see a limit being a problem, because again this is a config language.
I don’t see why HTML escaping needs Turing-completeness. It shouldn’t need any unbounded iteration (it should be limited to the size of the input string) or unbounded loops. In general, I can’t think of any typical data processing code where turning completeness is required, but could be wrong. Do you have any practical examples of transformations that need unbounded iteration?
> I don’t see why HTML escaping needs Turing-completeness.
First of all, let's avoid "Turing-completeness" because then we might start arguing about whether a language with unrestricted recursion is or isn't Turing-complete since there are stack depth limits / memory limits / universe will end one day / etc.
I would phrase this question as "why would HTML escaping need unrestricted recursion or loops" -- since in practice config languages either have unrestricted recursion or loops (Nickel), or they don't (CUE, Dhall, Starlark).
For HTML escaping specifically, just having `.map(f)` and `.concat` available (in functional languages), or `for char in string` (in imperative languages), would be enough.
For something like HTML un-escaping, it's already trickier. If you are using recursion, your language needs to understand the concept of the string becoming "smaller" at each step. If you are using loops, `for ... in ...` is not enough anymore.
An even trickier example would be mergesort:
merge(xs, ys) = ...
mergeSort(xs) =
let len = xs.length
left = mergeSort(xs.slice(0, len/2))
right = mergeSort(xs.slice(len/2, len))
in merge(left, right)
It might seem obvious that this should terminate, because of course `.slice` will return a smaller array, but the actual termination proof in Agda is already something I wouldn't want in my config language: <https://stackoverflow.com/a/22271690/615030>
(Not to mention that this implementation is faulty and will loop at len=1.)
Limiting stack depth at [arbitrary number] -- this is similar to (1). I don't know why configuration languages don't do it, to be honest.
2a. The language includes limited primitives for recursion or loops.
If that’s done right, somehow proving that your program will terminate becomes trivial.
For example, allowing looping over a previously defined array with (key,value1) pairs to generate many more complex definitions that include common value2, value3, etc fields trivially guarantees termination, but a generic “while” loop doesn’t.
That will make you language less powerful, but IMO shouldn’t be problem for a configuration language.
In this example, I’m not sure you would even need that, as the language has ways to share common config values.
See my examples with html un-escaping and mergesort, down the comment chain.
Limited recursion/iteration is ok if all you need is to fill existing values into a template and possibly reduce repetition.
But in a large system with many components I might want to take a single timestamp, parse it, and generate timestamps in five different formats for five different subcomponents.
Or I might want to generate a piece of yaml config for an Ansible playbook that needs it, and now my config language needs to know how to escape yaml strings.
Or a config for a static site generator needs to be able to calculate a hash of a local css file because I’d like to use links like `style.css?hash` (a popular cache-defeating mechanism).
Or a certain path has to be split on “.” and reversed (Java-style com.example.blah things).
Or a Unix path needs to be converted to a Windows path, except [some special-cased paths that live in a map in an adjacent config].
There are endless reasons to want arbitrary logic in my config files, beyond reducing repetition. A lot of things I’ve listed are provided as primitives in various config/templating languages, but you always end up stumbling upon something that’s not provided.
Of course, one could say “You should use a real programming language for this kind of stuff”, and I’m happy that the JavaScript ecosystem is converging on allowing .js/.ts files for configs, because that’s exactly what I want too. But I’d like to have the same features available in projects that aren’t allowed to touch JS.
Many data transformations that you take for granted in other languages are either impossible or require amazing feats of contortion of the language to make happen.
Javascript/typescript don't have introspection or any autogen between static and runtime types either.
Cue is not general purpose language, with emphasis on it - because it's a good thing.
Asking for upstream embedded support feels like asking for bash interpreter, why would you need it in the first place?
It's based on completely different, logic based paradigms, use it as it is meant to be used - as top level configuration aiding language. Declare policies and generation in it and interface with other languages/tooling though input/output json/yml.
I think everyone appreciates links to similar projects for comparison, but a more in-depth comment would probably come across better - the "less principled and more primitive" sounds like a thoughtless off the cuff ad hominem dismissal.
Consider that some engineers poured a lot of heart into what they were building, and are probably excited to finally share it with the world.
I am not saying you have to love it, but just brutally putting it down with no justification seems really rough. Snark is easy.
I wish the Cue docs were better. I arrived here https://cuelang.org/docs/usecases/configuration/ but it doesn’t answer basic questions like “can I define my own validation functions (the > < != operators used in the example)?”.
Yes. I am sure it’s that simple. I’m sure that there are all downsides and no upsides. This is the first time in history where one technology is a strict superset of a competing technology, from all perspectives. /s
Primitive and less principled doesn't imply "downsides". Go is more primitive and less principled than Haskell, yet it's useful and oftentimes better due to its primitiveness. Cue is written in Go for example.
[0] https://cuelang.org