I investigated APL pretty seriously at the end of 2019, after a very inspiring conversation with Aaron. I discovered that everything in this post is true, and (dyalog) APL is a pretty great language.
However, I also decided that it is still a bit too dead, at least for me. Specifically, it does not work well with other programs, at least on *nix. This is reflected not only in the lack of external libraries, but even in some basic areas – both FFI and even reading/writing to standard input and output are surprisingly challenging. Summing all this up, writing a basic Linux CLI app (read from stdin, take and parse arguments, print to stdout) is still a pretty large challenge.
As great as APL is, for me, it's hard to overcome the ability to work with the whole existing Linux/FOSS ecosystem. (I might feel differently if I did any Windows development, which does feel like much more of a first-class citizen for Dyalog APL)
--eval ≡ evaluate the text
f ≡ formatting function for printing nested values
s ≡ partition a line of text and remove the partition character
⎕FIO ≡ read stdin as byte stream (see FILE_IO.apl)
⎕UCS ≡ convert bytes to characters
⍎ ≡ execute an expression (convert characters to numbers)
SUMMARY:
read stdin as a bytestream
partition at newlines and commas
convert to characters
interpret characters as numbers
disclose into a rank-2 array
format to stdout
That is indeed often overlooked, I thought it was not ready for primetime and that somehow stuck in my head all these years. I hear no-one mentioning it either usually when these conversations come up.
Your description and summary blocks have started me wondering... has there been much work done on converting APL statements to more verbose explanations?
The sigil-to-meaning mapping is likely automatic for seasoned APL programmers but it might make such one liners more effective for evangelist purposes by making them somewhat more self-documenting.
it indeed becomes much easier to understand after ~ a year. this thread included some discussion about making APL-family languages easier to read for novices:
https://news.ycombinator.com/item?id=20728715
I'm of the opinion that code should be documented. Short description, arguments, return, purpose etc. However rebinding primitives kindve defeats the purpose after a little familiarity. For example:
commute ← {⍵ ⍶ ⍺} ⍝ or commute ← { ⍺ ⍶⍨ ⍵ }
will get tedious quickly. Its better to lean into the terseness of the language, rather than attempt to transform it into something it isn't. There has been work done for easing the learning curve, for example https://aplcart.info/ . Keep in mind aplcart generally targets dyalog, however many of the idioms should work for alternate implementations with possibly some tweaking.
I'd be very interested in reading more about the differentiating features of GNU APL. There is a great big announcement at the beginning of the GNU APL docs that talks of a decision needing to be made around a gap in the ISO specification. Is that the primary source of controversy or is there more discussion I could read somewhere else?
I've been looking for a good "diff" of the language differences between Dyalog APL and GNU APL, as much as to understand the extent of the progress since 'APL2' (or whatever the ISO standard name is considered equivalent to it) as any for any specific list of what GNU APL can't do relative to it's modern commercial compatriots.
(EDIT: Fixed final sentence to be a complete thought/sentence).
dyalog has a few more operators. for example @ & ⍠. It lets you use statement separators in lambdas. In gnu apl you can simulate that with '⊣'. It allows multiline lambdas. Dyalog uses ⍺⍺ & ⍵⍵ for operators vs ⍶ & ⍹. Dyalog lets you use ∘ as a composition operator. Dyalog generally has better performance, which becomes noticeable with large arrays. It has a lot more libraries available. Gnu APL behaves more like a traditional unix based interpreter. Gnu APL has erlang, python, lua, and C interfaces.
I admit, GNU APL gets sluggish when dealing with values with more than ~100k elements. For most ad-hoc work it is satisfactory and can let you sketch out ideas before attempting to push the performance characteristics.
from article "In some cases, users and developers pair program together, and the source is their common language."
from example: f←{4⎕CR ⍵}◊s←{⍺~⍨¨⍵⊂⍨1++\⍺⋸⍵}◊f ⊃{⍎⎕UCS ⍵}¨¨44 s¨10 s⊣⎕FIO[41] 0
Mmmm... I code in C, Java, PHP, CaML, Prolog... for a long time... so I guess that I'm quite used to programming. But really: I can't make any sense of these hieroglyphs !!!!
And NO user I ever worked with would either...
So I guess that I'll need a lot more sugar to taste APL...
its a golfed example to show that its possible to use APL as a cli tool. I would not put any code into production that looks like this. Generally it would go into a library.
APL has an initially steep learning curve, because it is so distinct historically from other languages. But it is a quite simple language once you learn it.
1. Every statement is executed right-to-left. Statements are executed top to bottom (or left to right with ◊ as a separator).
2. The only precedence rule is parentheses
3. Functions can be called with one operand (monadic) or two (dyadic)
4. Values in APL have a shape (dimensional size), a rank (number of dimensions), and a ravel (a list of elements). Values can contain other values (nesting)
{} define lambdas
⍺ is the left operand
⍵ is the right operand
⍶/⍹ are the left/right operators (higher-order functions)
← assignment
◊ statement separators
¨/\⌿⍀⍣⍤⍨ builtin higher order functions
After some familiarity it becomes as easy (or easier) to read as any other language. You can encode a remarkable conciseness of expression into each lambda, that you plumb together (similar to unix pipes in a way) in higher order patterns.
Indeed, I was playing with Dyalog some months ago, and the most painful point was I/O. I have implemented the MAL (make a lisp) interpreter in APL but I cannot seem to make the tests for the self hosting interpreter run correctly because of the way standard error and standard output are treated :-/
According the recent 18.0 release presentation, it sounds like they are aware of how different an experience this is relative to other interpreted "scripting" languages. To that end, they have added support for a Run function that will execute with access to to the invocation arguments.
EDIT: They also discuss how the upcoming convergence of .Net Framework and .Net Core is going to have a major impact on their cross-platform capabilities.
That is great to hear; perhaps some of the issues I have will be addressed when version 18.0 is released (which, despite the present tense in your comment, hasn't happened yet). I really hope so – as I said, there's a lot to like in APL.
Summing all this up, writing a basic Linux CLI app (read from stdin, take and parse arguments, print to stdout) is still a pretty large challenge.
I suspect that's because it isn't the primary use-case of APL, much like trying to use MIT Scratch[1] to write one. Sure, they're all Turing-complete, but I/O and other interfacing varies greatly between different environments.
Smalltalk, yeah blissful silo, but I'd argue most lisps are heavy on pragmatism which means streams, I/O, FFI, signals, etc. For example, I believe this forum is running on Arc :)
Scheme is interesting because you can carry a Lisp around in your brain's pocket, implement it anywhere, and integrate it with anything. Accordingly there are Schemes (like guile and scsh) which work well with Unix, Schemes (like tinyscheme, s7, and guile again) which work well as embedded languages, Schemes (like Kawa) which work well with the JVM, etc.
Common Lisp can do the same, but it's a much bigger language so you see less of this effort and more standalone CL implementations like SBCL.
I'm awaiting delivery of a printed Mastering Dyalog APL book while reading this! I landed on kdb+ after seeking a reasonable alternative to the so called "best in class" Elastic Search/Kibana tooling, fell in love with K once I understood that the syntactic terseness is all for the sake of fitting the entire interpreter into L2 cache, and have now landed at the decision that learning Iverson's classic is the only way to satisfy my desire to live a life free of them dang stinking loops!
I don’t believe that the terseness of k is necessary to fit into the I$. I think you could have reasonably longer operators and do fine.
I think it’s partly about style, partly about having a small number of operators (which compose well together), partly on using simple data structures (it doesn’t take much code to iterate an array.
I’m not even particularly convinced that fitting in the instruction cache is a trick that magically makes everything fast anyway. Most of your memory accesses in a typical data processing program (ie the kind of program one would write in k) will be to the data and hopefully these will be nearly always linear and mostly sequential accesses.
The original K design was laid out in the 1980s when the constraints were even tighter than what they are today. The utilization of very short operators means not only the interpreter easily fits into cache but also the custom function definitions you have written will as well.
When dealing with high performance computing or real time processing of high volumes of data, any fetch to RAM for loading a function call to dispatch is going to have _some_ impact in a tight loop. Add that up for all the libraries you have loaded for your application verses a ground up implementation in K... Does that whole thing live in L3 along with the VM or intepreter + dependencies underneath it? It's doubtful.
My experience was simply using their Kx's free Developer IDE and experiencing the performance differential on datasets myself. YMMV but my (admittedly limited) experience leads me to believe that there is a serious case to be made for the performance advantages of having all your computational logic living as close to your computational cores as possible.
See also the PhD by author of the OP article where he presents language where:
"The entire source code to the compiler written in this method requires only 17 lines of simple code compared to roughly 1000 lines of equivalent code in the domain-specific compiler construction framework, Nanopass, and requires no domain specific techniques, libraries, or infrastructure support."
Surely none of that is an argument for terse user-facing syntax? Anything the user types can trivially be converted into K's "bytecode", ahead-of-time.
The arguments for terse user facing syntax are related, though:
The ability to see a whole program (17 lines vs 1000 lines) means you need much less human “working memory” or whatever the biological equivalent of cache is, to reason about the program.
It also means you can use your visual system’s pattern matching abilities because patterns are 5-10 characters rather than 5-10 pages as they often are in C++.
Totally different hardware, but it’s still about the L1 and registers.....
I agree with these points. Also keeping a language small allows it all to fit in your brain (though of course there are all those idioms in apl) which isn’t really true of larger languages
True but if I understand correctly (and I'd be happy to be corrected by someone with more K experience), the smaller size of input data should also have an impact on real world parser performance (including the memory usage involved therein).
Any sane interpreter would parse things beforehand and so you shouldn’t expect a significant fraction of performance to be parsing dependent for anything except a trivially sized dataset. Two reasons to care a lot about parsing performance:
1. Silly reasons mean that you need to block while you parse and reducing this blocking time is important (eg JavaScript. One trick is to optimistically assume that the script won’t do a document.write and try to process the html after it. Another would be to eg only parse enough of a function definition to find out where it ends and only parse it more (or block on parsing it) if it gets called)
2. You get a large amount of source code that needs to be quickly parsed all at once to be interactive (eg JavaScript and booted web pages)
I think in the case of k you only parse a small amount at a time in interactive use and any non interactive use would be for a big enough job (yet still generally small enough program) that parsing time would not be significant.
If parsing efficiency really mattered, surely the language would use something easier to parse (eg something rpn based to avoid parens, using single characters for each operator and just disallowing variable names from containing those characters).
In summary: I think parsing speed is basically irrelevant to whether k is fast or not, so long as the parser is reasonably fast, it’s efficiency shouldn’t be a constraint on the language.
I don't really understand these array based J/K/APL languages but from a distance it would seem like they would primarily be useful for numerical based work such as finance or calculating patterns on 2d arrays (game of life). The languages seem more mathematical than software engineering based.
It reminds me about when I was learning my first programming language. The book had a ton of example with simple numerical computations (e.g., calculate leap years) and I soon became competent with the language's facility for basic mathematical operations. But as a programmer now I have very little use for these operations so when I learn a new language I have only a rudimentary understanding of how to do math in that language.
Is this correct or am I misunderstanding the usefulness of array based languages?
APL is best thought of, not as a "programming language", but as a DSL for array manipulation. The syntax is sadly dead. The semantics live on in the most mainstream of places, notably Numpy.
Trying to write entire full-features programs in APL should be thought of as roughly the same as doing so in Awk. Possible, and even beautiful - but not a serious endeavor. We'd be better off if other languages incorporated APL as a first class DSL, the same way they incorporated regex.
Numpy has a very rough approximation of APL's semantics. The broadcasting rules are quite different, and it's missing some rather fundamental things like automatically lifting functions with argument rank other than 0.
It requires a different mindset. You might not need leap year calculation often, but many things you do have a short, simple, efficient J/K/APL implementation entirely different from how you’d do it in other languages.
E.g. to check if a text string s has balanced parantheses in C/Java/Python, you would have a loop to increase/decrease depth based on character, error if negative at any point, and balanced if zero at end.
In k, you would compute the running balance at each point
b:+\(s=“(“)-(s=“)”)
And then check the minimum of it &/b to see if it is negative, and the last element *|b To see if it is zero.
This is a text parsing problem, but has an elegant vector solution. Most things do, whether it is graphics, database, whatever - but it’s rarely intuitive if you are not used to thinking in vectors.
Also, the expression for b above can be half as long, but you would really need to be versed in k to understand it.
Will the vector solution thing still be elegant if parentheses that are escaped with a backslash don't count? Oh, and backslashes can also be escaped, so the paren in "\\(" still counts. Oh and you can have a quoted segment in the string where parens don't count. As expected, quotes can be escaped..
I bet there are elegant solutions, and perhaps they are obvious to those who are well versed in array programming. What I think the world needs is a book/tutorial/whatever that demonstrates elegant solutions to things that seem ugly and not amenable to vector processing (at least in the obvious and naive way). I think we have enough (too many) tutorials showing how to calculate means and such :(
Elegance is in the eye of the beholder, of course - a state machine is easier to code in K than in almost any other language, and is likely the most elegant overall solution.
A common kludgy solution you’d see would probably be to delete or zero out characters that shouldn’t be counted, and then do the original thing; some would consider that elegant and others won’t.
There are many examples if you look for them, many well explained, e.g. https://nsl.com ; the problem is APL/K is almost as different from C/Java as Japanese is from English ; there’s no amount of chewing anyone will do for you that will save you from going deep, and if you’ve decided to go deep, the material is already available.
Sounds good, I love state machines and I often write them in C where other people would just go with ad-hoc loops and branches and temporaries. I'll keep an eye out for examples..
Recently there was a link on simdjson, a C++ parser for JSON which aims to be record fast. It uses vector instructions of modern processes, but if you'll watch the video presentation, Daniel Lemire will also explain how they handle finding string boundaries - with all those escapes needed - without branching, purely logical operations.
Could be a good example for how APL approaches things.
The one thing that makes you think different when working with these languages for a bit, is that your mind has to move differently from when you are doing 'regular' programming.
> Will the vector solution thing still be elegant if parentheses that are escaped with a backslash don't count? Oh, and backslashes can also be escaped, so the paren in "\\(" still counts. Oh and you can have a quoted segment in the string where parens don't count. As expected, quotes can be escaped..
You name 4 additional 'features' to the issue; in a c-like language, for example, you would amend the original problem with extra code to add these in. And, usually, you can recognise the original problem solution somewhere inside the final code. In k you might find that the final solution is the same size as the original but has no (seeming) overlap at all. You might find, that when you bring the problems as managers often do; after you implemented 1 thing, they would in with 'oh, it does this as well, amirite?', that you could end up with 5 solutions that are completely different but the last one delivers the features you required.
Good or bad, that's a matter of taste, situation etc.
That’s a very trivial example though. For a more real-world example, consider multiple different delimiters for matching. The C/Java/Python version is still trivial if you use a stack instead of a numeric counter, returning an error if you try to pop an empty stack or one whose top element does not correspond to the delimiter you’re matching. How would you do this in K?
The stack makes it much less trivial than a counter in Python, and even less so in e.g. C which lacks a built in data type useful as a stack (leading many to allocate a fixed size stack and either reject valid strings going too deep or have buffer overflows)
Indeed, a multi delimiter version is harder in K as well. I am writing this on a phone without K (as I did the prev post) so will not attempt a runnable example; but, I would implement it as a state machine, taking Stack and one char as input; and returning new stack as output; then reducing it on s; if final stack is empty, it is valid. It would still be one line.
K loves state machines. When tokenizing/parsing in K, you’ll often see people write DFA tables by hand because it turns out to be simpler than writing a lex spec; also shift/reduce tables because it is simpler than yacc grammars.
I would not recommend these if you want to write a C++ compiler, or you need yacc’s error reporting and recovery. But 95% of the time, yacc/lex are for simple calc or DSL expressions that ARE easier to code by hand in K.
And that’s another place K/APL mindset differs significantly from common practice: Avoid solving the general form of your problem if you can solve the particular one in one line (which you often can, and is often enough)
It’s not very hard to code up a dynamically allocated stack in C. I think it’s of fundamental importance for a programming language to let you easily implement lots of different data structures efficiently.
Are you saying K has a stack built-in (or another structure that can easily implement a stack)? Okay then how about a problem for which K lacks a built in data structure? I would google it myself but I can’t seem to get any results on K.
K has lists somewhat comparable to Python’s, which is what i’d use to implement a stack in either (append to push, shorten/pop to pop)
You are moving the goalposts. You can construct problems where eventually K cannot be shorter/simpler - this is a consequence of Turing equivalence and kolmogorov complexity.
The claim I made was that, for problems that people actually need to solve, K And APL often have a short, simple, elegant and usually performant solution.
I stress that I am talking about the actual problems - that is, creating a mapping from an input set to a respective output set - I am NOT claiming that the solutions people reach for in language $x can be expressed more elegantly in K/APL. Another example is “flatten” - expressed as ,// in K, and which could be read “join over until-converged”. This is not how you would implement it in Lisp or Python likely - and it is likely less efficient at runtime than how you WOULD implement it in Lisp. But it is arguably short, simple, elegant and (depending on implementation and input structures) likely acceptably efficient.
Added p.s: it is not hard to code a dynamically allocated stack in list, but it is about as complicated as checking the balance, which makes the balanced parentheses problem twice as complicated in C compared to even Python; which is why so many C buffer overflows exist - the buffer Management was accidental Complexity and therefore gets less attention than the security implications of a buffer overflow would warrant.
The reason I'm asking about further problems is not because I want to "move the goalposts" but because I (and likely other people) are skeptical about the ability of K (and other APL family languages) scale beyond simple problems or specific niches (such as finance).
It's fine if it can't though. Different languages have different niches. You wouldn't write an operating system in python. I'd just like to know if there is a use case for K outside of finance (or other sorts of math on large data sets). Has anyone written a video game in K?
> The reason I'm asking about further problems is not because I want to "move the goalposts" but because I (and likely other people) are skeptical about the ability of K (and other APL family languages) scale beyond simple problems or specific niches (such as finance).
Yeah and I've become wary of languages that are an island where everything is nice as long as you can change the problem until it has a nice solution in the language. Forth is a bit like that.
The risk is that all these nice and elegant things fall apart like a house of cards once you try to merge the island with the ugly nasty real world, where legacy compatibility and rigid requirements from various stakeholders define the problem.
I think most people out there are primarily interested in general purpose languages that they can learn, master, and use to attack just about every problem, ugly or not. In that case, it's only natural to be reserved about proposals that are sold as very elegant but the demonstrations of which are quite removed from the ugly real world problems.
I'm kind of on the edge. I can tell that a bunch of cool stuff has been done in APL and derivatives, but I'm still not sure whether it'd hold up e.g. in my day-to-day job (which involves lots of networking, async i/o, timers & callbacks, parsing, message passing, crypto, etc). And I still feel like there aren't enough relevant real world examples (or I'm just not able to find them much).
> The risk is that all these nice and elegant things fall apart like a house of cards once you try to merge the island with the ugly nasty real world, where legacy compatibility and rigid requirements from various stakeholders define the problem.
I've had more of that experience with Java than with K in the late 1990s and early 2000s. Java wasn't new, and was well accepted, but the ecosystem was horrible and interfacing to other systems required great effort and was abysmal in general.
The issue you are talking about is not a language issue, it's an ecosystem issue. K recognizes it by not trying to be all things to all people - it's incredibly easy to FFI directly, and to connect through various interprocess methods (including a native efficient KIPC well supported from just about every other language environment).
Arthur also dropped native GUI in 2005, even though it was incredibly effective (by far the easiest way to get a GUI - way easier than VB6 for example; http://nsl.com has a spreadsheet, calculator, and more showing just how easy). It was incredibly effective, but also incredibly unattractive (the term "milspec" was often used by users).
This the issue you mention - users wanted beautiful GUIs, often web ones, that they could put their own touch to. The requirements are rigid and have inherent complexity that general elegance cannot help with. K is decoupled enough to not have to solve everything to be extremely useful.
I don't know if it's still the case, but in 2004, if you called Kx systems (the people who sell K) and asked for an evaluation, they would send a person who would solve YOUR problems with K on YOUR system, working 1-2 weeks. Then you got 1-2 weeks of evaluation; and then they charged tens of thousands of $$$ for the solution if you wanted to keep using it. And many did.
Most project specs today include requirements that are accidental complexity, rather than inherent. K/APL (and their philosophy) is about solving the inherent complexity elegantly. It's often possible, but rarely practiced, that you can solve the inherent complexity one way, and the accidental complexity another way, and it ends up being much simpler overall.
> The issue you are talking about is not a language issue, it's an ecosystem issue. K recognizes it by not trying to be all things to all people - it's incredibly easy to FFI directly, and to connect through various interprocess methods (including a native efficient KIPC well supported from just about every other language environment).
I'm afraid this only makes my point stronger. If I'm pushing the ugly complexity outside of K with an FFI, I'll be stuck writing the ugly plumbing (a major part of the job) in some other language. For the uninitiated it is very difficult to say whether or how much will be left for K to handle, if anything. That kind of thing would make K look like a one-trick pony, and maybe not worth investing in.
> Most project specs today include requirements that are accidental complexity, rather than inherent.
I agree, and I'm afraid that in many projects (including my dayjob) the accidental complexity is the bigger job. Even if I could solve the inherent complexity of all we do in one beautiful line of code, we're still stuck with a mountain of crap from 2-3 decades of legacy and bad design that really can't be changed.
> If I'm pushing the ugly complexity outside of K with an FFI, I'll be stuck writing the ugly plumbing (a major part of the job) in some other language.
It’s kind of the opposite - you get an easy to use escape hatch where it makes sense - which enables you to work with large complex legacy systems.
It lets you do the ugly plumbing in K.
Contrast this with Java - back in the early 2000s my company ended up rewriting huge parts of our legacy systems to Java simply because JNI was so bad.
I'm interested in this are (building a toy relational lang with array capabilities) and one are I have not find example is CRUD apps. How exactly a simple todo or invoice work here?
I don't have a crud example off hand, but https://nsl.com/ has a lot of examples - chat server/client, instant messenger, spreadsheet, class tracker, 15-puzzle, calculator, raytracer, pivot tables, and more.
You may like http://nsl.com/k/t.k - a 14 line in-memory, quite efficient, relational-style database (no parser, just the ops).
Note most of nsl is k2/k3 ; the differences aren't huge, but if you dig deep it helps to know which version you're reading.
I agree that array based programming languages are especially useful for solving mathematical problems.
For anyone new to the field, a very approachable introduction to array programming is in the Klong book, here an excerpt: http://t3x.org/klong/ap-excerpt.pdf
They're useful for a lot more than math, certainly. APL has a lot of world-firsts that wouldn't really make sense for a language useful for nothing more than math; looking at the history of IPSA and similar will probably give you some good ideas. The user 'dTal in this thread is far from giving you an accurate story.
Not just 2d, nd, three, four, five, whatever dimensions, a bit agnostic in this, as is algebra. I can't claim to have dabbled in the language but was somewhat aware in the 80s and 90s. An APL magazine from the 90s was called Vector if I recall correctly. Yes quite mathematical, and an interesting community then of some quite smart melding of math heads, still a huge focus on performance, played with it on a VAX.
A run time license for multi-user applications? you must be joking, right? does still anyone produce single-user applications, but the odd shell script?
So no, I won't use Dyalog APL in my company and won't invest my time on it. Sorry.
What would getopt look like in APL (also K, J, what have you)? I couldn't find any examples. Similarly, I'd like to see some examples of handling multiple I/O endpoints (sockets, stdio, files?) asynchronously (or should I say without blocking) using whatever mechanism works best in APL.
Uhh.. no, that's just argv. Getopt is something you'd use to actually parse flags and arguments out of argv.
EDIT: Ok, .Q.opt looks a little better, although it's still not close to getopt (where -abc would be shorthand for three flags, -a -b -c; but other flags might be defined to take (possibly optional) arguments). Is it a built-in implemented in whatever K/Q is written in? How would you implement .Q.opt in K if it were not built in?
Yep you're right, it's definitely not doing as much as getopt, though it's the same general use case. It's output is a dictionary representing your command line arguments, which you'd then use in your program to act accordingly.
It's implemented in K and comes standard with the distrubution. If you've got a KDB+/Q distribution you can view the actual function definition by executing it without an argument:
Disclaimer: I cannot read K, and I'd consider myself a beginner in Q, so the above is completely alien to me. But what that does, in words: any "-single-dash-params" is treated as an option. It splits your 'argv' by those options. Everything between the options become the 'values' for the prior most recently preceding option. If you've got a no-arg option, it just becomes a member in the dictionary without a corresponding value:
Having used APL professionally for ten years from the early 80's to early 90's I am a big fan. I used it for a wide range of applications, from business systems to DNA sequencing. I even published a paper at a conference eons ago.
So, yeah, I am an "APL guy". And, yeah, I think it's dead.
Well, not really, but, yes really at the same time.
Yeah, $1,600 USD for a language license? No. Thanks. Go away.
In addition to this:
"In addition to a current Developer Licence, a Run Time Licence is necessary if:"
Yeah, no. Go away.
And then...
"When several users use one or more applications on a shared server, a Server Licence is required."
No. Definitely no. Please, pretty please, with sugar on top...go away.
I could go on. Just have a look at that page and see how you could possibly justify getting into this monster of a business case for your particular business or any other business who's financials you understand.
If you are still interested after that, take a moment to go browse the source here...
Oh, wait. Yeah. No. Go away.
That said, I still do love APL. I love the language, the ideas, what it represents and the superpowers it gives you. It truly enables the creation of computational solutions for complex problems at the speed of thought once you learn to think in APL (which likely takes about a year to fully internalize).
During the 80's I often dreamed of designing a purpose built computational engine (hardware) specifically designed to execute APL commands in one clock cycle. Hey, I was young and stupid. While I was using APL I was also coding embedded systems in C and assembler. I would often wonder "wouldn't it be amazing to be able to execute this APL statement in one clock cycle, like some of these assembler instructions on this tiny little MCU?".
Anyhow, let's end this on a positive note. Here's a list of language improvements I authored a long time ago. This list isn't exhaustive (it was a five minute stream of consciousness exercise). Also, some of what's on this list might already exist in modern variants of the language. Feel free to criticize, no problem, and comment. Understand I am issuing advance notice that this is not a perfect list or even a current one.
APL should go FOSS or adoption will forever be limited to esoteric high-ROI
applications where it might make sense or legacy APL code already exists.
The option for compilation would be very interesting.
Within a function, variables are global by default. Huge pain in the ass.
Objects. Yet be careful to avoid the mess OOP can (and often does) become.
A language-defined interface to C (or some other suitable language) for
when you need to speed things up.
Inline C or Python, or both, as a first class citizen. Make it first
hybrid high level language.
An extension to nested arrays that allows you to create C-like structures.
I am going after the self-documenting nature of having structure members
have name-based access.
An extension to function definition to allow for more than two arguments.
This would require a little language re-thinking.
I would also look into making a hybrid that includes some C-like syntax
with an APL core. C/C++ -like comments and conditionals with bracketed
groupings of APL code would go far in making code easier to read and
organize.
Native C-like switch() primitive with APL-inspired extensions. For example,
switch could take a vector argument.
APL-run-time-definable comparison operator. Sometimes I want to apply a complex
evaluation function to data rather than the simple equal, not equal, grater/less
than, etc. operators.
The ability to enforce data types if desired.
A means of telling the compiler/interpreter to optimize code
--at the statement level-- for speed or resource (memory footprint) and
other criteria. Certain operations can explode into huge multidimensional
memory-sucking arrays which isn't always the best idea.
Real-time extensions integrated into the language.
Ability to use and request GPU resources for computation.
Ability to use and request CPU cores for computation.
A simple-minded example of the above two would be wrapping a code
section with "CPUn{}" or "GPUn{}, where "n" is the core or GPU number.
Standard interface for custom hardware-based acceleration (read: custom
FPGA boards). FPGA's are amazing, make them first class cititzens.
True binary vectors and arrays.
More flexibility in multidimensional array indexing.
A rethinking of the workspace model to better support multi-developer
environments.
Genetic and Evolutionary computing primitives built into the language.
Neural Network primitives built into the language.
Language native image processing (think OpenCV integrated right into the language as a first class citizen)
Same with NumPy. APL should be able to do all of this and more.
Built in primitives for multi-threaded/multitasked computing.
Built in primitives for network access and processing.
Built in primitives for multi-core and distributed computing.
Built-in primitives for data exchange. For example, ingest or output
nested array data from/to JSON, SQL, XML or other modern data formats.
Better pattern-matching primitives. I'm thinking at least regular
expressions.
A general cleanup of the notation to remove lame text-based "fixes"
from an era when rendering non-ascii characters required custom ROMs
on the graphics card. All APL notation should be symbolic. ASCII
should be limited to strings, function, variable, object and other
constructs that require text. I'm talking about a lot of the quad
stuff, etc.
Make it open source and make the open source version far superior to
any available commercial version.
I'll end with a slightly edited portion of one of my HN posts from about seven years ago:
One of the reasons I abandoned languages such as APL, Forth and LISP, languages that I used extensively for many, many years is that they became less and less practical and relevant.
I can apply C to nearly everything from embedded to workstations/servers and even in modern hybrids where FPGA's are integrated with capable microprocessors. I can hire people, even entire teams, both locally and around the world to work on anything from embedded to web projects in many of the the C-like languages. Trying to do the same with APL, Forth or LISP is very close to impossible (and the quality and capability choices you have are much reduced).
On the hardware front languages like Verilog are very reminiscent of C and, as long as you understand that you are actually describing hardware and not writing software, are easy to pick-up with the appropriate background.
You move up to languages such as C++ and other layers open up. PHP, Python, Objective-C when I absolutely must and Java if I have no choice.
All of these are very flexible and relevant tools that have, for the most part remained relevant and useful for years. This range of applicability will never be achieved with something like APL.
If an APL-like language is going to come to the forefront it will be for very specialized applications where it makes sense. It will not be to run a shopping cart on a website or control a servo on a robot. That's just reality.
I love APL. I devoted a huge chunk of my professional life to it. I can't see using it or the wanna-be variants for anything today. Sorry.
I tend to agree with your broader points. In my personal life I often reach for APL to do ad-hoc data analysis. It has grown on me in the last year and has fundamentally changed how I perceive and write code. However, I don't think I could introduce it to a company.
For specific points, I would love to see an APL dialect that adopts parts of J, K & haskell. Something that retains the symbolic notation, that provides a deeper & more fluid interface to asm/C, adds some ALGOL-like control-flow constructs, and takes advantage of the intervening decades of programming theory.
APL to me is an alternate history of what programming could have been. Its not perfect by any means, but it boldly challenges assumptions about what it means to communicate mathematical thought. It was ahead of its time, and its unfortunate its taken this long for languages to superficially adopt its insights.
PS: gnu apl is a great FOSS implementation that is enjoyable to use and rich in features.
Hi Martin, thanks for sharing your insightful comments and the improvements list. I think if anyone can do that, the impact will be probably nothing less than the impact of introduction of spreadsheet to do PC industry, and arguably it is still the killer computer application to beat until today.
Just sharing about the improvements of array based programming language from the academia inspired by APL. Two of the most promosing approaches are the Single Assignment C (SAC) from Hertfordshire University, UK (proprietary) and Furthark from Copenhagen University, Denmark (open source). SAC try to the introduce better syntax (as in Algol C) for functional programming array based language with concurrent multi-core CPU support. Furthark also try to better the syntax for programming array language for GPU parallelism by being the more intuitive intermediate language.
From non academic, I would say the closest to your improvements list will be the D programming language. Unlike Python and NumPy, where there is a clunky impedance mismatched, using D it is feasible to create a seamless array based integration by virtue of its CTFE and metaprogramming capabilities including native support for object oriented, imperative and functional programming paradigm. For initial work check the Mir GLAS library for the native D implementation where it has managed to outperform the venerable BLAS/LAPACK Fortran based linear algebra library where normally Python or other programming languages depend on [1].
Additionally D also support native nested functions that can be handy for processing nested multi-dimensional arrays.
I have got the feeling that for seamless CPU, GPU, FPGA and TPU programming integration, there will be based on the Static Single Assignment (SSA) form that is currently being proposed independently by MLIR (from LLVM team) and LLHD (from ETH Zurich). If you have noticed that the SAC is also utilizing this compilation technique as well.
Thanks for the comment. I'll look into the items you suggested and see what I can learn.
My perspective on APL going forward is that it must retain symbolic notation. This is a very important element of the value delivered by this language and one of the aspects that tends to be difficult for non-APL-ers to understand (the value) from what amounts to casual contact with the language. You have to live APL for a while before the "tool for though" realization --and the importance of the symbols-- can be understood.
In the '90's Iverson developed J to get around the difficulties technology of the era had with symbols. This was a huge mistake and something that went exactly opposite the "Notation as a tool for thought" idea he promoted for years when describing APL. I watched him give this lecture in person at one of the APL conferences in the mid 80's.
The problem with any language --APL being no exception-- is that a business case must exist for wide adoption. Engineers tend to get lost in the technical minutiae, which is important when you are doing the job but it isn't a requirement for the job to be done. Or put more precisely: IF there is no quantifiable need for APL and a mechanism for better ROI, it is hard to justify shifting entire teams to this, or any other language, simply because we like the technical aspects of it.
At the end of the day things are simple: It either makes sense or it does not. I would LOVE for APL to become relevant again and for it to find mass adoption. I love the language. And yet I understand business and know the probability of this happening is so close to zero it might as well be zero. Sadly.
> Having used APL professionally for ten years from the early 80's to early 90's
That might be part of problem. Things have happened since then:
Dyalog has made the full product available for free, only asking for 2% of any profit you make once you cross a certain threshold. Open-sourcing the interpreter is under consideration.
Dyalog has a built-in compiler.
Dyalog's new function form, dfns, has variables local by default.
Dyalog has fully integrated C#-style OO system.
Dyalog has ⎕NA for seamlessly calling C functions as if they were APL functions.
Names Associated (⎕NA) C functions can be called inline
Dyalog namespaces are objects that are not class-based, but with named members
Dyalog's tradfns allow name-lists for multiple required arguments (and results)
True, there are no multi-line comments in any APL yet, but Dyalog allows writing {condition : result if true ⋄ result if false}
Dyalog has :Select which is similar to switch but allows case-lists.
I'm not sure what you mean by "APL-run-time-definable comparison operator", but any function can be used in place of the built-in ones. (This wasn't always the case, but certainly is now.
It is quite easy to check for the data type of given input, e.g. using the Data Representation function (⎕DR).
Putting the statement in a named function allows you to compile it separately from the surrounding code.
Right, no real-time extensions integrated into the language.
The co-dfns library allows you to offload computations to the GPU.
Dyalog's 1111⌶ allows you to select how many cores to use.
Right, no built-in interface for custom hardware-based acceleration
Dyalog's binary arrays use true bit-binary representation.
Dyalog's at operator @ and the index function ⌷ combined with the rank operator ⍤ provide full control over and flexibility in multidimensional array indexing.
Dyalog has supported SCM monitored text files for many years, and the built-in tools are only getting better and better.
Not sure what you have in mind with regards "Genetic and Evolutionary computing primitives built into the language."
The stencil operator ⌺ is a Neural Network/Cellular Automaton primitive built into the language.
Dyalog has some native support for image manipulation, has various libraries too, and deep integration with .NET.
Much of NumPy is of course APL breakfast, and there are math libraries for certain other things. In the worst case, use PynAPL to call Python.
Dyalog has the spawn operator & for multi-threaded/multitasked computing.
I'm not sure network access and processing would be suitable as primitives with glyphs, but Dyalog's Conga library provides this, and may well get further integrated into the interpreter in the future.
Dyalog's multi-core and distributed computing primitives are currently accessed with cover functions, but the ¤ and ∥ symbols have been reserved already.
Built-in primitives for data exchange? ⎕JSON, SQAPL, and ⎕XML etc.
Dyalog's ⎕R and ⎕S provide Perl-compatible regular expressions.
I'm not so sure all quad names should have symbols. Many of them have related or prefixed names, e.g. ⎕Nxyz for file functions and ⎕Txyz for thread-handling. That said, plenty of new Unicode symbols have been added lately.
> That might be part of problem. Things have happened since then
Of course.
> Dyalog has made the full product available for free, only asking for 2% of any profit you make once you cross a certain threshold.
Not good. People will pay for tools, not languages.
If Dyalog wants "APL for all" make it FOSS and support it with tools, etc.
I realize there's a business and, behind it, people and families to support. I get it. That said, today's reality is that large scale adoption of languages requires FOSS.
Put a different way: People are not going to bet the farm on a language that is fully dependent on a single company for it's very survival and existence.
I have a simple example of this. In the 80's there was STSC APL. I used it extensively. In fact, I used STSC APL for my work in DNA sequencing during the human genome project.
They evaporated from the planet. I hope the code-base got picked-up by someone. I haven't followed the ecosystem as closely as I used to back when I was attending meetings with the likes of Ken Iverson.
From my perspective, if Dyalog wants to propose "APL for all" and be the leader in this domain --have their flavor of APL become the standard-- they have to let go. How they mutate their business in order to make that happen is another question...and it could be an impossible task.
Not saying I have all the answers. All I know that as an APL guy it would take a major change in the APL community to get me interested in risking my business on the language.
The simple fact is that, as much as I love the feeling of programming in APL (it's like playing music, your thoughts become code) I am mature and experienced enough in both technology and business to understand that business and technology decision need to be made as objectively as possible. This is why, as an example, I am happy to pay JetBrains license fees for their excellent tools rather than screw around with vim and the deep dark organizational hole that shit can become.
One of the interesting items to note about your reply (which is excellent and this is not a dig) is that it is 99% features and 1% benefits or business.
This is how we engineers tend to relate to the world. I was the same until multiple encounters with entrepreneurship forced me to understand nobody gives a crap about features, not because they don't care, but rather due to the reality that their forcing function isn't feature-based but rather benefit or value based.
You can deliver tons of value and benefits with any language. Just look at the myriad large, medium and small companies out there delivering such value in the internet ecosystem, from consumer to industrial, as evidence.
Another way to put it is: Nobody needs APL to deliver value. Sure, maybe a few corner cases here an there. OK. Outside of that, nobody is clamoring for it. That means the road to promoting scale adoption is incredibly steep, if not impossible, to climb.
No, APL is dead, sadly. Again, I say this as an APL guy.
I still think it should be taught in school, along with Forth and LISP. Beyond that, who knows?
BTW, I continue to be puzzled by how often APL threads make it to page one on HN. Not sure what it means, other than it is an interesting curiosity.
> The result is a very solid language with some of the best tooling out there.
Hard to take an article seriously when it makes this kind of claim about APL.
> It turns out, OOP and software engineering are not the Godsends we thought them to be, particularly in the presence of rapid change.
I don't know what the criterion is to be qualified as a "godsend", but the fact that an enormous majority of the code that powers our civilization today is OOP is a part of the answer that can't be waved off that easily. It doesn't mean it's the best possible approach, but it at least contradicts the claim about OOP not being adaptable to rapid change.
And, really, claiming that APL is better at rapid change? A language that requires a specialized keyboard that's almost impossible to find these days?
> We live in an international world full of non-English scripts.
Source for this?
As far as I can tell, most languages in existence and being created every day use English keywords.
> APL is designed as executable math notation
That is not true at all. Only a very tiny number of APL symbols are mathematical (and some even use these symbols wrong). Most of the APL alphabet is made of invented symbols.
> Here again, APL has proven itself far ahead of the curve.
The opposite. APL tried to experiment with brand new symbols for a programming language, which was a commendable attempt to innovate, but since no languages in existence today do the same, it's clearly an experiment that completely failed.
The only reason why APL can still be considered not dead is that once a year, somebody posts the one liner Game of Life on a public forum and everybody marvels at it while being secretly happy APL never took off.
I generally agree with your sentiment, but I feel that you are a bit uncharitable in your interpretation of what you are responding to.
The way I read "We live in an international world full of non-English scripts" is that the scripts meant are not computer code but things like the cyrillic script or the devanagari script.
I also read "executable math notation" not as meaning that it's a way of executing some specific other math notation (I don't think there really is a single standardised math notation) but a way of writing the specific executable math notation of APL.
If you ignore that the tooling is proprietary, it is pretty great. Most people can't/won't (myself included), so it's a bit more dubious of a claim under that light.
However, I also decided that it is still a bit too dead, at least for me. Specifically, it does not work well with other programs, at least on *nix. This is reflected not only in the lack of external libraries, but even in some basic areas – both FFI and even reading/writing to standard input and output are surprisingly challenging. Summing all this up, writing a basic Linux CLI app (read from stdin, take and parse arguments, print to stdout) is still a pretty large challenge.
As great as APL is, for me, it's hard to overcome the ability to work with the whole existing Linux/FOSS ecosystem. (I might feel differently if I did any Windows development, which does feel like much more of a first-class citizen for Dyalog APL)