Ofc I wouldn't us it for extremely high scale event processing, but it's great default for a message/task queue for 90% of business apps. If you're processing under a few 100m events/tasks per day with less than ~10k concurrent processes dequeuing from it it's what I'd default to.
I work on apps that use such a PG based queue system and it provides indispensable features for us we couldn't achieve easily/cleanly with a normal queue system such as being able to dynamically adjust the priority/order of tasks being processed and easily query/report on the content of the queue. We have many other interesting features built into it that are more specific to our needs as well that I'm more hesitant to describe in detail here.
So perhaps don’t use kafka at all? E.g. Adyen used postgresql [1] as a queue until the outgrew.
In this case it seems there are a lot of things that can go south in case of major issue on the event pipeline. Unless the throughput is low.. but then why kafka?
RDBMS are pretty well understood and very flexible, more still with the likes of JSONB where parts of your schema can be (de)normalized for convenience and reducing joins in practice. Modern hardware is MUCH more powerful today than even a decade and a half ago. You can scale vertically a LOT with an RDBMS like PostgreSQL, so it's a good fit for more use cases as a result.
Personally, at this point, I'm more inclined to reach for a few tools than to try to increase certain types of complexity. That said, I'm probably more inclined to introduce valkey/redis earlier on for some things, which I think may be better suited to MQ type duties without an actual MQ or more complex service bus over PG... but PG works.
Especially for systems that you aren't breaking up queues because of the number of jubs, so much as the benefits of a logical separation of the work from the requestor. Email (for most apps), report generation, etc... all types of work that an RDBMS is more than suitable for.
I became fascinated by Lean last week while watching Terrence Tao stream his proof work on Youtube. At least on the surface his thought process and workflow looked very similar to what I do in my IDE when programming, especially when coding in a very strongly typed language like Scala with FP style.
If I can work through math proofs in an IDE with Lean helping check my work I might finally get around to diving into the higher level math I’ve always wanted to. Could open some doors to us working folks that don’t have an instructor to review our proof attempts.
Even worse for me, some of my coworkers were doing that _before_ coding LLMs were a thing. Now LLMs are allowing them to create MRs with untested nonsense even faster which feels like a DDOS attack on my productivity.
Companies should have more limited speech than individuals. Nerfing the concept of “corporate personhood” will be a key part of fixing our problems IMHO.
Only if they’re wealthy or get extremely lucky and live near a randomly good school. By many metrics I was the smartest in my class, but my family had little money and lived in a rural area with a single underfunded school. I spent my days in class with kids that were still struggling to sound out “cat” in third grade. A few times a week I got to spend an hour in “gifted” class but that was mostly art projects, nothing that would help make up for the rest of the day being wasted.
I think of ADHD and autism as brothers, or at least cousins. Lots of overlap, but not quite the same.
With the older concept and diagnosic criteria for autism there was less overlap. ADHD and autism used to be considered mutually exclusive diagnoses. Autism used to imply delays in speech and most people diagnosed also had significant intellectual disability. Autism is now a dramatically larger umbrella than it used to be with the last couple of DSM editions merging Aspergers and other conditions with it and making it not be mutually exclusive with ADHD. The shift in definitions makes autism extremely difficult to discuss in public because people have wildly different concepts of it depending on when they learned.
Side comment, merging Asperger's into autism was motivated in part by awareness of Hans Asperger's evil deeds and not wanting that association. It was also championed based on enabling help/protections given to autistics to folks with Aspergers as well, and maybe because being part of a larger and harder to ignore pool would help autistics with intellectual disabilities get more attention as well. So far I haven't seen a clear explanation of why it made sense in terms of better understanding the condition/difference, everyone that discusses it focuses on perceived potential social good.
A big factor was the defining feature of Asperger's being a lack of language delay, which turned out to be poorly correlated with life outcomes. The categories were also used very inconsistently - a lot of people, and some clinics, thought the Asperger's distinction implied far more than it did on paper, but not everyone went along with that.
> the idea that, "Every should must come reinforced by a want."
I haven't heard that before...I need time to ponder and process that, but on the surface it sounds like gold!
The parents and town I grew up with were extremely religious and authoritarian. I grew up hearing how our personal desires should be ignored and suppressed, and only the will of God and authority figures matters. I think neurotypical people may be more able to function with such a mindset, but it's a losing battle for ADHD brains to try to force themselves too far from their natural wants/interests. Especially since lack of emotional regulation is a significant(but less discussed) difference for ADHD, focusing first on emotions and reorienting wants before focusing on getting the task at hand done might make a lot of sense for us.
From decades of experience and lots of reading it's exactly this, and this is the important detail that most neurotypicals including many doctors are missing that would allow them to understand instead of being dismissive.
I find monotropism an apt way of understanding it. A normal person's attention is like a flashlight they control that illuminates much of a room at once. Autistic brains are a tight beam flashlight, almost a laser for some, with its aim difficult to change. ADHD brains are more like a tight beam flashlight on a motorized mount that swivels in all directions, but you're not always in control of where it swivels to...it's like an AI constantly overrides your direction inputs and points the light at what it deems most exciting or urgent at the moment.
> but you're not always in control of where it swivels to.
If we're going with that metaphor, I'd phrase it more in terms of that mount being prone to jamming. My attention doesn't go to completely random places, it only goes to places I want it to, but then gets stuck on things that wouldn't hold the attention of a neurotypical person.
> If we're going with that metaphor, I'd phrase it more in terms of that mount being prone to jamming
> but then gets stuck on things that wouldn't hold the attention of a neurotypical person.
Indeed, attention is frequently held for many hours or even days on the current subject of hyperfocus. I too have great difficulty deliberately directing my attention toward a subject I don't find exciting, but find it easy to get locked in on my natural interests. Hyperfocusing on one of my passions is delightful, even if it detracts from other important things.
> it only goes to places I want it to
That's true for me in a sense, but I've had so much trouble from hyperfocus states that I don't think about it quite that positively. I hyperfocus on a new hobby every week and buy tons of stuff for it that I abandon for the next week's hyperfocus. I get locked into politics and flame wars that make me stressed and sad the whole time, but waste hours on them. I would like to get locked into one of my passions that I always enjoy and that also enrich me in some way, but I end up spending hours hyperfocused on researching some esoteric topic of the day instead. Or worst case I hyperfocus on potential sources of doom and how to prevent them, but maybe that's not due to ADHD alone. All of those are things I "want" to focus on in some sense, but in many senses I very much don't want to.
That's why lack of control of the focus' subject seems one of the most important aspects to me. In my metaphor I guess I'd say something like the focus of the AI that's overriding my direction inputs is what's getting stuck on a subject.
> that I don't think about it quite that positively.
I don't think about that in a positive light, it's just factual information that helps me understand the shape of the problem. It's about recognizing that the places my focus goes to aren't random and arbitrary — I've never gotten stuck on, say, the intricacies of tap dancing, because I have little to no interest in tap dancing, and no reason to care. I did get stuck on knitting for a short while because I bought some pattern books as a gift for my mother, and had a peek inside — I _wanted_ to have a peek inside, what I didn't want was to lose a few hours to it.
Is it possible that autism is like a mode that gets switched on by the environment and in modern day it doesn’t switch off?
In many parts of the world during tribal times, feeling like you didn’t have control and feeling like the world is unpredictable is probably going to lead to death. Hyperfocus on getting things under control (in specific areas) would be useful, for example hyperfocus on shelter before winter if you feel like that is not under control, or hyperfocus on the food supply if that is not under control.
In the modern world it’s plausible to me that the conditions are just so out of control of the individual that the mode never turns off, and you get random seeming intense focus and irritability around lack of control and lack of focus on social fluidity (which in the tribal environment was basically N/A since everyone was basically the same as each other and you didn’t need these flexible protocols for socialization) etc etc.
I’m not diagnosed with autism but I feel like I have a lot of the classic traits and when I pay attention to my environment a lot of things bother me and I feel better when I fix them. But so many things bother me that it’s basically impossible to fix them all and then I disassociate from them. The things that bother me are like “this design is bad, fix it” but I can’t change all the objects around me like one could in a tribal setting where everything was made in house. Same with routine that is impossibly complex compared to tribal life.
> or set any ground rules that apply specifically to their joint activities.
I suspect we can, and I know we should make corporations not have ALL of the same rights as a citizen. The first thing that comes to mind is barring them from political donations. It would also be great if their “free speech” didn’t extend to being able to censor legal content they don’t like or to payment and Internet infrastructure providers being able to cut off service to sites with legal content they don’t like(porn, certain politics, hateful content, etc.).
I work on apps that use such a PG based queue system and it provides indispensable features for us we couldn't achieve easily/cleanly with a normal queue system such as being able to dynamically adjust the priority/order of tasks being processed and easily query/report on the content of the queue. We have many other interesting features built into it that are more specific to our needs as well that I'm more hesitant to describe in detail here.
reply