It’s like fake news is taking in science now. Saying any stupid thing will attract much more view and « likes » than those debunking them.
Except that we can’t compare twitter to nature journal. Science is supposed to be immune to these kind of bullshit thanks to reputed journals and pair reviewing, blocking a publication before it does any harm.
This is something I think many people don't appreciate. A perfect example in practice is the Journal of Personality and Social Psychology. It's one of the leading and highest impact journals in psychology. A quick search for that name will show it as the source for endless 'news' articles from sites like the NYTimes [1]. And that journal has a 23% replication success rate [2] meaning there's about an 80% chance that anything you read in the journal, and consequently from the numerous sites that love to quote it, is wrong.
The purpose of peer review is to check for methodological errors, not to replicate the experiment. With a few exceptions, it can't catch many categories of serious errors.
> higher retraction/unverified
Scientific consensus doesn't advance because a single new ground-breaking claim is made in a prestigious journal. It advances when enough other scientists have built on top of that work.
The current state of science is not 'bleeding edge stuff published in a journal last week'. That bleeding edge stuff might become part of scientific consensus in a month, or year or three, or five - when enough other people build on that work.
Anybody who actually does science understands this.
Unfortunately, people with poor media literacy who only read the headlines don't understand this, and assume that the whole process is all a crock.
partially due to legacy of science historically being rooted in "it matters more who you (or your parents) are" societies (due to them having had the money in somewhat modern history) (or like some would say the "old white man problem", except it has nothing to do with skin color, or man and only limited to do with old)
partially due to how much more "science (output)" is produced today and ways which once worked to have reasonable QA don't work that well in todays scale anymore
partially due to how many flows
partially due to human nature (as in people tend to care more about "exiting", "visible" things etc.)
People have been pushing for change in a lot of ways like:
- pushing to make full re-poducability a must have (but that is hard, especially for statistics based things only a few companies can even afford to try to run. But also hard due to it requiring a lot of transparency and open data access, and especially the alter is often very much something many owners of data sets are not okay with.
- pushing for more appreciation of null results, or failures. (To be clear I mean both appreciation in form of monetary support and in the traditional sense of the word of people (colleges) appreciating it).
- pushing for more verifying of papers by trying to reproduce it (both as in more money/time resources for it and in changing the mind set from it being a daunting unappreciated task to it being a nice thing to do)
but to little change happened in the end before modern LLM AI hit the scene and now it has made things so much harder as it's now easy to mass produce slob but reasonable looking (non) sience
> Science is supposed to be immune to these kind of bullshit
You have misplaced confidence in the scientific method. It was never immune to corruption, either by those deliberately manipulating it for their personal gain, or simply due to ignorance and bad methodology. We have examples of both throughout history. In either case, peer review is not infallible.
The new problem introduced by modern AI tools is that they drastically lower the skill requirement for anyone remotely capable in the field to generate data that appears correct on the surface, with relatively little effort and very quickly, while errors can only be discovered by actual experts in the field investing considerable amounts of time and resources. In some fields like programming the required resources to review code are relatively minor, but in fields like biology this (from what I've read) is much more difficult and expensive.
But, yes, science is being flooded with (m|d)isinformation, just like all our other media channels.
Yes. And let's not get started on that ML Quantum Wormhole bullshit...
We've taken this all too far. It is bad enough to lie to the masses in Pop-Sci articles. But we're straight up doing it in top tier journals. Some are good faith mistakes, but a lot more often they seem like due diligence just wasn't ever done. Both by researchers and reviewers.
I at least have to thank the journals. I've hated them for a long time and wanted to see their end. Free up publishing and bullshit novelty and narrowing of research. I just never thought they'd be the ones to put the knife through their own heart.
But I'm still not happy about that tbh. The only result of this is that the public grows to distrust science more and more. In a time where we need that trust more than ever. We can't expect the public to differentiate nuanced takes about internal quibbling. And we sure as hell shouldn't be giving ammunition to the anti-science crowds, like junk science does...
Almost nobody is "anti-science". The source of that labeling and division came from appeals to authority. You must do or believe this because it's "the science." If you don't, or you disagree, then you are anti-science.
It has nothing to do with science, but rather people not finding that a sufficient justification for unpopular actions. For instance it's 100% certain that banning sugary drinks would dramatically improve public health, reduce healthcare costs, increase life expectancy, and just generally make society better in every single way.
So should we ban sugary drinks? It'd be akin to me trying to claim that if you say no then you're anti-science, anti-health, or whatever else. It's just a dumb, divisive, and meaningless label - exactly the sort politicians love so much now a days.
Of course there's some irony in that it will become a self fulfilling prophecy. The more unpopular things done in the name of 'the science', the more negative public sentiment to 'the science' will become. Probably somewhat similar to how societies gradually became secular over time, as it became quite clear that actions done in the name of God were often not exactly pious.
Yes, there are a lot of people who are anti-science. As in they do not believe the scientific method is a good way to find truth. There are people today who are rejecting very basic science that was accepted over a century ago.
No, there aren't. Most people don't realize when they're being trolled by things like e.g. flat earth types. Go read one of the groups, it's a trolling meme largely turned into something by the internet + media. Thanks to social media even if let's say 0.1% of English speakers believe (or pretend to believe) something, that'd be 1.6 million people, so you can get a false impression, especially when the media takes the trolling and then amplifies it with a straight face largely to amplify these dumb and farcical divides.
I've seen science (the process, not just the results of that process) denounced as an aspect or tool of Western colonialism / imperialism. And there's that related "indigenous ways of knowing" thing that Canada has going on.
---
Also the flat earth people actually aren't trying to argue against science (the process). They're arguing that everyone except them made either observational errors or reasoning errors.
Science is a great way to model reality. It’s unclear whether science accurately describes reality. It’s also unclear whether science is capable of determining metaphysical truth.
- 15% of Americans don't believe in Climate Change[0]
- 37% believe God created man in our current form within the last ~10k years
(i.e. don't believe in evolution)[1]
I don't think these are just rounding errors.
They're large enough numbers that you should know multiple people who hold these beliefs unless you're in a strong bubble.
I'm obviously with you in news and pop-sci being terrible. I hate IFuckingLoveScience. They're actually just IFuckingLoveClickbait. My point was literally about this bullshit.
90% of the time it is news and pop-sci miscommunicating papers. Where they clearly didn't bother to talk to authors and likely didn't even read the paper. "Scientists say <something scientists didn't actually say>". You see this from eating chocolate, drinking a glass of red wine, to eating red meat or processed meat. There are nuggets of truth in those things but they're about just as accurate as the grandma that sued McDonalds over coffee that was too hot. You sure bet this stuff creates distrust in science
I think one of the most important 'social values' for science to thrive is a culture with a freedom to disagree on essentially anything. In most of every era where there was rapid scientific progress from the Greeks to the Islamic Golden Age to the Renaissance and beyond, there was also rich, and often times rather virulent, disagreements over even the most sacred of things. Some of those disagreements were well founded, some were... not. It's only in the eras where disagreement becomes taboo that science starts to slow to a crawl and in many cases essentially die.
Disagreeing with some consensus is not "anti-science". The term doesn't even make any sense, which is because it's a political and not a scientific term. I mean imagine if we claimed everybody who happens to believe MOND is more likely than WIMPs as an explanation for dark matter, to be "anti-science". It's just absolutely stupid. Yet we do exactly that on other topics where suddenly you must agree with the consensus or you're just "anti-science"? I mean again, it makes no sense at all.
I don’t think that’s quite right. Disagreement for the sake of disagreement is not particularly meaningful. The basis for science is iteration on the scientific method. Which is to say: observe -> hypothesize -> falsify.
Anti science means to make claims that have no basis in that process or to categorically reject the body of work that was based on that process.
People disagree because they hold a different opinion. In many eras publicly expressing differing opinions, let alone publicly challenging established ones, becomes difficult for various reasons - cultural, political, social, even economic. And I think this is, in general, the natural state of society. When people think something is right, changing their mind is often not realistically possible. And this includes even the greatest of scientists.
For instance none other than Einstein rejected a probabilistic interpretation of quantum physics, the Copenhagen Interpretation, all the way to his death. Many of his most famous quotes like 'God does not play dice with the universe.' or 'Spooky action at a distance.' were essentially sardonic mocking of such an interpretation, the exact one that we hold as the standard today. It was none other than Max Planck that remarked, 'Science advances one funeral at a time' [1], precisely because of this issue.
And so freedom to express, debate, and have 'wrong ideas' in the public mindshare is quite critical, because it may very well be that those wrong ideas are simply the standard of truth tomorrow. But most societies naturally turn against this, because they believe they already know the truth, and fear the possibility of society being misled away from that truth. And so it's quite natural to try to clamp down, implicitly or explicitly, on public dissenting views, especially if they start to gain traction.
> none other than Einstein rejected a probabilistic interpretation of quantum physics
That has been communicated to you wrong and a subtle distinction makes a world of difference.
Plenty of physicists then and now still work hard on trying to figure out how to remove uncertainty in quantum mechanics. It's important to remember that randomness is a measurement of uncertainty.
We can't move forward if the current paradigm isn't challenged. But the way it is challenged is important. Einstein wasn't going around telling everyone they were wrong, but he was trying to get help in the ways he was trying to solve it. You still have to explain the rest of physics to propose something new.
Challenging ideas is fine, it's even necessary, but at the end of the day you have to pony up.
The public isn't forming opinions about things like Einstein. They just parrot authority. Most HN users don't even understand Schrödinger's cat and think there's a multiverse.
A core component of the Copenhagen interpretation is that quantum mechanics is fundamentally indeterministic meaning you are inherently and inescapably left with probabilistic/statistical systems. And yes, Einstein was saying people were wrong while offering no viable alternative. His motivation was purely ideological - he believed in a rational deterministic universe, and the Copenhagen Interpretation didn't fit his worldview.
For instance this is the complete context of his spooky action at a distance quote: "I cannot seriously believe in [the Copenhagen Interpretation] because the theory cannot be reconciled with the idea that physics should represent a reality in time and space, free from spooky action at a distance." Framing things like entanglement as "spooky action at a distance" was obviously being intentionally antagonistic on top of it all as well.
---
And yes, if it wasn't clear by my tone - I believe the West in has gradually entered onto the exact sort of death of science phase I am speaking about. A century ago you had uneducated (formally at least) brothers working as bicycle repairmen pushing forward aerodynamics and building planes in their spare time. Today, as you observe, even people with excessive formal education, access to [relatively] endless resources, endless information, and more - seem to have little ambition in exploiting that, rather than passively consuming it. It goes some way to explaining why some think LLMs might lead to AGI.
You still gravely misunderstand what has happened and what the conversation in physics is. I'm not just making shit up or guessing, but I do have a degree in physics. There is much more nuance to this than you'd get from Pop-Sci or even basic classes. So I'll let you in on what physicists are talking about over beers and to one another. What's going on in the papers and between them.
What you seem to misunderstand is that science is not a mechanical process. It is artistic.
You have to find new ideas and you have to challenge conventions. It is about how you challenge these ideas. It is about how you prove you are right. You're right to say that claims are one thing and proofs are another, but this is why Einstein worked on this problem rather than stating it and moving on. There's a big difference.
Let's look at Schrodinger's Cat
I hold the same belief as Einstein and this is also true for most physicists. The cat is EITHER alive or dead. Regardless of our act of checking.
Here's the big problem... and is a big part everybody misses:
A photon is an observer. The cat is an observer. The detector that releases the poison is an observer. The literal particle that is being radiated from the isotope is, you guessed it, an observer. Literally everything is an observer. An interaction necessitates observation.
So when Einstein says "Do you believe the moon only exists when I look at it?" he's talking saying he isn't special. It would be silly to think that that happens. And truth is, he's right! We can be on opposite sides of the planet and you can see the moon while I can't and vise versa. But hey, maybe I don't exist and this is all in your head! So stop arguing with yourself I guess?
MOST physicists believe what Einstein believed.
Most of us don't believe there are these infinite universes spawning to "brute force search" the universe, checking literally every possible path. This infinite multiverse is the same thing as "we all live in a simulation". Instead, we believe that we simply do not have access to this information. That is a VERY different answer. But notice something critical, how do we differentiate the two? How would we differentiate the two? Unfortunately, so far, it looks like that is not possible. But we have reasons to believe one side over another. Given everything we know so far, the universe doesn't like to just needlessly use energy.
So we're presented with two (technically more) options:
1) There are an infinite number of universes, corresponding to all possible events as would be viewed by all possible observers. Thus, the universe is doing a brute force search on whatever its solution space is.
2) The information is unavailable
2 a) We can't access that information
2 b) We don't know how to access that information
Einstein believed 2b. Most physicists sit in 2, and I'm willing to bet even Max Tegmark believes #2 is right. More people are split between 2a and 2b, but no one is really ruling either option out. Certainly we hope the answer is 2b, but until someone proves that we can't access the information, we're going to be having this debate. Of course, there's actually one more answer that will change the debate: someone proves the answer is unprovable (this actually seems to be the current likely option btw).
We should also believe 2 because we have examples of both and they're quite common.
2a) If you get into actually doing physics work you will see how complex measurements actually are. You can't actually ever directly measure something, it is always through some chain of proxies. Really, the big question with the cat here is trying to come up with a clever solution so this. Maybe we're making false assumptions. Maybe we can detect the sound of the glass cracking when it releases the poison. Maybe the cat always purrs while it is alive. Those would be ways to indirectly determine if the cat is alive or dead. But it is a thought experiment for a reason.
2b) Heat is the great example of information loss. We have time, which forces a one-way computation. We can watch particles float around and progress from time t_0 to time t_n. We know that there was a unique path and a truthful answer to how these particles moved. But if you hand someone only the data for t_0 and t_n they will be unable to tell you what trajectories those particles took! They can only do this probabilistically!
That's doesn't mean all these potential universes exist, that just means we lost the information!
Similarly, the math says a blackhole is a singularity. A point where there's infinite density in an infinitely small space. But this doesn't mean this thing definitely 100% positively unquestionably exists. That's a laughable idea. There's other explanations that have yet to be ruled out. There's other explanations that have yet to be found!
So it must be the math that is "broken". Doesn't mean it is easy to fix, but it needs to be fixed. Our physics models are incomplete. That's okay! We still have work to do.
Think about what you've said. If it were true, no progress would ever happen. Like the universe didn't suddenly change when Newton and Leibniz invented calculus! Obviously our physics models back then were wrong. The question is how wrong. As best as we can tell, we are still converging. There is noise, but you can still converge with noise. Yes, there are big problems with academia today, and they should be pushed back against (I'm not shy about doing this myself if you check my comment history), but that's different than what you're suggesting.
So here's your mistake: you think your information is complete. Really, we have barely scratched the surface here. And go ahead, prove me wrong. There's a multitude on Nobels to be awarded for proving any of these points.
> Disagreeing with some consensus is not "anti-science".
Be careful of gymnastics.
Yes, science requires the ability to disagree. You can even see in my history me saying a scientist needs to be a bit anti authoritarian!
But HOW one goes about disagreeing is critical.
Sometimes I only have a hunch that what others believe is wrong. They have every right to call me stupid for that. Occasionally I'll be able to gather the evidence and prove my hunch. Then they are stupid for not believing like I do, but only after evidenced. Most of the time I'm wrong though. Trying to gather evidence I fail and just support the status quo. So I change my mind.
Most importantly, I just don't have strong opinions about most things. Opinions are unavoidable, strong ones aren't. If I care about my opinion, I must care at least as much about the evidence surrounding my opinion. That's required for science.
Look at it this way. When arguing with someone are you willing to tell them how to change your mind? I will! If you're right, I want to know! But frankly, I find most people are arguing to defend their ego. As if being wrong is something to be embarrassed about. But guess what, we're all wrong. It's all about a matter of degree though. It's less wrong to think the earth is a sphere than flat because a sphere is much closer to an oblate spheroid.
If you can't support your beliefs and if you can't change your mind, I don't care who you listen to, you're not listening to science
The stats you mention seem to suggest you are a believer in `The Science` - an anti scientific idea if ever there was one and one that's undergoing erosion day by day.
Me? I barely believe in the results of my experiments. But I also know what this poll is intending to ask and yeah, I read enough papers, processed enough data, did enough math, and tracked enough predictions that ended up coming true. That's enough to convince me it's pretty likely that those spending a fuck more time on it (and are the ones making those predictions that came true!) probably know what they're talking about.
According to your model, scientists who believe in God are anti-science.
That's almost weirder than declaring that 15% of people not believing in anthropogenic global warming is some sort of crisis. It's a theory that seems to fit the data (with caveats), not an Axiom of Science.
It's actually bizarre that 85% of people trust Science so much that they would believe in something that they have never seen any direct evidence of. That's a result of marketing. The public don't believe in global warming because it's "correct"; they have no idea if it's correct, and they often believe in things that are wrong that people in white coats on television tell them.
> According to your model, scientists who believe in God are anti-science.
In a way, yes. But every scientist I know that also believes in God is not shy in admitting their belief is unscientific.
The reason I'm giving this a bit of a pass is because in science we need things that are falsifiable. The burden of proof should be on those believing in God. But such a belief is not falsifiable. You can't prove or disprove God. If they aren't pushy, they're okay with admitting that, and don't make a big deal out of it then I don't really care. That's just being a decent person.
But that's a very different thing than not believing in things we have strong physical evidence for, strong mathematical theories, and a long record of making counter factual predictions. The great thing about science is it makes predictions. Climate science has been making pretty good ones since the 80's. Every prediction comes with error bounds. Those are tightening but the climate today matches those predictions within error. That's falsifiable
Climatologists have certainly invested much more in PR than geologists. So much so that their activities now look more like a global cult than science.
this seems strange to me, shouldn’t we expect a high quality journal to retract often as we gather more information?
obviously this is hyperbole of two extremes, but i certainly trust a journal far more if it actively and loudly looks to correct mistakes over one that never corrects anything or buries its retractions.
a rather important piece of science is correcting mistakes by gathering and testing new information. we should absolutely be applauding when a journal loudly and proactively says “oh, it turns out we were wrong when we declared burying a chestnut under the oak tree on the third thursday of a full moon would cure your brothers infected toenail.”
I think it might come to understanding what is "high quality" journal. Maybe such journal should be focused on much more proven and mature things. Where there would be lot less retractions as they have more mature and thus more proven information.
But I think problem is what is seen is "high quality" == "high impact". Which means that prestige and visibility is important things. Which likely lowers the threshold quite a lot as being first to publish possibly valid something is seen as important.
> shouldn’t we expect a high quality journal to retract often as we gather more information?
This is complicated, and kinda sad tbh. But no.
You need to carefully think about what "high quality journal" means. Typically it is based on something called Impact Factor[0]. Impact factor is judged by the number of citations a journal has received in the last 2 years. It sounds good on paper, but I think if you think about it for a second you'll notice there's a positive feedback loop. There's also no incentive that it is actually correct.
For example, a false paper can often get cited far more than a true paper. This is because when you write the academic version of "XYZ is a fucking idiot, and here's why" you cite their paper. It's good to put their bullshit down, but it can also just end up being Streisand effect-like. Journal is happy with its citations. Both people published in them. They benefit from both directions. You keep the bad paper up for the record and because as long as the authors were actually acting in good faith, you don't actually want to take it down. The problem is... how do you know?
Another weird factor used is Acceptance Rates. This again sounds nice at first. You don't want a journal publishing just anything, right?[1] The problem comes when these actually become targets (which they are). Many of the ML conferences target about 25% acceptance rate[2]. It fluctuates year to year. It should, right? Some years are just better science than other years. Good paper hits that changes things and the next year should have a boom! But that's not the level of fluctuation we're talking about. If you look at the actual number of papers accepted in that repo you'll see a disproportionate number of accepted papers ending in a 0 or 5. Then you see the 1 and 6, which is a paper being squeezed in, often for political reasons. Here, I did the first 2 tables for you. You'll see that has a very disproportionate ending of 1 and 6 and CV loves 0,1,3 These numbers should convince you that this is not a random process, though they should not convince you it is all funny business (much harder to prove). But it is at least enough to be suspicious and encourage you to dig in more.
There's a lot that's fucked up about the publishing system and academia. Lots of politics, lots of restricted research directions, lots of stupid. But also don't confuse this for people acting in bad faith or lying. Sure, that happens. But most people are trying to do good and very few people in academia are blatantly publishing bullshit. It's just that everything gets political. And by political I don't mean government politics, I mean the same bullshit office politics. We're not immune from that same bullshit and it happens for exactly the same reasons. It just gets messier because if you think it is hard to measure the output of an employee, try to measure the output of people who's entire job it is to create things that no one has ever thought of before. It's sure going to look like they're doing a whole lot of nothing.
So I'll just leave you with this (it'll explain [1])
As a working scientist, Mervin Kelly (Director of Bell Labs (1925-1959)) understood the golden rule,
"How do you manage genius? You don't."
https://1517.substack.com/p/why-bell-labs-worked
There's more complexity like how we aren't good at pushing out frauds and stuff but if you want that I'll save it for another comment.
[1] Actually I do. As long as it isn't obviously wrong, plagiarized, or falsified, then I want that published. You did work, you communicated it, now I want it to get out into the public so that it can be peer reviewed. I don't mean a journal's laughable version of peer review (3-4 unpaid people that don't study your niche and are more concerned with if it is "novel" or "impactful" quickly reading your paper and you're one of 4 on their desk they need to do this week. It's incredibly subjective and high impact papers (like Nobel Price winning papers) routinely get rejected). Peer review is the process of other researchers replicating your work, building on it, and/or countering it. Those are just new papers...
Except that we can’t compare twitter to nature journal. Science is supposed to be immune to these kind of bullshit thanks to reputed journals and pair reviewing, blocking a publication before it does any harm.
Was that a failure of nature ?