I like reading about IPFS, but I do not really have the time to learn about it and get involved.
Last I remember, Z-Library was having an issue scaling the DHS to handle the number of files [1]. Did those issues get resolved? How is it going now? Also, is there anything being done to ensure every file has seeders?
I’ve dived deep into IPFS and built several prototypes on top of it. It ended up not being performant enough for me, and that was after heavily modifying the codebase so that it was true p2p browser & server (their webrtc transport had a lot of issues and they didn’t seem too interested in it, but my needs required it as a backbone).
The security was also a concern, and the scaling had issues. Pinning millions of small items got so slow it would not function. Then I ended up having concerns over hashed based addressing being easy to censor with the architecture IPFS was using (more hub & spoke than anything, given signaling and relay servers were centralized).
I could go on but I ran into so many issues I ended up implementing my own solution that did everything I wanted. Wanted to squeeze even more performance, I’ve been converting that solution to Rust.
This was a couple years ago so maybe things have changed since I used it. Last I checked, they seemed busy on Filecoin.
The idea of IPFS is great and I want to see it succeed, but I think that they got so caught up in their jargon and modularity, the project lost track of some fundamentals.
Honestly I think Consensys/IPFS/Libp2p is just some corporatized way to derail real P2P and decentralization. Their libraries are total garbage. Lots of complicated code that simply doesn't work. No documentation. I mean look how much IPFS and Libp2p is pumped but IT DOESN'T WORK. IPNS is a joke. All way overengineered crap that does everything but actually nothing. Look at the $$$ and pedigree behind Consensys it's 100% establishment.
Anything is possible, but having some limited github interactions with the core team, that seems unlikely. My impression is that they are a passionate group that hit the jackpot a bit prematurely.
Their intentions seemed good to me they just have an ungodly amount of financing while perhaps lacking a core vision & understanding of what is at stake.
Put another way, seems more like an academic building something rather than a seasoned industry pro.
In any case, I don’t want to disparage their project. I learned a lot from their code & concepts. We are all on our own roads towards brilliance, contributing to each other in all kinds of ways.
I certainly wouldn't disparage the talent- no doubt vast majority have good intentions but unfortunately they are being paid handsomely to have their work be rendered ineffective. Gobble up all the best P2P devs and get them focused on work that is largely ineffective. Lots of code and specs but nothing really usable. Code is not very original, they just take other libraries and tweak them. mplex is their version of yamux, autonat is their version of stun/ice... but then none of their libraries really work together seemlessly. Embrace extend extinguish? If you look at their massive code output then google "what apps use libp2p" you won't find anything because nobody uses it. Berty is really only app worth a mention.
I’m not saying conspiracy doesn’t exist — it most certainly does — but it just seems unlikely in most cases.
In this case, to be blunt, give a bunch of nerds millions / billions of dollars and see how fast their objectives change. Going from being students to having material wealth unimaginable to most suddenly gets you invited to a lot of conferences / parties / inner circles. You’re doing a lot less engineering and a lot more socializing & experiencing, and suddenly those lofty ideals & nerdy curiosities may just not matter as much any more driving around in a lambo with what you’d consider super models. But making even MORE money to do cool things just might.
Very few people exist on this planet that can resist that, and it is the explanation for most people / projects / orgs that make it big losing their way. Once you have a lot of money, suddenly life without it seems impossible, and you start to compromise values to secure more of it. It is a steep downhill journey.
Most projects are unoriginal. Just a derivation slightly improved on, or a recombination of prior works in a compelling new way. Nothing wrong with that; our universe is composed of relatively few atoms combined in all kinds of special ways.
you both are correct on all assumptions. and are now discussing how corruption with money within the existing system is both undetectable/deniable and effective.
ipfs is now, for all purposes, a honeypot against dweb. or at least a way for opposition (dweb proponents) to waste valuable time. as your experiences proved.
I think that this is a misunderstanding but I understand where this is coming from.
Let me share my 2 cents as the Developer Advocate for IPFS (employed by PL).
You claim that our work is largely ineffective but what are you referring to? The IPFS ecosystem is like a cluster of several ecosystems, e.g. Libp2p, IPLD, IPFS, Multiformats.
> Lots of code and specs but nothing really usable
Some of the libraries may not be as actively maintained as others — this is the nature of open source. But to claim nothing is usable is not accurate — there's a lot of stuff that works pretty reliably.
> Code is not very original
Who cares....
> autonat is their version of stun/ice.
It's actually Autonat, DCUtR, and Circuit Relay V2 which are pretty amazing given that it's like STUN/ICE but decentralized! Anyone in the network might help you hole punch through NAT and from network measurements the success rates average around 70% (https://youtu.be/bzL7Y1wYth8?t=1091). Saying it's just "their version of STUN/ICE) takes away from the main qualitative difference of doing it without centralized coordination servers.
> "what apps use libp2p" you won't find anything because nobody uses it.
Either way, I appreciate your comment, we need to be doing a better job of surfacing some of the great progress being made in the IPFS/Libp2p ecosystem.
Peer-to-peer is too general and too useful to be spoilt by "blockchain". But there is so much money behind this crap it is a formidable virus that can infect any project.
ConsenSys is a company formed by Ethereum co-founder. He has sold out to the big banks, in the opinion of some Consensys shareholders.
"Falls says he believes both MetaMask and Infura - what he calls "the crown jewels" of ConsenSys - could have been decentralized and tokenized, and that their projected use is "completely anathema" to the peer-to-peer principles of the space.
Meanwhile, a number of teams are looking to bring "institutional DeFi" - with its known counterparties and compliant custody arrangements - to the marketplace.
"Forget about the shareholders for a minute," Falls said. "Think about the consequences of the change in the influence over these infrastructure pieces.""
Out of curiosity I was looking at the evolution of IPFS hashes. They have gotten more complex. IPFS now uses base58btc exclusively. That's "btc" as in Bitcoin. Something like rhash, which has traditionally supported hashes used in peer-to-peer protocols, has no support for base58btc.
People are now trying to associate IPFS with "Web3". For example, check out this paper published a couple of weeks ago.
"Studying the workload of a fully decentralised Web3 system: IPFS"
Above I stated, "IPFS now uses base58btc exclusively." A more precise statement would have been something like, "IPFS CIDs originally used base58btc exclusively." It looks like IPFS is moving to a new CID protocol that allows use of non-BTC hashes. That's a step in the right direction, IMHO.
"In CIDv0, hashes are always encoded with base58btc. Always."
You can convert any CIDv0 to CIDv1, because the implicit prefixes from v0 become explicit in v1. However, because CIDv1 supports multiple codecs and multiple bases and CIDv0 does not, not all CIDv1 can be converted to CIDv0. In fact, only CIDv1 that have the following properties can be converted to CIDv0:
> IPFS now uses base58btc exclusively. That's "btc" as in Bitcoin.
It is true that Base58 is named for its original implemention in bitcoin but other than the name it has nothing to do with bitcoin.
The difference from Base64: It excludes non-alphanumeric characters and visually similar characters (I, l; O, 0) to make the output more human-friendly[0].
The problem it attempts to solve: Making bitcoin addresses more human-readable. That is likely also why IPFS uses it.
ConsenSys was founded by Joe Lubin, and is the corporate entity behind MetaMask, Infura, etc. It counts MasterCard and JP Morgan as investors.
Protocol Labs was founded by Juan Benet and is the primary sponsor of IPFS and the dev team behind Filecoin. They're funded primarily from the private token sales of FIL.
Agree with the sibling comment that the focus on modularity and solving "all of the things" rather than focusing on adoption seems to be a pattern with Protocol Labs, though I'm a big fan of what they're trying to build.
No, Protocol Labs it to IPFS as Blockstream is to BTC.
ConsenSys is an entirely different company focused on Ethereum, and neither invented nor maintain IPFS. That's Protocol Labs.
Protocol Labs didn't start out of ConsenSys either — in fact, I don't believe they have too much beneficial ownership in common at all. In addition to the separate founding stories already mentioned, you can read about how Protocol Labs was initially funded by Union Square.
The #1 lesson they failed to learn from Bittorrent and Tox: your DHT must be sessionless.
QUIC is not a solution to this particular problem. Although QUIC uses UDP, it still creates and manages sessions. This is incompatible with decentralized scaling. At the scale of Bittorrent and DNS you cannot manage this kind of session state without centralization.
IPFS encompasses more than just a DHT:
- representing content addressed data with hashes in CIDs
- Discovering them (via the DHT)
- Transfering (Bitswap, HTTP, Graphsync, sneakernet and more in development)
I'm not 100% certain what you mean by the DHT being sessionless.
> At the scale of Bittorrent and DNS you cannot manage this kind of session state without centralization.
You don't have to route content via the public DHT. Depending on what kind of data you're publishing, you can use a private DHT swarm, or delegated routing with HTTP.
> This is incompatible with decentralized scaling.
Absolutely. Too many people try to implement DHT on top of stateful connections, it doesn't scale. It must be as cheap as a single packet or two, with no context.
That should probably read "doesn't scale without centralization".
The full answer to that question is way, way, way beyond the appropriate length for an HN comment.
DHTs need to accept requests from any random IP address, anywhere on the Internet. If you allocate state, you have to deal with resource exhaustion attacks. Dealing with them turns out to be way harder than anybody thinks when they first start down this path.
Cloudflare and Google can use stateful HTTP(S) because they are so massive. They can employ hordes of people to constantly tune DOS filters to block griefers before state allocation occurs, and lawyers to deal with any griefers who are as large as they are. If you're happy with that kind of centralization you shouldn't be wasting your time with DHTs; you don't need them.
Aside from resource exhaustion attacks, there are some pretty deep lessons connected to systems and architecture. The bittorrent bootstrap nodes serve some really insane number of requests/second on very modest hardware (I'm still trying to find the link that detailed this...). The Internet root zone DNS servers were, for a very long time (and perhaps still today in one or two cases) just a few good workstations at universities.
Sessionless protocols scale like crazy, on cheap hardware. But they are very hard to design correctly. If you have VC bucks, spend them on hardware and use sessionful protocols.
You can restrict DHTs to accept well-formed requests and further restrict them using a web of trust. With a large enough network and an on onboarding path using nodes that have higher bandwidth, it can work to mitigate that vector I’d think.
It would be really hard to launch that attack with a DHT that implements appropriate pruning, drop, and block rules for bad behavior. All that can feed into a personal (& autonomous) web of trust. Any references for me to read more about this?
You can score member nodes and gauge bandwidth in the known network. Nodes can set their constraints accordingly and take more requests than others. Nodes that are open to edge & can handle a lot more connections can handle the bulk of onboarding requests.
If you find something, let me know. It would save me a lot of trouble.
Until then, what I’m building unfortunately doesn’t seem to have a peer. So I will keep at it. If it gets to a point where I believe it can be useful to others, I will share it freely.
I'll second that - even if the code is currently in poor condition.
IPFS sounds great but it has a bad reputation for performance/resource hogging but from your original comment that seems to be the least of its issues. I'd love to use something like that for my project.
Trust me, when I can share I will, and gladly at that. Hopefully it will be useful and many other people take up the cause. After that I’m taking a rest-of-my-life adventure around the world’s most pristine wildernesses.
I’m building a connectivity lib in Golang for relay-assisted p2p over tcp. The implementation is simple, uses only a single network connection when idle, and establishes connections very quickly and reliably. Downside is it’s not decentralized and it’s also not UDP, in case that’s your thing. Oh and BYO auth.
I just did a quick test, it could do with slightly fuzzier searching ('Epub' as an extension got me no result while 'epub' did, a drop down menu with options might be better/simpler for some fields) but, otherwise, it seems functional and useful.
I got an error when visiting zlib.zu1k.com/.
Error code: 1020
Ray ID: 7819e5389c898e4e
Country: IL
Data center: tlv03
IP: 85.65.187.66
Timestamp: 2022-12-30 09:54:40 UTC
It's possible they were "hugged" enough to instigate a firewall rule dramatically cutting down allowed visitors, which would explain the Error 1020.
Might be worth reaching out to the site's creator -- info taken from their site [https://zu1k-com/about/]:
"If you have any questions to communicate with me, the preferred way is to comment directly in the comment area of the corresponding blog post, and I will receive an email notification
In addition, you can also reach me in the following ways:
Email: i@zu1k.com
Twitter: @zu1k_lv
GitHub: @zu1k
Keybase: @zu1k
Matrix: @zu1k:mozilla.org
My PGP public key can be obtained at https://pgp.zu1k.com or hkps: //keyserver.ubuntu.com
People should stop over-using CloudFlare for everything. IMO it is not good for IFS "hosting", either.
They act like gatekeepers. Along with blocking, as in this case, they make browser assumptions that force visitors to use one of the most recent mainstream "approved" and unmodified browsers.
For other browsers, they serve "Checking if the site connection is secure" with javascript fingerprinting and a captcha, and sometimes even solving the captcha won't let me in. i have also seen frontends misbehaving due to them not expecting cloudflare error response codes.
Some dvelopers apparenly just blindly enable cloudflare and don't care. :(
Is the catalog on IPFS different to the catalog available via one of the libgen front-ends? I just performed a search on both for one title and it found it right away on a libgen front-end, but not here.
Z-library began as a Libgen mirror but split off and stopped sharing uploads. It's kind of scummy, but I guess that just means people need to put more effort into uploading its contents back to Libgen.
I had the opposite... a book I was looking for on a lib gen front end a few days ago but could not find came up here (the Narada translation of Dhammapada if curious)
Something that has always bothered me about z-lib and libgen and so on is the impossibility of being able to tell quality from the different uploads.
Sometimes the epubs are very good, with a good cover and metadata as well as chapters and no weird aberrations; other times, they're unreadable. But the only thing you have to judge quality is size and general description of the upload which in this website it's been reduced so everything looks the same.
I think the website looks good, but yeah, would be nice if we could rate quality in some way.
I think that really depends on the type of book. I've found the large PDFs to be by far the most reliable for books printed before the 20th century because - save for Google Books - no one's OCR is tuned for the kind of printing errors, age related wear and tear, and fonts used over the centuries.
Manuscripts (which are admittedly very niche) often have to be high resolution scans or photographs just to be readable, but zlib and libgen don't really have many of those.
They are but someone has to be in possession of the book and go through the trouble of digitizing it. Usually it’s museums, libraries, and private collections and a variety of enthusiasts eventually upload from those sources to Zlib/libgen. Since no one cares about copyright there anyway it can be more complete in some genres than other central databases.
A straight-text book can run as little as 500--750 KiB for 100--200 pages or so.
ePubs may run smaller as they're a compressed archive of HTML files, so how large the straight text is is largely a matter of the additional markup and stylesheets applied. I've generated PDFs from Markdown which are fairly comparable.
Most mainstream trade press books run about 3--6 MiB as PDFs, if they have few images or graphics.
Books with a heavy graphics content can easily swell to 30--300 MiB.
And scanned rather than generated PDFs tend to run similarly sized. Many of those scans are, however, excellent quality.
The largest ebook within my own collection for quite some time was a copy of Lyell's Geography, downloaded from the Internet Archive and based on a library scan of a 19th-century printing. On a colour tablet, it's actually pretty nice reading, on a B&W E-ink tablet, such scans often have a significant background ghosting which may or may not be eliminated by contrast adjustments in your reader.
For newer materials, I've seen any number of issues with ebooks:
- Clean single-page scans with OCR'd text for highlighting and copy/paste are generally my favourite. There's something about original print layouts that I still find preferable to ePubs and other digital-native formats.
- Native PDF can also be quite good though they're far more prone to designeritis, where someone has thought they could improve over ink-on-paper conventions. If you think you can do this, you are almost certainly wrong.
- ePubs rerendered as PDFs are among the worst options. I prefer PDFs generally, but not in this case.
- ePubs ... can ... be reasonably good, though again the principle problem is excessive flexibility for designers. Less is more. I really miss having a single consistent layout of the text, rather than something which reflows as font faces, sizes, and spacing are changed. As a saving grace, cringeworthy (and migrane-inducing) font choices can be overridden with The One True Serif Font.
- Various low-quality scans. 2-up, lots of page skew and placement variation, heavily-marked texts, and such. I'll still almost always prefer these to an ePub or other digital-native format, though the reading can be much harder.
- Tiny-font 3-column scientific publications. Even on my 13" e-ink reader, these can be a challenge, and I'll occasionally resort to a sub-page rendering (this is natively supported in the Onyx NeoReader and several other e-book readers). Why such publications continue to insist on this format as we approach the 2nd quarter of the 21st century I've no idea at all. Scans based on older journal pubs (from ~1950s -- 1980s or so) with both physical wear and scan artefacts can be especially challenging.
The books are free, if you download the wrong version of a book you can simply try another and it costs you no more than a minute of your time.
Do you know how it worked before we had the internet for books? You had to go to a library yourself, which may have taken an hour of your day just in the journey, then hope the library had the book you wanted. If not, you may have to wait weeks for another library in the system to mail the book to your library.
I'm describing a problem, the fact that the books are free and that this problem causes a minor inconvenience doesn't make it not a problem. It's important to be aware of these things.
Don't get me wrong, I do appreciate the access that libgen and z-lib provide and think on the whole it is beneficial to society. It is still not paying for people's work though.
As a counterpoint, have you looked at your local library's ebook offerings? Publishers have created a situation where digital books are significantly more expensive than physical copies and expire far more quickly. We're also just coming out of a two year period where physical libraries effectively didn't exist and most library systems hadn't expanded their digital offerings to meet demand.
I still buy books, but these sites have largely replaced traditional libraries for my partner and I because they allow us to read what we actually want instead of Oprah's book club titles or the 50 digital self-help guides my local library stocks with their limited resources. I recognize that they're simply serving what most people want, but the alternative for me is simply not reading recreationally.
The other side of this conversation is that even if we did get rid of pirate libraries there would still be huge, unresolved issues with author compensation. There's been a dramatic decline in how much authors (and the other people directly involved) take home over the past couple decades [1] that has little to do with the marginal number of pirate readers.
A big reason piracy exists is because the functionality of these systems is better than the official channels. There's a reason streaming services led to a dramatic decrease in piracy. There's also a reason that piracy is making a return. Yeah, pirates are picky and they push other services forward.
A question about z-lib, libgen, and regular libraries
Suppose I check an e-book out of my local library. Because Reasons, the library doesn't "own" unlimited "copies" of the book, so each "copy" can only be checked out to one patron at a time and if there are enough holds then a patron can't renew a checked out book. In short, ebooks in libraries are just like regular paper books, except to you don't have leave your house.
I had this ebook on hold for three months because it's very popular, but finally have it on my device, but I only have 3 weeks before I can no longer read it because of holds. I get about 2/3rds the way through before lock closes on the bits on my device. It's a very popular book, it will be 3 months before I can read it again.
I go online, use some service, find the exact same book, down to all but the locking bits, and download it, finish reading it a few days later, and then forget about it. I might delete it later if I need room.
Is that a crime? Have either the author or publisher lost money? On the one hand, I can say yes because if the libraries purchased more "copies", the waiting list wouldn't be so long, I might have been able to renew it and finish it. On the other hand, I wasn't going to buy the book myself, and the library has to balance budget and demand, so they probably wouldn't purchase additional "copies".
What ethical questions do authors and readers see here?
In many countries it is at least a copyright violation, although it should be noted that downloading (not sharing while torrenting) is legal in some places.
> Have either the author or publisher lost money?
On average, probably yes, because some people would buy the book, some libraries would get more budget or allocate more of the budget to that book, and most importantly, if you did re-borrow it in 3 months it'd further expand the waitlist, perhaps making someone go "screw that, I'm not waiting 4 months" and buy it.
OTOH, if you took all the money being spent on entertainment (books movies etc.), collected that as a tax, distributed it to the entertainment industry based on what people actually consume, and in exchange made everything available to everyone, there would be no loss of revenue and everyone would benefit from much higher, friction-free levels of access.
That's part of what made Netflix work. It extracted ~$120/year of revenue and gave access to everything. Now, with all the competing services, they probably still don't extract more than one service's yearly fee over the same population (because few people have more than 1-2 services, and the ones that just gave up and went back to piracy probably make up for the ones that now pay for 2 instead of 1 service) but people now have much less access.
Without a massive "media tax" which would have practical issues that don't make it feasible or desirable, some friction on free access is necessary or nobody would pay for content, and at the same time that friction harms society by limiting access. Piracy puts an upper bound on how much friction can be added before people go the yarrr way.
My layman's understanding of copyright law says you didn't commit a crime, but whoever sent you the book did commit a crime. My understanding is that it's not illegal to download a book, it's illegal to distribute a book. (Or other copyrighted work.)
No, it is illegal to download (make a copy of) the book. To be legal, you would have to merely view the book without making a copy. This is the operating principle behind all those video streaming sites. This isn't a comment on what OP should have done.
Doesn't this vary according to the location of the participants?
eg in some countries participation in some parts of it (uploading/downloading/etc) is legal, whereas in other countries it can be the other way around.
Legally, no. That question has come up regarding things like browser caching as well. But the precedent is that it is not illegal to merely view copyrighted material you haven't licensed or bought, even if you happened to seek out that viewing.
I did read that. The format he had wasn't really locked down, it was just a regular epub behind a crappy web-based viewer. The library epub downloads really do have DRM locks.
I think the central ethical question to piracy is that ultimately, buying a book someone wrote is fair to the author. The author is part of a society -- our planet, did something to contribute to society, and it's fair that we reward that person if we also benefit from their work. Therefore, whenever you benefit from someone else's work, how do we reward them.
Now, I think this question is nuanced. For example, let's take Harry Potter. That book has sold millions. In my mind, J.K. Rowling has already been rewarded by humanity for her work many times over, so if you were to pirate her book then I think that is a far lesser evil than pirating a book written by an author who has barely made any sales.
Basically, I view the question of piracy as follows: (a) have we rewarded the author enough for that specific work? and (b) if not, then it's not right to pirate their book, and I should buy it. I think everyone should ask themselves that question.
The same with movies. Marvel movies are a money-making machine and they have more than enough money to keep them going. An indie film much less so.
The question of whether an author has been rewarded enough is tricky. For example, if Elon Musk wrote a book, I would have no problem pirating it in a microsecond because society rewarded him enough. But maybe that's not so clear-cut because the editor might be poor. Thus, I think it's safer to go on the basis of sales rather than the wealth of the author, except of course if the author is self-published. In that case, if they are rich, then who cares. Of course, what is rich? Well it's hard to give exact numbers but if they have 10 million dollars that's definitely rich enough.
Not an answer to your question, but interesting context for how eBooks work with libraries. And how not checking out a second time effectively is a loss of one “use” that may have directed the library to spend more on this book, rather than another. Sounds like you’re aware. Loved this Planet Money podcast
You didn't say where you are. Copyright infringement tends not to be a criminal offence, unless you're doing it as part of a business (so if you print out copies of that ebook and sell them) or you do so much of it you distort the market (you distribute so many copies of that ebook you destroy the profits that publisher might have made).
The rights holder can sue for their losses, but this tends to be the cost of the item, so they tend not to do that.
That's really down to your personal ethics and how much you are willing to tolerate.
As you're not distributing it, the chance of someone coming after you for downloading a non-legal copy is almost non-existent, and pretty much 0 if you've taken some basic precautions.
It sounds like you've answered your own question there; that as you'd read the majority of the book, that there's no real loss to you finishing it for free.
Of the past twenty ebooks I've downloaded, 18 have dead authors and only one is alive.
Unless you focus on the very latest releases, most authors are already dead. If you round up just about any list of great works from the 20th century, you'll find that most of those authors are dead and have been for years. If you like 19th century books, all of those authors are dead.
It is difficult to determine the quality of content on websites like z-lib and libgen because the only available indicators are size and a general description of the upload, which can make all content appear similar and leave users unable to differentiate between high and low quality content.
What exactly is Z-Library right now? The domains were seized, and there doesn't seem to be a way to access the library or add new books to it. Is it now simply a collection that exists alongside libgen? Is it being run elsewhere? Does the Ipfs mirror get updated from somewhere?
I've always been able to access it via tor which is operating normally.
Just be sure to confirm a disposable email address with them or they'll never let you in. Registering an account by itself is no longer enough when using tor and their website will block you and not tell you why.
Sorry - missed this. "The" answers to almost all the questions in this thread could not be consolidated in a single post. Looking in the dedicated channels on the discord server dedicated to this topic will allow searching for all related info. Unfortunately posting invites in a large community like HN could only endanger the well being of the discord site - wish there was a DM option in HN (that I know of ... maybe there is?!?)
Unfortunately posting invites in a large community like HN could only endanger the well being of the discord site - wish there was a DM option in HN (that I know of ... maybe there is?!?)
Seeing that the US managed to extradite a British citizen from his own country for hosting links, I would definitely say so. The right way is to do this is to host this on tor or similar and only advertise it on the internet behind a good vpn.
It's ridiculous that this is what people need to do to help others, but that's what ridiculous laws and circus courts get you.
> Britain has authorised the extradition to the United States of a student who created a website allowing people to watch films and television shows for free, the interior ministry said on Tuesday.
> ..[The 23-year-old student] allegedly earned tens of thousands of pounds (dollars) through advertising on the TVShack website before it was closed down by US authorities.
> The student would be the first British citizen extradited for such an offence. He faces jail if found guilty of the charges, which were brought after a crackdown by the US Immigration and Customs Enforcement agency.
> His lawyer had argued in court that the website did not store copyright material itself and merely directed users to other sites, making it similar to Google.
> The lawyer also argued that his client was being used as a "guinea pig" for copyright law in the United States.
Anglo countries tend to aggressively extradite, even for things that are not major crimes by their own law, to the US over copyright related offences. I'd tread with extreme caution.
I don't know tantivy well enough to know if it'd be possible to put the folded in one field and the literal in another and then use the search syntax to choose between them
Nothing wrong only the site owner block it temporarily surely with the large number of requests since sharing otherwise I shared in a previous comment another site to access the tool:
Working on this now, cherry-picked audio from such AI systems abound, but submitting arbitrary text for audio synthesis requires a much higher quality bar to be listenable. Planning to have something out early 2023.
What's the best off the shelf solution you've found? I've been trying a few different apps to read my queued articles to me, but all of the voices are awful.
I wrote my own script to send article text through GCP text to speech, which I then put on a private podcast feed. If you go through the voices there’s some decent ones. I also found that speeding up audio in podcast app greatly improves listen ability. I also find picking an AI voice with an accent (for me British or aussi or Indian) helps trick my brain into tolerance.
However I also got tired of manually extracting text and running my script so now I mostly use: https://www.listenlater.fm though on this I just use the “Matthew”. For 5-10 min articles it doesn’t really matter, you get used to the voices so long as you can do something else in the meantime lol.
Honestly I haven't found one that I'd call really usable. That's why I'm building it myself. There are a lot of applications that are fine to listen to for like super short text (a few sentences), but for any long-form prose it gets intolerable IMO. And that's from someone who has listened to lots of intolerable audio from my own system.
EDIT: I'll also add that although there are *many* TTS apps, the true diversity is far less because most of these services are using Amazon Polly, Azure, or GCP's equivalents.
I've tried Coqui for generating ebooks and found the edge-cases to be too severe for this to be usable; Coqui has a nasty habit of having strokes, for instance "hello" may become "hellooooooOOoooooooooooOOOOoOOOOOoOOOOOooooooOOOOOOOOOOOoooooooooOOOOOOOooo...." With some manipulation of the input (like adding punctuation) you can get around this, but if you're trying to automate this for a whole book then I think you'll need a lot of trial and error before you get the input massaging heuristics right.
The best TTS system for generating ebooks (that I have used so far) is MacOS's TTS. The process is fairly straight forward; first break the book into sentences using the standard heuristics. Run each of those sentences through TTS to generate an audio file for that sentence, then create a subtitle file with the text of the sentence. Then stitch them all together with ffmpeg, optionally adding a dummy video track if your media player of choice needs one to display subtitles (mpv does not, but some do.) Now you have a subtitled "audiobook."
MacOS's TTS has a robotic quality to it, but it generally works and you can become accustomed to it fairly easily if you give it a chance.
That's the one I've been using mostly, but am fairly displeased with the voices. GP's point about nice samples that fall apart in real use is accurate.
The Google Books mobile app will read any(?) book in your library, and at least they used to support uploading your own books, although I haven't tried it in a while: https://play.google.com/books/uploads?type=ebooks and in the individual book, tap on it, tap the bamboo menu, choose "Read aloud"
Thanks - that is really goodand didn't realise that was something I already had installed.
I'd be happy to pay for a service where I can adjust the readers voice and speed, but this worked amazingly well, and the enhanced version of the audio is really good.
A Z-library who's who would be useful. Is there any relation between the original operators and the IPFS effort? What about the new .to domain that's as far as I can tell identical in functionality to the original z-library?
I think it lacks a lot of data, before when I search "head first" on Z-library, pretty sure it got a ton of results, and now only 5 pages. What's the reason behind this?
If you like a book, buy it. I do. I love physical books. But, my interests are varied enough that I need multiple sources - sometimes I’m only mildly curious. Libby/Overdrive has been a lifesaver in this regard.
I would love to have a digital library of my physical books though, and that’s where this is coming in handy because I don’t want to buy it twice, especially when the ebook almost costs the same as the physical one. It would be nice if there was a “for ${1..5} more, get the digital one too!” - but then there’s DRM to fight with also.
And you are absolutely sure that there is no causal relationship between publishers getting paid and authors getting paid?
Yes, the status quo is ridiculous, with libraries having to limit "copies" of ebooks and even simulating wear and tear of these on one side, and having to deal with DRM buying books on the other.
But according to at least one author that I‘d trust in the matter, publishers also provide valuable services to them, such as taking the financial risk of the first print run (including editing, marketing etc.) and paying authors upfront for it.
That said, I really hope that we‘ll see an "iTunes store for ebooks" (open format, no DRM) sooner rather than later.
So, you mean, the current copyright solution? That has seen creation of works skyrocket? Or do you mean a system where people can steal what they want because 'reasons' and 'they want to pay less because xyz'? Do you propose a system that takes ownership from authors and redistributes their works maybe? Why shouldn't owners get to set the pricing? Just because people want to steal doesn't mean it's ok to. Just because people don't like the price doesn't mean they should get to set it. The current system not working would look something like no works being released (because the author felt it wasn't worth it). The fact that people put in so much effort to steal these works shows the current system is in fact working. People wanting to steal does not equal not working. Authors can set their prices to free, but funny, they choose not to (as is their right).
For production the current copyright system kind of works (except the whole debate on remixes) I'll give you that but for keeping older content...
Try to find some lesser known book or newspaper from the 80s in epub legally or non-top films from the 90s and you'll see what I'm saying.
Everything is trashed just after it's being produced. The uncomfortable truth for those large copyright conglomerates is that right now it's the piracy scene which keeps most of the content alive, to the point they even have to pirate their own content to release it again sometimes.
And then there's the whole debate on the length of copyright itself, there's no conceivable damage in any shape or form of pirating the Beatles.
How about a system that - when the author / rights holder "abandons" a work by no longer actively publishing it, it automatically falls into public domain after 5 years? 70+ years after the author's death is literally a lifetime of people who may be unable to access copyrighted but abandoned work. This benefits no one.
Too many works are currently unavailable commercially/legally.
> Or do you mean a system where people can steal what they want because 'reasons' and 'they want to pay less because xyz'?
If you have a copy of something, and I make a copy of it, and you still have your copy of it, what was stolen? If I find a book in the library and take pictures of each page, is that also theft? What if I write all the words down? What if I use a hand scanner and OCR to copy and transcribe it, is that theft?
The obvious response is that you stole the author's income, by obtaining a copy of their work without compensating them for it.
So the obvious retort is that you may not have bothered to purchase the author's work, forgoing your opportunity to read it, thus the author did not lose their sale.
The conversation will then turn to defending the existence of copyright as an incentivization to produce valuable works worth reading, and then to how the publishing industry eats those incentives anyway.
In the end, no one will exit the conversation having gained any insight.
> So the obvious retort is that you may not have bothered to purchase the author's work, forgoing your opportunity to read it, thus the author did not lose their sale.
I can't speak for all pirates, but I will pirate things I want to sample and then often buy it if it's what I was looking for. At least with books. I haven't bought music in 20 years.
> The obvious response is that you stole the author's income, by obtaining a copy of their work without compensating them for it.
Given that I (as an European) pay extra fees for copyright management companies for each photography-enabled device, each memory device, each scanner and each printer, I should be enabled to make as many copies of any copyright-protected work as I want.
Which is incidentally how it works over here. What gets you in trouble is making works accessible to copiers, not owning copies.
These fees are an even worse solution than the current maze of DRMs and walled gardens.
Maybe they were an acceptable compromise in a time before e-books and e-readers, but they are neither fair to users, nor do they seem high enough to fairly compensate content creators without their normal revenue channels.
If the intention behind it is a kind of public subsidy for content creation: Sure, there are much worse things to spend tax money on.
But please do it properly then, i.e. something like a "public Spotify/Netflix" for books and films that correctly attributes view counts, rather than just handing money almost exclusively to already popular/financially stable creators as measured by past sales records.
Not an author, are you. Not a lawyer either. When you have spent years honing your craft and spent many months creating a work so that you can buy what you need to live and can't afford to support your self and your family without that income and then YOU give that work away - then I may find some value in your self serving questions. By the way, the answer to your questions is yes.
> By the way, the answer to your questions is yes.
You say it's theft to take pictures of the pages of a library book? I would need you to cite your sources please.
If it matters, I recently ordered 6 hardcovers from an author who's book that I downloaded from libgen was so impressively written, well sourced and thought provoking that I just had to put his best works (the book I "stole" by downloading, and 5 of his other works) in my physical library.
On the flipside, I recently wanted to read a book (different author) so badly but it wasn't available anywhere for less than $100 since it's out of print, so I downloaded it and don't feel bad at all. I haven't read that one yet, and if I love it I might spend the money for a physical copy, but it wouldn't matter since it's out of print and used and wouldn't make a difference to the author (also, he's dead).
Germany has a system where the books purchased by libraries and also printers and scanners contain a fee that contributes to a fund for reimbursing copyright holders (Verwertungsgesellschaft Wort, cf. https://www.vgwort.de/startseite.html ). Very fast copiers/scanners are more expensive - inter alia - for that reason, according to a colleague of mine (I'm not an expert in this).
So you copying one page from a book in a library would still lead to some form of compensation, although one could debate whether it is fair or not, whereas using a P2P search engine to locate an online copy of a PDF would not.
Also, IMHO, the discussion of such online libraries should separate the topic of fair compensation of authors (I believe Norway or Finnland have "author stipends" paid by the government that one can apply) and the topic of locating and researching whether a book is what one needs or to one's liking.
I would imagine authors get compensation from their books purchased by US libraries too. But it's not going to scale with how many times a book is checked out, or even prevent it from being copied after checkout.
imagine if you built a building and then someone purchased it... and everyone that visited it also had to also pay you.... thats a bit like how copyright is today...
But what is even worst is that they are trying to make you pay them forever (subscriptions).
I wonder why the very concept of public library is not illegal at the first place..
I mean, imagine this concept is being popularized at first in the modern era. I'm pretty sure there's a huge opposition, using the very same argument as digital pirates & AI art debate : "It could damage author's income! People will just go read it for free in public library instead of buying it! Public library is morally wrong and should be burned!!"
That is a crude rendition of the AI art debate, and also of piracy.
A public library buys the book. Many people can read it, but it guarantees one purchase, and often many libraries will buy the book. It also limits the amount of readers to those who live relatively nearby. Books were also traditionally written with the ideas of libraries in the author's mind, and also many authors benefited themselves from libraries.
AI Digital art has many facets which make it detestable for some: it floods the market with stuff that can be generated at near zero cost, it weakens the connection between people because people are no longer looking at something another human created, it cheapens the spirit of creation.
While libraries, Library Genesis, and piracy might have some facets in common, they are separate entities and should not be conflated in any sense.
By that logic, then controlled digital lending -- where a public digital library buy one digital book and "lend" it by artificially limiting the amount of concurrent reader -- should be legal then, since it simulates the limitation of physical library in the digital realm.
The DRM ebook deals made by libraries don't have the same properties as physical book lending. The library is essentially covering the rent charged by the ebook publisher.
I don't know how old you are but there was initially a very large amount of opposition to digitizing media delivery over traditional methods for fear of piracy. It was a huge problem for Netflix.
FWIW https://libgen.rs already has good search for basically the same books. I like the denser layout that makes better use of whitespace than this HN submission does.
I think the best thing about libgen is their detailed filter to search books with covers which makes it really easy to scan books you are looking for assuming you know the cover pages.
ffs is there any way people can just not talk about z-library for a year..what is wrong with you all? this is a public forum. you should not share any copyrighted material without the rights to do so
My buddy has nudes of his wife on his phone. I want them so I bypass his 'drm' that prevents me from taking what I want and make copies. He still has his copy so no one's hurt. Information wants to be free. I take what I want.
I hope every project you work on has it's source code stolen resulting in failure. And this is coming from a pretty crappy person, but this website sure does love stealing from authors (so that it can enjoy the fruits of their labor).
A wealth-indexed fee averaging about $5/mo per household in the US and collected through broadband providers could provide access to every work ever published, pay authors, and eliminate both the need and practice of piracy.
The publishing industry prefers, however, to hoard its Smaug-gold, make criminals of the curious, deny access both to the vast majority of the population and to the vast majority of works, all to line its own pockets at an immense cost to the rest of us, authors, artists, musicians, and other creators included.
I am opposed to copyright in its current form, but this is a very naïve suggestion. How exactly do you distribute the fee among authors? Who collects it? How does dispute resolution work, if I don't feel like I'm being compensated fairly? Can I opt out if I don't read books? What if everyone opts out and just continues pirating?
Let's say I just published one hundred books of semi-AI generated bad fiction. When can I expect my cheques?
Yes, legislatively-sanctioned rent seeking is common in Europe. In what way has the existence of such a scheme ameliorated the excesses of copyright law in Europe? Copyright terms are the same in Europe as in the United States (life + 70), but Europe tends to have weaker fair use provisions than the US (e.g. 'fair dealing' in the UK and Ireland). In what way have these schemes ended piracy, as the original commenter suggests they would?
Q: How exactly do you distribute the fee among authors?
The model of mechanical royalties already exists in the music industry. That provides a useful model for other copyrighted works. Effectively this creates a regulated, universal, legally-defined market. I can see some variations in rates based on content types, discussed in previous HN discussions.
A "~$5/mo per household in the US and collected through broadband providers", as stated in my parent comment.
Q: How does dispute resolution work?
Ownership disputes are adjudicated as they are today.
Auditing disputes are adjudicated as they are today, modulo restrictions against binding arbitration and other conditions as discussed by Cory Doctorow and Rebecca Giblin in Chokepoint Capitalism: <https://bookshop.org/p/books/chokepoint-capitalism-how-big-t...>
Payment rate disputes are addressed through a regulator, genre / content-tier basis, and the legislative / political process reflecting interests of the public, creators, publishers, and other stakeholders.
Q: Can I opt out if I don't read books?
No.
On the basis that the common wealth created by others having access to information is a public good and benefit, much as police, fire, health, sewerage, and defence services are. Note that the cost is indexed to wealth/income, such that those of limited means pay little or nothing. Keep in mind that you are already paying roughly $800/year per head for content access through advertising. This isn't an additional fee, it's a restructuring of an existing financing model with many perverse outcomes in terms of manipulation, the type(s) of content incentivised, a pervasive surveillance model, and frauds against both advertising targets and advertisers.
Q: What if everyone opts out?
See above.
Q: What about AI-generated crud?
This is where I see a graduated compensation model with a number of tiers. AI-based glurge would see rates based on the costs of production, as would investigative journalism. The former would result in very low compensation rates, the latter should see higher. Quality-determination by various means might well be an element. Whilst individual negotiation by bidders isn't something I see as part of the model, a collective bidding based on sectors, guilds, and/or major aggregated publishers could be.
> The model of mechanical royalties already exists in the music industry.
I refer to your earlier post, though:
> …could provide access to every work ever published, pay authors, and eliminate both the need and practice of piracy.
I don’t think the model of music royalties, as it exists, could or does any of those things, nor do I think the model could ever be adapted to achieve those ends.
Moreover, I don’t think in a free society there can possibly be a place for a governmental or quasi-governmental body that would somehow assess the objective quality of all art, and decide on the compensation of artists accordingly. It would mark the death of all avant-garde art, and the rise of a massive rent-seeking class of mediocre and unchallenging artists. Not to even mention the implications for speech: there’d be no more artists holding up the mirror to the face of power, no more social criticism through art.
I don’t often say this, but that is a truly chilling vision of the future. Thank God it will never happen.
Apologies for the late reply, I've only just seen yours. If you'd like you can contact me by email to continue the discussion (HN username at protonmail).
Q: I don’t think the model of music royalties ... could do any of these things ...
My larger point is that the money is already in the system, and is being paid by households. Only that it's via advertising with multiple perverse effects on the media ecosystem itself.
You're being vague as to what the failings of a royalty-based system are, and whilst I could presume some of those problems and/or possible solutions (<coff> UBI <coff>). Much of what I'd suggest revolves around a number of concepts:
1. That content-creation as it exists is a high-stakes, infrequent-reward, winner-take-most sweepstakes. Much of existing publishing and distribution actually mitigates for this, as with "tentpost" cinema, music, and publishing titles and/or talents.
2. That in-depth detailed work in journalism, education, science, and technology especially relies strongly on durable institutions with a reliable income as well as defence mechanisms against the various forms of attacks and corruption these draw, e.g., libel, co-option. Large publishers often have strong legal defence teams, as well as contracts. Educational institutions, NGOs, publishing groups, news organisations, and creators' guilds might be possible such durable organisations. Groups need not be large. I'm reminded of visiting a local library to inquire about free room reservations: these were available to any citizen group requesting one, where a group was three or more people. There's power in numbers, especially when given an organisational banner.
3. That "reward for audience" is itself a fairly poor compensation metric. Yes, there are justifiable winners, but there are far too many unjustified losers, or works recongised only at much remove for their quality. Some way of balancing this out might be useful, and I'm open for ideas and criticisms.
- The system I propose would not be the only compensation mechanism, but it would be a guaranteed one (perhaps again with other mechanisms such as UBI and group/guild organisations).
- I'm honestly baffled as to why you think that dissenting voices would not have access to such a system. It is universal to both audience and creators.
- The extant system has marked the death of numerous forms of desirable and even essential authorship, most notably quality, incisive, independent local journalism, but extending to artistic works in text, audio, and film.
- What conditions would promote the types of works you describe? How well, exactly, does the present model work for those?
- Complaints about the rise of a massive rent-seeking class of mediocre and unchallenging artists in light of the present state of publishing seem ... archly ironic.
Again, if you care to follow up by email, invitation's open. I'm also @dredmorbius@toot.cat on the Fediverse.
Strong agree, I still haven't forgiven myself for that time ABBA starved to death after I taped "Dancing Queen" off the radio.
I use OverDrive/LibGen etc because it's generally more convenient than the public library. If I like a book enough to actually finish it, I buy a paper copy because ebooks are inherently inferior and probably always will be. If I was still a student I would use OverDrive/LibGen etc specifically to steal from textbook publishers who charge $100 for an 18th edition riddled with factual and grammatical errors in hope that they would go bankrupt, but it wouldn't work. We have a long way yet to go.
I won't say if I have or haven't downloaded ebooks without paying.
But I will say that I have bought both ebooks and physical books from authors that I like and appreciate the work and effort they have put into their craft. I've spent rather a lot on it, actually.
No one is entitled to a particular form of revenue. We live in an attention-based economy. Talented and savvy authors can find any number of ways to derive revenue from the attention their work attracts apart from charging IP tolls.
If not, they can always get a job where they have no ownership stake in the fruits of their labor, like almost every other job and 99.9% of people writing code for money.
Whilst I've some sympathies with this argument, it dismisses the larger question of how to provide societal benefits of information and entertainment creation.
It's quite possible that the size of such sectors today is excessive, though there's a strong argument that information is under-incentivised whilst entertainment is excessively incentivised, to say nothing of various forms of manipulation and propaganda, or the negative impacts of surveillance and fraud that are intrinsic to the adtech field.
So the question of incentives, compensation, financing, and revenue collection do deserve addressing.
I do agree that the present model is addressing those specific elements quite poorly.
Except that making a living as an author or via another creative pursuit is a luxury and privilege premised upon the hard labor performed by countless others which actually reproduces our society.
It’s also not really a matter of entitlement. It’s just an objective fact that trying to gatekeep knowledge behind IP toll bridges is a losing battle which only serves to hinder the development of many realms of human endeavor.
You want to kill the goose so you can consume the eggs. You want to steal their products but condemn the creators. You must not know creative types and how much most of them sacrifice to try and create from the passion inside them.
Actually I do. Their passion doesn’t entitle them to anything, and their sacrifices pale in comparison to those made by people who do hard labor for a living.
IP laws don’t protect small creators anyway, most of whom will fail regardless. They protect the big conglomerates from multitudes of potential creators who might otherwise find education and success outside of the established institutions.
Last I remember, Z-Library was having an issue scaling the DHS to handle the number of files [1]. Did those issues get resolved? How is it going now? Also, is there anything being done to ensure every file has seeders?
[1] https://news.ycombinator.com/item?id=33716560