Can you go into that a little more - the switch to Golang and the old version being too slow?

What book?

While stretching most definitions of "periodical", Wait But Why https://waitbutwhy.com/ is one of my favorites!

Awesome tool! How do you handle scraping data that's hiding behind layers of ~fancy~ JS libraries? Is it as simple as triggering click events, pausing for loading, and then grabbing the information?

This tool basicaly performs the simplest data loading, it opens the webpage, then waits till most xhr requests are done, wait's a second (tio give JS time to manipulate DOM) and then loads data from the page. This way, it has what user sees when he opens the page in browser. So if the data is visible, or loaded through XHR or hidden in global JS variable it will see it.

For more advanced usage (like clicking, or submiting a search request) it would need to have some kind of scenario like: "Click on this" -> "wait till this loads" -> "type something here" -> "scroll to this" -> load data.

Which is possible with headless chrome, so the trick is to make it general and easy to use (something like recording what user does through chrome plugin). Maybe in future versions :)

Could be an interesting enhancement. Sounds a little bit like what Capybara, a test framework for Ruby apps can do[1], things like

  click_link('Link Text')
  fill_in('Password', with: 'Seekrit')
  choose('A Radio Button')
  check('A Checkbox')
  uncheck('Another Checkbox')
  select('Option', from: 'Select Box')

1) https://github.com/teamcapybara/capybara#navigating

I'd have money on using Horseman with phantomjs in node.

No, it says on the page it's using headless Chrome.

PhantomJS is dead. The arrival of Puppeteer rung the death knell.

Social media is also more consumable. It's easy to read a few posts while you're waiting for a few minutes in line or something. With a book, it takes time to get up and going with the story as well as hitting an appropriate stopping point.

You don't believe some training is at play here? If you get used to reading isolated snippets of text over and over and over, you might think a tweetstorm is a better medium than a book, but it probably has a far worse useful content ratio, and I can almost guarantee you that you'll have a harder time remember 90% of that social media than you would in that "less consumable" book that nonetheless is structured in a way to facilitate learning.

People have been taking books with them, reading parts, placing a bookmark, and returning five minutes later for centuries. Is social media really more consumable or does it taste good and we've gotten used to it?

I believe you're spot on. Training and being used to it is what I'd describe as an unfortunate reality. It reminds me of a few startups that I've seen who've created novels in the format of text message conversations to provide the content of a book in the format we've been trained for. I've definitely felt like my attention span isn't what it used to be when it comes to reading.

A large source of power for social media is that it is a 'pushing' force - it constantly is throwing notifications and other stimuli at you.

A book, on the other hand, does not share that same strength in power. But it has gotten better in pushing (i.e. Goodreads, Kindle, etc.).

Surprisingly I found out recently that reading on my phone was not that bad. I was sure I will not like it, and never gave it a try.

But visiting NYC this December, I found myself in situation GP described. Spending time in the subway, or waiting on a line for eating or buying stuff. I transfered the book I was reading (and really liked so I wanted to read more) from my tablet to my phone, and reading when it was possible.

On the phone, the pages are small, so you can read them quick.

Of course it was a novel and not some tech or science or whatever needs great focus, but I discovered a new way of reading.

So why not give it a try and see where it leads :) ?