Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

This was a great point to make at the time, when people thought “my days has exceeded Excel’s row limit, therefore I should set up a Hadoop cluster and run Spark jobs against it”

Since then … it’s become a bit of a meme, unfortunately. Definitely there still exist workloads assigned to Spark clusters that could run on a laptop, especially if the data happens to be there already. But the space as a whole provides immense value, both enabling jobs that really don’t fit on laptops, and moving the compute for laptop sized jobs to where the data happens to be.



Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: