It's perfectly possible to make money recruiting by matching exactly what your customers want, but at that point, you are exactly like every other recruitment firm out there. What separates you? Asking people to do awesome CTFs doesn't really help of success at CTF doesn't line up with success at passing your typical Silicon Valley interview.
Anyone that has spent time working outside of firms that hire this way (which includes stockfighter's founders), realizes that the technical interview today is very expensive, throws away a lot of candidates, stifles all axis of diversity, and ultimately it's not really all that predictive of on the job performance. A CTF is not an ideal predictor either, but I'd argue it's better.
Stockfighter found that their best candidates by the CTF were receiving terrible ratings by the companies they send them to, because they lacked the shibboleths required to pass the interviews. They'd only have success if the company filtering after being presented a candidate was minimal, and then they saw that yes, the job performance of the candidates that were good at the CTFs were at least as good as those that came through the traditional recruiting process. Apparently there wasn't enough appetite for that kind of process, and thus, Stockfighter made no sense as a company.
That's a pity, because I share their view that technical interviewing is broken. I just don't see how we can systematically improve it without the kind of data that you can only get at huge companies that hire thousands of engineers a year.
Anyone that has spent time working outside of firms that hire this way (which includes stockfighter's founders), realizes that the technical interview today is very expensive, throws away a lot of candidates, stifles all axis of diversity, and ultimately it's not really all that predictive of on the job performance. A CTF is not an ideal predictor either, but I'd argue it's better.
Stockfighter found that their best candidates by the CTF were receiving terrible ratings by the companies they send them to, because they lacked the shibboleths required to pass the interviews. They'd only have success if the company filtering after being presented a candidate was minimal, and then they saw that yes, the job performance of the candidates that were good at the CTFs were at least as good as those that came through the traditional recruiting process. Apparently there wasn't enough appetite for that kind of process, and thus, Stockfighter made no sense as a company.
That's a pity, because I share their view that technical interviewing is broken. I just don't see how we can systematically improve it without the kind of data that you can only get at huge companies that hire thousands of engineers a year.