I wonder if you could do this on the NN-generating NN from https://news.ycombinator.com/item?id=39458363.

Then you could have it start to create a vocabulary and an understanding around what all these parameters and weights do, since to us they're just a bunch of seemingly random floating point numbers.

At which point we may be able to start creating NNs from first principles, without even requiring training.