The T in TPU stands for tensor, which in this context is just a fancy matrix. These days both are optimised for matrix algebra, i.e. general ML workloads, not just LLMs.

If LLMs become unfashionable they’ll still be good for other ML tasks like image recognition.