Why is 4090 so slow in your tests? It regularly demolishes my 3090 in my own tes... | Hacker News

Hacker Newsnew | past | comments | ask | show | jobs | submit

		treprinum on Aug 10, 2023 \| parent \| context \| favorite \| on: Making AMD GPUs competitive for LLM inference Why is 4090 so slow in your tests? It regularly demolishes my 3090 in my own tests (2x speedup in most tasks).

junrushao1994 on Aug 10, 2023 [–]

LLM decoding is dominated by memory bandwidth, and 3090Ti and 4090 happen to have the identical theoretical memory bandwidth

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact