Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

I'm seeing this as well. I don't quite understand how it's doing that in the context of LLMs to date being a "next token predictor". It is writing code, then adding more code in the middle.


Are you seeing a `<span>...` suffix when the model is in the middle of writing a code block? I'm pretty sure that's just a display bug in lmsys.


Is it something similar to beam search (https://huggingface.co/blog/how-to-generate#beam-search) or completely different (probably is not beam search if it's changing code in the middle of a block)?

(I can't try right now because of API rate limits)




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: