Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

Can someone share a good tutorial how to prepare the data? And for fine tuning, does a 3090 have enough VRAM? I want to do what the author mentioned by fine tuning the model on my personal data but I’m not sure how to prepare the data. I tried using vector search + LLM but I find the results very subpar when using a local LLM.


I just streamed this last night https://m.youtube.com/watch?v=TYgtG2Th6fI&t=3998s

I’ve been live streaming myself fine tuning llama on my GitHub data (to code like me)


Fantastic job! Very easy to follow


Thank you! I have some other streams where I do little projects like these check them out


As mentioned in the OP's blog post, checkout https://github.com/facebookresearch/llama-recipes.git. specifically files in ft_datasets directory.

I am able to finetune meta-llama/Llama-2-13b-chat-hf on a 3090 using instructions from quickstart.ipynb.


Oh interesting, I didn’t know the documentation expanded so much in this past month.


Working on this now!


I’m looking forward to this! Are you using an adapter (I don’t see it mentioned in your article)? I was under the impression you cannot fit 7B at 4 bit since it’ll take 25GB of VRAM or so.


Yes using the qlora adapter that hugging face provides with peft


ahh, I was on my phone before so I must of glimpsed over it, I see it on the last section. Thanks!




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: