In Honour of This Year's NeurIPs Test of Time Paper Awardees
•
2
Jaward Sesay
Jaward
AI & ML interests
I like to train large deep neural nets too 🧠🤖💥 | First Paper (AutoAgents: A Framework for Automatic Agent Generation) Accepted @ IJCAI 2024 | Role Model Karpathy
Recent Activity
posted
an
update
about 24 hours ago
minimal single script implementation of knowledge distillation in LLMs. In this implementation, we use GPT-2 (124M) as student model and GPT-2 Medium (340M) as teacher via reverse Kullback-Leibler (KL) divergence, trained on a small chunk of openwebtext.
Code: https://github.com/Jaykef/ai-algorithms/blob/main/llm_knowledge_distillation.ipynb
liked
a model
2 days ago
deepseek-ai/DeepSeek-V3
Articles
Organizations
Posts
69
Post
880
minimal single script implementation of knowledge distillation in LLMs. In this implementation, we use GPT-2 (124M) as student model and GPT-2 Medium (340M) as teacher via reverse Kullback-Leibler (KL) divergence, trained on a small chunk of openwebtext.
Code: https://github.com/Jaykef/ai-algorithms/blob/main/llm_knowledge_distillation.ipynb
Code: https://github.com/Jaykef/ai-algorithms/blob/main/llm_knowledge_distillation.ipynb
Post
1323
Huge AI win in medicineđź‘Ź
"Large language of life model" just dropped!!
Full paper: https://www.nature.com/articles/s41586-024-08391-z
"Large language of life model" just dropped!!
Full paper: https://www.nature.com/articles/s41586-024-08391-z
Papers
1
models
3
datasets
None public yet