This post has been de-listed
It is no longer included in search results and normal feeds (front page, hot posts, subreddit posts, etc). It remains visible only via the author's post history.
Today, we are releasing Mistral-7B-OpenOrca. This is the first 7B model to score better overall than all other models below 30B. We've achieved 98% of Llama2-70B-chat's performance!
thanks to MistralAI for showing the way with the amazing open release of Mistral-7B! So great to have this much capability ready for home GPUs. Also thanks to @winglian @Teknium1 @TheBlokeAI and all Alignment Lab AI friends! We're so close to beating a model 10X larger now!
check out our huggingface space for you to try our model live on fast GPUs in the browser right now!
We have used our own OpenOrca dataset to fine-tune on top of Mistral 7B. This dataset is our attempt to reproduce the dataset generated for Microsoft Research's Orca Paper. We use OpenChat packing, trained with Axolotl.
This release is trained on a curated filtered subset of most of our GPT-4 augmented data. It is the same subset of our data as was used in our OpenOrcaxOpenChat-Preview2-13B model.
HF Leaderboard evals place this model as #2 for all models smaller than 30B at release time, outperforming all but one 13B model.
This release provides a first: a fully open model with class-breaking performance, capable of running fully accelerated on even moderate consumer GPUs. Our thanks to the Mistral team for leading the way here.
We affectionately codename this model: "MistralOrca"
We are in-process with training more models, so keep a look out on our org for releases coming soon with exciting partners.
We will also give sneak-peak announcements on our server!, which you can find here:
or check the OpenAccess AI Collective server for more information about Axolotl trainer here:
Subreddit
Post Details
- Posted
- 1 year ago
- Reddit URL
- View post on reddit.com
- External URL
- reddit.com/r/LocalLLaMA/...