5
Has anyone gotten JSON working with Llama.cpp python and Llama 3 8b?
Post Flair (click to view more posts with a particular flair)
Post Body
I'm just slotting it in to my old mistral set up and it runs fine for single requests, but when I try to force JSON output it just hangs on the request without returning an error.
Using a q8 GGUF from a few days after release on OS X if that matters.
Author
Account Strength
100%
Account Age
14 years
Verified Email
Yes
Verified Flair
No
Total Karma
191,930
Link Karma
20,358
Comment Karma
168,560
Profile updated: 1 week ago
Posts updated: 6 months ago
Subreddit
Post Details
We try to extract some basic information from the post title. This is not
always successful or accurate, please use your best judgement and compare
these values to the post title and body for confirmation.
- Posted
- 8 months ago
- Reddit URL
- View post on reddit.com
- External URL
- reddit.com/r/LocalLLaMA/...