Any alternative for koboldcpp?

This post has been de-listed

It is no longer included in search results and normal feeds (front page, hot posts, subreddit posts, etc). It remains visible only via the author's post history.

Post Body

Unfortunatly my PC isn't the best, so I was pretty happy when I found Koboldcpp which enabled me to run local chat models with reasonable speed. Unfortunately the "SmartContext" (a function that re-uses some of the context and thus avoids having to process the full context every time which takes to long on my system) has been broken for me for a few month now and the developer doesn't seem to be able to reproduce the issue.

So do you know of any alternative projects that might enable me to run KoboldAI on my system (64GB system RAM, GeForce 1660 SUPER)?

Author

Account Strength

50%

Account Age

2 years

Verified Email

Yes

Verified Flair

Total Karma

154

Link Karma

Comment Karma

Profile updated: 1 day ago

Posts updated: 1 month ago

AMPosts

Subreddit

r/KoboldAI

Post Details

We try to extract some basic information from the post title. This is not always successful or accurate, please use your best judgement and compare these values to the post title and body for confirmation.

Posted: 1 year ago
Reddit URL: View post on reddit.com
External URL: reddit.com/r/KoboldAI/co...