So. I'm in the middle of creating a dataset of questions and answers have been abstracted from the Chief Judge of the Northern district of Illinois. I expect to be finished with it later this week. I was just going to fine-tune a get 3.5 turbo chatbot on it. Part of the reason why is because GPT is already trained on an extensive legal database--at least for Circuit Court of Appeals and Supreme Court cases. (I used to be able to check what cases GPT was trained on by asking the holding of various cases, but after that one lawyer relied on GPT for research, they took out the ability to explain the holding of anything but the famous cases like Brown v. Board of Ed.)

Also the dataset may be somewhat large. I'm expecting the dataset to be reducing 10 years worth of opinions to something like 2,500 question and answers (Narrowed down from like 10k potential questions). It'll be expensive, but I think the project is really interesting so I'm okay with paying some money to put it out there. There are some limitations due to the possibility of something being overruled in the last 10 years without me knowing off-hand (am lawyer), but anywho. Anyway, are there any good alternatives for fine-tuning to GPT 3.5 Turbo? I'd like to put it out there for anyone to play with and ideally don't want to be billed every time someone enters something, but I may just generate a bunch of answers based on current cases before this judge and publish the results. The biggest thing for me is accuracy. Doesn't have to be local (I don't have the hardware--I'd have to buy cloud computing anyway which doesn't exactly make it cheaper). I've played around with some other models, but don't know how easy / expensive to fine-tune and don't know what other models were trained on legal data.

Fairly new to this (though, most of us are). I'd appreciate any input that I can get.