I just spent the last few days hacking and adding AI assistant behavior into a prototype app i'm making... (a hybrid chat command prompt to my react (next) app) and I realized that the "boundary" is pretty toolkit oriented (feels like it could be packaged as a tool like any other component-y tool or like a package). Before I commit to maintaining this piece of code myself, I realized I should try and ask if anyone is making anything like this already or if there are already toolkits out there I should be using. My quick searching didn't actually find anything, but the terms in this space are so new I might not be searching well.

Specifically my needs:

• I want to offer AI chatbot in my react app (It should be able to answer questions about "what it sees" in the app / in it's context view)
• I want the chatbot to be able to take actions in the app on the users behalf
• I want there to be a "command prompt" style interface (eg. command shift p in vscode, or command shift k in github, windows-key in windows, command space on mac, etc...) that I put in a request, and it takes action.
• bonus: It would be nice if I could add an annotation on specific text boxes to turn them into auto-completing copilot style text-boxes

API wise:

• I want to define a stateful object (like a jotai atom) that I put in the current "view/app context" (`.put({...}`) (some <1000 character description of what is on the page currently, what misc/specific component state is etc.) It'd be great if I could just shove too much into it, and it does some LRU-cache random sampling to keep it small or something...
• I want to define a set of "described" callbacks/actions (functions) that the AI can chose to run (essentially `callbacks={'click submit on form': () => onClick(), 'add a square at x,y to visual': (x, y) => setObjects([...objects, Square(x, y)])}`)

(Essentially, treating the AI as an RL agent, where it has "State" and "Actions"... and it uses language to make a choice for action based on input from user, or in the chat-bot, can describe the current state to the user)

UI Component wise:

• I want a simple chat box (like those added by all of the other chat-box integrations)
• I want a command prompt that i can ask for it to do something, and it "does the action" out of the possible callbacks that are currently registered

I am "feeling" a vibe like this is a toolkit/component that I should adopt from a standard rather than inventing/maintaining my own, but I'm not aware of any "add an AI assistant to a react app" yet (So far all I've found is a voice assistant by AlanAI) -- can anyone point me towards any that exist or answer if you're actively making something like this?