Hello!

I have posted here in the past about this, but this was a while ago in terms of how quickly the AI field has been progressing, so I’m sort of curious–– have there been any tools released that a blind user could use to play video games? Or get live audio descriptions of movies or shows?

What I mean by this is some tool that allows a blind user to use their voice to ask questions about what’s appearing on a computer screen at any given time. For example, if I’m playing a video game, and I need to locate a chest, pressing some kind of hockey or using some kind of weak word to ask the AI to look out for a chest, and have it scan my screen and real time for what it thinks a chest would be. And after locating that chest, playing some kind of tone at the coordinates of that chest to guide the player to it. Alternatively, some thing that’s not even real time would still be incredibly useful. So as opposed to scanning the screen, simply having some thing you can ask questions about what’s on your screen at any given time would be immeasurably helpful.

Or even taking this a step further, and allowing the AI to jump the mouse cursor to those coordinates it located and click to add an even further layer of automation or assistance. Is there a tool that does something like this? Or even just a tool that lets me ask questions about my screen in general? I miss playing video games, and I haven’t really been able to go ever since I lost my vision back in 2021