Our main release post can be found here -
https://www.libretro.com/index.php/retroarch-1-8-2-released/
However, in specific, the accessibility feature for blind people is why we are posting this here.
This link can be found here -
We'll include the article in full below
------------
RetroArch singlehandedly takes big strides to cater to an underrepresented group of people, opening up a whole new world of entertainment.
See here the perspective of a blind person talking about the new accessibility features available in the latest nightly versions of RetroArch. This and more will be available out of the box starting as of version 1.8.2.
Also read our version 1.8.2 release announcement post here.
Written by Devin Prater, Certified Assistive technology instructor
Edits by Barry Rowe, AI Service and Accessibility contributor
Background
For decades, video games have offered entertainment for many people. Childhoods were changed by iconic franchises like Super Mario, Zelda, Final Fantasy, and Castlevania. People can reasonably count on others to understand the meaning behind video game references.
For people who are blind, however, these games could only be enjoyed through their great music, or by reading novelizations or fanfiction. Audio games have been created, to fill the void of video games which could not be played, an some blind people braved fighting games by memorizing menus and special attacks, but audio games were few in number, and didn’t usually have much content.
Emulation
Emulation has helped many people who are blind relive their childhood playing fighting games. With the rise of machine learning, however, blind gamers now have another tool in their arsenal: optical character recognition, the extracting of text from images. With this being a part of many screen readers, blind people could use that to read menus, character select screens, and unspoken dialog.
RetroArch
RetroArch is the first “emulator” to now offer Accessibility to blind people by speaking the interface. Along with the text-to-speech AI service, RetroArch has not only become the first emulator to implement accessibility for blind people in menus, but also in reading game text as well.
This doesn’t, however, mean that all games are accessible. A blind person still cannot get Super mario into the castle in Super Mario 64, nor defeat Lavos in Chrono Trigger, although perhaps one could probably play Radical Dreamers now. Much more work will be needed to make video games completely accessible to blind people, even portraying health bars in fighting games through sound cues. Even so, the accessibility of RetroArch means that blind users of Windows, MacOS, and Linux can enjoy the state of the art in video game accessibility through emulation.
How to enable accessibility
Once you’ve downloaded and installed RetroArch, there are two ways to enabled accessibility. The first way is by turning it on via the menu. Once RetroArch is started press: right, then up seven times, then enter (on some systems this could be the x key), and then right. You should hear “Accessibility Accessibility Enable ON” at this point. If this doesn’t, restart RetroArch and try again. This method navigates the menu, which may change in later versions, so you should read the RetroArch Accessibility Docs for any updated key presses.
The second method is to enable it via the command line. This is done by running the RetroArch executable (for example: retroarch.exe) in the command line or terminal. On windows for instance, once you’ve opened the command line, navigate to the RetroArch folder, and run “retroarch.exe –accessibility” and you should hear “RetroArch Accessibility On. Main Menu Load Core.” From there you can navigate right to the settings submenu, and then down to the Accessibility option, and then turn set Accessibility Enable on. Now you’ll be able to start RetroArch with accessibility from outside the command line as usual.
If these options don’t work for you, it could be that your OS does not have the required speech libraries or voices that RetroArch needs. For windows, RetroArch uses the Windows Narrator, which you can read how to download additional voices for here. On MacOS, it uses the “say” command, which you can read how to download voices for here. And on Linux it uses Espeak. For Ubuntu, you can install espeak by running “sudo apt-get install espeak” and then “sudo apt-get install espeak-data” for the additional voices.
Using the AI Service with Accessibility
The AI Service can also use the Accessibility narrator for Text-to-Speech. This can be done by going to the AI Service settings section, and changing the AI Service Output to “Narrator Mode.” This handles the Text-To-Speech, but the AI Service still needs to process the game screen to get that text. You can follow the setup instructions for the AI Service here.
Conclusion
While people without disabilities have been able to play thousands of video games, both current and past, blind people have not had the ability to enjoy more than a handful of video games. Through
emulation, this is beginning to change. Games which were once only playable if one could memorize menus and selection screens are becoming accessible using OCR, and more will be possible through the hard work of developers who may build upon this foundation for accessible video game emulation.
RetroArch is the first emulation center to provide accessibility for the user interface, and an AI service to perform OCR on video games, allowing blind users of all three major desktop operating systems to enjoy playing fighting games now with knowledge of any text that appears onscreen. It is hoped that this is only the beginning of a great advancement in accessibility, with RetroArch paving the way to even greater video game accessibility for people who are blind.
------------
RetroArch is available as a Mac and iOS/tvOS application. You have to sideload it on an iOS/tvOS device though, and the Mac version is not available on the Apple App Store due to store policies preventing its distribution in adulterated form.
Comments
Thanks!
Thanks for posting this here. Just to let everyone know, the accessibility mode only works on Mac and other desktop operating systems, for now, and does require time to learn and set up.
This sounds cool
I'm glad that more developers are hearing our call for accessible content. The more developers that answer our request, the more others may feel obliged to do the same.
I think the end goal should
I think the end goal should be in the future to not need extra tools and jut have everytign at least for us totals read, and for us with some site (all of us have vission) to have high contrast etc. I might try this consept out on forza motor sports which I have on my computer, unless someone already beat me to it.
Take care all.
Well, since I can't for what
Well, since I can't for what ever reason leave a comment on the blog, maybe it's because I'm not logged in, can I even use this on forza which I have on my pc? If so I can try this on my ringing in 2020 marathon stream which will probably happen third week of January as I will be in pain the day I was going to do it I'm thinking wi Ould be able to, but the devs can probably answer this.
Motor Sports
This is for emulating older game consoles, not for playing new, PC titles.
Darn, yeah I don't play those
Darn, yeah I don't play those older titles, in fact I never grew up with those systems, was not aloud to, had to do homework and get a high school education and all that first. So I don't really have an apprecawtion to those systems, I used to in fact kind of when I was older bully kids into not playing those and say they would get adicted and to stop. Well, maybe one day this could be made for the newer games. It never hurts to dream.
older games
I grew up with older games. I didn't have OCR with them when I was growing up, of course, but now that I do, that makes them all the more fun, being able to not only use sound, but text as well. It's not a Curaga, but it does a lot, for me at least, and others on other forums. Also, the in-game OCR actually uses a translation service, so with it, we can play games from Japan and other countries which weren't translated to English.
OOO like teckin, or is that
OOO like teckin, or is that one of the newer ones. Soudns cool.
for my understanding
hey! please tell me if I am wrong
when I enable the software/ app/
it can read the menu and thing when I playing any video games right?
answers to questions
#8: Tecken 1 to 6, yes, but not Tekcken 7.
#9: Yes.
ok!
well,
so,
how can I find which games title it can support right now?
I like playing street fighters and the king of fighters as well.
and also some shooting games
cores
Retroarch has support for many systems, from the NES, Arcade systems, the Super Nintendo, Play Station, up to the GameCube, PSP, and Wii.
This is innovative
This really is something special. It's not perfect, but it's a huge step in the right direction, especially for people like me who can play mainstream games, but who read in-game text very slowly. I've experimented with a few super Nintendo roms and it worked very well. There is still the age-old seemingly unsolveable problem of needing the game's text to be a simple font, and without any background art, but maybe I was just using the wrong AI service. That's really my only complaint so far is implimenting the translation service is a little convoluted. It would be helpful to have an easy tutorial for this. I'm a bit confused at the "speech" and "narrator" choices too. Otherwise it's a great thing they're doing. It would be really, really cool to one day have this sort of technology integrated directly into windows itself. Imagine if Windows used something like Seeing AI's features to read screens, and allow you to browse menus with the arrow keys like JAWS and NVDA do. NVDA actually works quite well for recognizing text in games, but navigating them is another matter. That, and there are times where it doesn't read the text it captures properly even when the text is clean. Still, this is a great start, and a glimpse into what may one day be possible.
\I know how it works... but,
after watching a few video on youtube. I I think I know how it's work.
but, my question is\:
how can I get the games?
after installed RetroArch ?
and I think it is a big step for gaming accessibility
even it still have improvement!
games
You'll need to provide the games yourself. When getting games, do not download executable programs, only the game files themselves, and read Retroarch's documentation on the system you want to play the game on, in case there are extra steps needed.
games and controllers
For games, Dissidia Final Fantasy has a few parts where text is important, like stats of opponents before battles, character select screens, story text that isn't spoken, things like that.
Yes, Retroarch works with controllers.
Retroarch user's guide available
Okay y'all, I've completed the Retroarch User's guide:
https://docs.libretro.com/guides/retroarch-accessibility-guide/
Thank you
This is a very comprehensive and easy-to-follow manual. Thank you for taking the time to put it together. I'm still a little confused about the Speech Versus Narrator aspect though. I understand narrator simply uses WIndows 10 narrator, while speech allows one to use ... what, another screen reader's speech synth?
Also, does Retroarch allow one to navigate menus with the AI service engaged? I saw the demo and it looked like the person playing had no trouble.
speech and menus
If you set the AI Service to speech, it'll speak using some online voice, I think using Google TTS but not sure. With Narrator mode, it uses your operating system's default voice.
About menus, it will read menus, but you have to remember which menu item is which and how far down it is according to the list read by the AI Service.
This is Great News
Great news indeed. I'll have to check this out when I actually have some more quality time to myself. I've not been gaming much over the past several years mainly due to time constraints, but this sounds very exciting and I can't wait to check it out. Thanks for posting, and happy New Year to everybody!
I hope....
well,
I hope steam or other gaming plactform can learn from it!
When will we get a iOS version
Will we get one soon? I would love to play mainstream games on my iPad too!!! Sighted people have al the fun.
Emulation on iOS
See the PPSSPP on iOS link for info on getting at least one emulator going on your iPad.
https://github.com/devinprater/accessible-retro-games/blob/master/emulation/readme.md
Has anyone had any success running Mames on this?
I had no idea running Mame files, in order to play older arcade games, were such a pain the butt to run, as opposed to Nintendo/Sega roms/emulators, but I'm finding this to be a really tedious process. I've found several sites that offer loads of them, but most of them, I've unzipped them, refuse to run, Retroarch continuously says "no items" or if I can get any of the .bin files, to attempt to open, it crashes. Am I doing something wrong, or is something about these files just unstable?