Text adventure interpreter makes Infocom's Journey and Arthur accessible

By Petter, 16 April, 2025

Forum
macOS and Mac Apps

See my earlier post here:
https://www.applevis.com/forum/macos-mac-apps/voiceover-users-wanted-test-hack-infocoms-journey

This is crossposted from intfiction.org:
https://intfiction.org/t/voiceover-testers-wanted-for-new-spatterlight-with-arthur-and-shogun-support

The previous version of my text adventure interpreter Spatterlight added VoiceOver support for Infocom's Journey. The new one has experimental support for Infocom’s Shogun and Arthur as well. I'll be happy for any bug reports or other feedback.

The new version can be downloaded here:
https://github.com/angstsmurf/spatterlight/releases/download/v1.4.4b/Spatterlight.app.zip

Here is a work-in-progress guide to using Spatterlight with VoiceOver:
https://github.com/angstsmurf/spatterlight/blob/master/ACCESSIBILITY.md

Download Arthur here:
https://curiousdannii.github.io/infocom-frotz/arthur-r74-s890714-graphics.zblorb

And Shogun here:
https://curiousdannii.github.io/infocom-frotz/shogun-r322-s890706-graphics.zblorb

There is also an early development version of Shogun available that has the debug command $SCENE which lets you jump directly to any scene of the game:
https://eblong.com/infocom/gamefiles/shogun-r278-s890211.z6

Any version of the games should be working. I’ve tried to make everything accessible with VoiceOver, with custom support for menus and for simplifying the Shogun graphical maze.

https://youtu.be/DLTK1fMp5-o?si=M_erjnXIkibRolHJ

The above is a video recording with sound of me playing a couple of moves in both games with VoiceOver. I’m also using VoiceOver Recognition to (incorrectly) describe some of the images (VO + Shift + L). As usual, VoiceOver will sometimes skip text or just stop speaking, especially when there are windows closing and opening. You might have to press the ”Repeat last move” shortcut (Option + Command + left arrow), sometimes repeatedly, to make it say all that it is supposed to.

In the video, I first change two settings, ”Format > Redirect text to main window” (more on this below) and ”VoiceOver > Speak Images > All”. The latter is needed to make VoiceOver take note of the images in these games. I then start up Arthur, try to make VoiceOver Recognition describe the images with varying success (including the room image), and then type HINT to make sure that the response is printed in the main buffer window. I then enter the hint menu and navigate to a hint.

After that, I start Shogun, using the intro menu, and play long enough for it to show two margin images, which I then try to get VoiceOver recognition to describe. (I navigate to the first one with the custom image rotor VO + U.) I also enter the “Function Keys” menu with the DEFINE command, and create a macro for TAKE ALL, activated by pressing F2. Yes, it is kind of useless, but it works.

Spatterlight has had basic support for Infocom InvisiClues style menus for some time now, but the help menus in the version 6 games are different enough to require special handling. Unlike the previous menu detection, it also handles the listing of actual hints, the ”bottom level” of the menu hierarchy. The seen hints for a puzzle can be stepped through using the command history rotor.

Arthur was always more playable with a screen reader than the other Infocom version 6 games, but it prints some interpreter responses to a separate ”error window.” To help with this I’ve added a new option to Settings > Format, ”Redirect text to main window,” which will make these print to the main text buffer window instead.

Another potential problem is that Arthur uses a timeout in a couple of places, so if VoiceOver is not set to speak at a high enough speed, as in my video, the game will wipe the text and move to a new screen before it has spoken all of it. If anyone is bothered by this, let me know. I might try doing something about it.

In Shogun, one major stumbling block for screen readers is the graphical maze. I gave up on trying to bypass it entirely, and instead just added an option to simplify it. If VoiceOver is active, you will be asked about this before entering the maze. I hope that some people will appreciate that a little of the challenge of the maze remains for VoiceOver users. If anyone really wants an option to skip it entirely, I suppose I might be persuaded to give it another try.

I figure the original graphical maze is a puzzle, if not necessarily a very good one. Likewise, navigating the simplified maze using VoiceOver is another puzzle of similar quality. Trying to solve it on your own might make it more interesting, but if not, here is a spoiler:

SPOILER

STOP READING HERE

The city ”maze” is now a single path. Every ”room” in the maze represents a 90 degree bend in the path, either to the left or to the right, relative to the way you came. In other words, there are always two exits: one back to where you came from, the other one either a left or a right turn. If you come from the west, the way back is to the east, and the way ahead is either north or south. There is no telling which until you try one of them, so there is a 50 percent chance you will pick the wrong one first, and the game will say that ”There’s a house there.” If the way ahead is to the south, the next turn will be either east or west, and so on. Of course, there is also the risk of getting confused, turned around, and ending up back where you started. Then you will have to get back in and do it all over again.

Options

Comments

By Sebby on Thursday, April 17, 2025 - 01:44

I might just try this!

Is there any way to disable the accommodations completely and just rely on reviewing the buffers in the usual way, or are they complementary?

Spatterlight is simply the gold standard of IF play for the blind now, in my opinion. Thanks so much for the work you've done and still do. It's much appreciated, not to speak of unexpected!

By Petter on Thursday, April 17, 2025 - 01:44

I'm not sure if I understand which accommodations you want to disable, but all text printing in Spatterlight is done in standard system text views, so you should be able to navigate the text using all the standard VoiceOver shortcuts, with all your custom settings honoured, just like in, say, TextEdit. Let me know if it doesn't work for you.

By Sebby on Thursday, April 17, 2025 - 02:44

You mentioned dynamically-added menus when VoiceOver was detected. I supposed that the experience would therefore be parallel to, rather than complementary to, the existing model of each game window being its own interactible element, and all the output would be self-voiced differently. From what you're saying though, it should still be possible to review the text as printed by the game. Just for context, I've used a DOS screen reader with frotz to play and win Journey, so I'm not unaware of the mechanics of the z6 format and the way it presented the two-pane view and menus.

But don't worry, I'm a long-time user of Spatterlight (thanks again!) with VoiceOver, so I'll figure it out and let you know if I run into any trouble.

By Petter on Thursday, April 17, 2025 - 08:44

Right. There is a setting: VoiceOver > Speak menu lines > Don't detect menus, which is supposed to turn off all special VoiceOver menu handling altogether. This can be necessary sometimes, because it has been known to happen that the automatic menu detection mistakes any old list of things for a hint menu. I fix these cases as I encounter them, but there are a lot of games out there.

Now that you mention it, I realise I haven't tested to turn off the menus in Arthur or Shogun. Thanks for reminding me, and please let me know if you run into problems!

By Petter on Thursday, April 17, 2025 - 14:44

I changed how the menus behave when the "Don't detect menus" setting is active. Spatterlight no longer keeps saying "No last move to speak" every time you change selection. The link in the original post is changed to point to the new version.

If you want to compare it to the previous behaviour for some reason, the old version is still here:
https://github.com/angstsmurf/spatterlight/releases/download/1.4b/Spatterlight.app.zip

Thanks for bringing this to my attention!

By Sebby on Thursday, April 17, 2025 - 14:44

Probably the weekend, I'll take this out for a spin.
And yes, the important thing is to have options, because it doesn't always work correctly, but when it does, it's awesome!

Thanks again and I'll let you know.

By Sebby on Monday, April 21, 2025 - 17:44

Yes, it works. It's workable. It's clearly superior to having no assistance at all, because as far as I can tell there's no textual representation of the cursor and, therefore, no obvious way to navigate from the keyboard using the arrow keys and space bar, as the game directs. The entire status window is always announced, which makes knowing what the options are possible and convenient, but navigation impossible.

Use of the menu bar is understandable and intuitive, but somewhat clunky, requiring player to keep using the menus rather than staying in the game window. It may however be the best option. Perhaps you could try working on alternative approaches that try to read the highlighted text in the status window. There again, it would make reading the transcript harder, and not everyone would grok the screen layout. Maybe this is the best way, really. I like your use of popup windows for completing commands, but VO sometimes lands hard, and left/right navigation gets stuck, VO announce that it's in a alert dialog, but unhelpfully, nothing else until interaction commands are used to move to child (VO-Shift-Down-arrow). The dimming of invalid menu choices is a nice touch, because it means information isn't lost.

In the Preferences window, the selected tab isn't tracked right by VoiceOver; the selected button is wrongly announced. And the popup hint for redirecting text to the main window is bogus.

In general, I think we need a place where all your notes on VoiceOver use can go. Right now everything I've learned comes from somewhere in the release notes on GitHub. This is of course great to have, but it does mean there's no reference to direct people to and that I might have missed something.

I'll grab the new build and keep playing ...

Edit: Build grabbed, autorestore worked (thank goodness), clarified alert dialog workaround.

By Petter on Monday, April 21, 2025 - 18:44

First of all, thanks for reporting this.

Debugging VoiceOver problems can be frustrating, partly because its behaviour tends to be inconsistent and upredictable, and it is often hard to reproduce the problems that happen, but also because some bugs are caused by private, opaque Apple APIs that I really can't do anything about. Add to that the fact that I'm not an experienced VoiceOver user myself.

The things you mention with the Journey dialogs ("VO sometimes lands hard, and left/right navigation gets stuck") and the Settings panel ("the selected tab isn't tracked right by VoiceOver; the selected button is wrongly announced") are things that don't seem to happen on my end, so without steps to reproduce them, I don't know where to even begin.

The settings panel is implemented with standard controls and boilerplate code, so I tend to disregard any problems as just "Apple bugs," but perhaps I don't understand how things are supposed to work. Do you have an example of an app with a settings panel that works as intended, that I could compare it with? Even better if there was an example with available source code, but that is asking a lot, I know.

I've corrected the tooltip for "Redirect text to main window", at least. Sorry about that.

If you haven't found it already, here is my informal work-in-progress manual for using Spatterlight with VoiceOver:
https://github.com/angstsmurf/spatterlight/blob/master/ACCESSIBILITY.md

I really should have provided a link to this in my original post. I've added it at the top now. If you have any ideas on how to improve it, please let me know!

By Sebby on Tuesday, April 22, 2025 - 06:44

I agree, it feels like a standard panel and I'm not sure why you aren't automatically getting this. For contrast, try the Finder. When you select a tab in the toolbar, VoiceOver then correctly tells you that the button in the toolbar is "selected", which means that this set of options is now being displayed. Specifically, use VO-left and VO-right to select the button in the tab bar, press VO-space to select it, and confirm that this button, alone, is given as "selected". If you can't fix it, it's really not the end of the world, but it would be nice if it could be fixed, as it makes navigating all the prefs that little bit easier.

For the difficulty of the modal dialogs in Journey, simply select an option (let's say, Individual Commands, Praxix, Examine) and, when the dialog opens, simply use VO-left/right to navigate. What I get is "Alert, dialog" until I interact, and only then do I see the popup control for target objects (you have to be at that point in the game, obviously, where you have such choices).

Then last night I ended the story, but got confused by the "Musings". I thought you were supposed to get musings from each player character, on each topic, but as soon as I chose a topic, I saw the dialog reappear; only by choosing all of them, then Cancel and reviewing the transcript, did I gather that your detection logic had perhaps fired a little too aggressively in this case and it would have made more sense to let the player listen to the game output first. Give it a try.

I apologise for the somewhat terse nature of these comments—feel free to follow up if you need more info.

By Petter on Tuesday, April 22, 2025 - 13:44

I was able to reproduce the problems you mentioned now, and I am working on a fixed version. Thanks again!

By Sebby on Tuesday, April 22, 2025 - 14:44

I look forward to it!

By Chris on Tuesday, April 22, 2025 - 16:44

I just want to come in here and say once again how amazing this program is for playing Interactive Fiction. Thank you for everything you do! I'm making a game with Inform 7, and this program makes it super easy to play and test.

By Sebby on Tuesday, April 22, 2025 - 17:44

Yes. Truly, really great work.

It's paradoxical, too, because it's basically a highly specialised accessibility experience for Interactive Fiction running on a platform, and using the support of the screen reader, that offers—or, at least, was best known in our community in the past for offering—a highly generalised and mainstream accessibility experience. And it has all the same contradictions and quandaries that go along with it, of the difficulty of balancing the general with the specific in order to get maximum accessibility and user experience.

But the end result is unmatched. Interactive Fiction that self-voices at a comfortable speed through the screen reader, but with functional sound, menus and open/save dialogs, that nonetheless allows screen review. It's just awesome. I never thought I'd be playing IF on macOS much if at all, and that if I did play again it would be in Windows using TTS or a plain console interpreter. There really hasn't been anything close to it since the DOS days. Or maybe, at a stretch, the console versions of Windows or Linux interpreters running in their stupid "Dumb" modes, with no use of anything but plain stdio, no terminal escapes (no scrolling in ncurses, no cursor movement).

Edit: @Petter Thanks! Grabbing it now.

By Sebby on Wednesday, April 23, 2025 - 09:13

Unfortunately it seems there's a regression, in that if I use the Individual Commands menu, I sometimes don't get automatic speaking of the last move, even though pressing Cmd-Shift-Left reads it for me (thanks for the accessibility doc, BTW, which is excellent and very handy!).

Then, I regret also to inform you that the toolbar issue still isn't fixed, if you worked on that. The button says selected when activated with VO-Space, but it's not identified as selected when navigating the toolbar.

I'll keep playing, and tweaking … and thanks again for all your efforts, as always.

By Petter on Wednesday, April 23, 2025 - 12:06

I changed it so that if a dialog alert follows directly upon another dialog alert, such as in when selecting Musings, any text printed to the main scroll view between the two dialogs, such as the actual text of the musing, is added to the new dialog. Otherwise, the player would not be able to read this new text until the dialog is closed. This was always the intended behaviour, but it has been broken for a long time.

In what little testing I did, this seemed to be working. Do you have an example of when it breaks?

I compared the Spatterlight settings to the Finder settings menu, and the settings of some other Apple apps, and the only difference I noted was that it would not say the words "<Tab> selected" when pushing a toolbar button, that is, "Theme selected," "Glk styles selected," and so on, so I added that. When you say that "it's not identified as selected when navigating the toolbar," could you elaborate on what the expected behaviour is?

Also, is the "Alert, dialog" bug fixed for you? I added a small pause before showing dialogs, and the problem seems to be gone, but it is hard to be sure, as it happened so randomly.

By Sebby on Wednesday, April 23, 2025 - 21:24

For now, concerning prefs, yes, you've got it announcing "Selected" when the button is pressed. But it must be announced as selected when navigating left/right across the toolbar, as well. So for instance, in Finder settings, press General in the toolbar, displaying the general settings, then use VO-left/right to confirm that General is selected; then, press Tags in the toolbar, and again use VO-left/right to confirm that Tags is now selected. It's not just the announcement that the button is selected when pressed, though that's useful. It's the state when the toolbar is subsequently reviewed, and it's the selected item when the toolbar is interacted with.

I have yet to reach the musings again—I'll get back to that once I'm out of hospital—but what I refer to is just general gameplay, the Individual Commands menu no longer seems to provoke automatic reading of the transcript in all circumstances. That means Journey is now harder to play than it was with the previous build.

I will likewise test the alert dialog behaviour, and let you definitely know whether you've fixed it. I don't recall running into this recently, so I think you have fixed it, but I need to test it thoroughly.