Looking for people interested in testing iPad keyboard with VoiceOver support

By André, 6 July, 2016

Forum
App Development and Programming

Hello everyone,

I have been working on an iPad keyboard extension and I am now at a point where I am looking for some outside feedback.
I implemented VoiceOver support, but as a non-blind person I am sure to miss stuff that might be obvious for experienced accessibility users.
I hope to find interested people in this forum.

So.. what is the keyboard all about?

Well, as already mentioned, it is iPad only.
The keyboard only has two keys, which - with active VoiceOver - take over the lower half of the screen.
One left, the other on the right.
This could be done just as well on an iPhone.
But to enter characters, users have to tap with up to four fingers and use swipe gestures.
And especially tapping down with multiple fingers is something phone users do not do that often.
So I concentrated on tablets.

Back to the point:
I got a TestFlight version of the keyboard approved and am now looking for people to do a beta test.
Anybody interested in participating?

Best regards,

André

Options

Comments

By Jake on Saturday, July 23, 2016 - 02:14

Hi there, I'll test it out. From your description, are we talking about a touch-based chorded keyboard? That's what it sounds like to me, and optimized touch input is something that is an interest area of mine.

By André on Saturday, July 23, 2016 - 02:14

In reply to by Jake

Hey Jake,

can you send me an eMail address so that I can add that to the TestFlight beta?

It is funny that you mention "chorded keybords".
My first foray into iPad keyboards (unfortunately without accessibility support) had 8 keys and users had to press up to three keys to enter a letter.
But the idea was, that there was one key for each pinky, pointer, ring and middle finger.
I only later learned about chorder keyboards and kind of blamed the name for it (-;

So Zwei - the name of the new keyboard - was developed with that experience in mind.

Here is the current version of the app description:

<blockquote>Zwei is an iOS keyboard extension with only two keys.
You can still enter all letters of the alphabet as well as numbers, punctuation and stuff, though.
But instead of a dedicated button per character, what you do with Zwei is use multiple fingers and swipe gestures on the two keys to get to a specific character.

The motivation behind this is that, once learned, you do not have to look down at the keys anymore. In fact, I am developing this keyboard with accessibility in mind so that blind people can use it just as well. For all users, Zwei will hopefully improve the text input experience on a touch surface.

So how does it work?

A tap with one, two or three fingers on the left key will enter the letter 'a', 'e' or 'i' respectively. A tap on the right key will enter 'n' for one, 's' for two and 't' for three fingers.
While a key is being pressed, the current character is previewed in the selected textfield and - if activated under the accessibility settings - VoiceOver will announce this character. You can still change the character by e.g. adding or removing fingers. But once you lift your fingers, the current character is confirmed and the next one can be entered. VoiceOver will play a confirmation sound, here.
Swipe gestures have to be used to enter the remaining letters of the alphabet. Here are two examples:
1. Tap the left key with one finger and swipe to the right to enter the letter 'd'
2. Tap the right key with two fingers and swipe to the left to enter the letter 'u'
As long as at least one fingers keeps being pressed down, you can swipe around and add or remove fingers to change the current character. So feel free to test it out.
If you want to cancel the current input, just swipe over the edge of the key (top or right edge for the left, top or left for the right key). VoiceOver users will hear a cancel sound, here.

Standard iOS keyboard operations such as 'Delete', 'Space', 'Enter' and 'Next Keyboard' are also implemented, but through a different approach:
While one of the two keys is being pressed, the other one changes into a function key.
So to delete the last character, hold down on the right key and then tap the left.
To enter a space, hold down the left key and then tap the right.
To keep function keys simple, they only require single finger taps and no swipes.
They change their function based on the number of fingers pressed down on the first key.
To enter a new line, use three fingers on the left key and then press the right.
And to switch to the next keyboard, hold down three fingers on the right and tap the left key.

So what happens when the function key is tapped for a two finger press and hold on either the left or right key?

In its standard mode, Zwei is set for lower case letter input as well as some standard punctuation (period, comma, exclamation and question mark).
The other modes are "upper case letters", "numbers and more", "numbers and maths" and "diacritics".
The former two can be selected in standard mode through the before mentioned combination:
The left key, two finger press and hold plus right function key tap activates the upper case mode. When any other mode is selected, it reactivates lower case mode.
The right key, two finger press and hold plus left function key tap switches to and between the two modes for numbers.
To get into the diacritics mode, you first have to change into a numbers mode. Here the option to switch to the next keyboard is not available. Rather, the right key, three finger press and hold plus function key tap turns on lower case diacritics. And when in this mode, the same combination activates the upper case diacritics mode. I really hope you do not have to use this that often (-; </blockquote>

Hope this helps!

By Jake on Saturday, July 23, 2016 - 02:14

Hi, what's the best way to get my email address to you? I'd rather not post it in the open, as it's not my typical email address (I use a separate one for my Apple ID).

By André on Saturday, July 23, 2016 - 02:14

In reply to by Jake

Hey Jake,

I am new to this forum, but I think the best way to message someone is to use the contact form when you click someones user name.
I just did it, so you have an eMail waiting.

Regards,

André

By Jake on Saturday, July 23, 2016 - 02:14

Hi, got your email and have Zwei installed as of this morning. I'll have a chance to test it tonight when I get home from work. Thanks.

By Jake on Saturday, July 23, 2016 - 02:14

Hi there,
I had a little downtime today and did some initial testing with Zwei. I have a few thoughts:
First, this is by far one of the most interesting takes on touch input that I've yet encountered, and the only one I've seen so far that I won't need to confirm the characters I enter once I get good with it. For now I have key echo turned on of course, but I can already write at least a little with it and not make too many mistakes. This is something I, at least, would certainly buy and use.
The only bug I have to report regards deleting characters. The action itself works just fine, but I've noticed that the word "delete" telling me that I've activated the delete function sometimes cuts off the character that was deleted. This is because VoiceOver does not have what we call a speech queue (I can explain in detail if you want). A way you could work around this would be to form a string with the word delete and the letter, so you could have it say "delete S" or perhaps "s deleted". You could then send that string to VoiceOver, rather than sending both the letter and the action separately so as to not interrupt each other.
I'll be doing some serious testing tonight when I get home, but I wanted to at least get some initial impressions to you. I really like it so far!

By André on Saturday, July 23, 2016 - 02:14

In reply to by Jake

Hey Jake,

thanks for the first impressions.
I was afraid that the first steps might be confusing or unclear.
So it is good to know you found your way.

The speech queue you write about would truely be something nice.
I looked for something like this but was not able to find anything.
Currently, there is also a difference when VoiceOver calls out a character like 's' - which the system does automatically - and when I tell VoiceOver to say 'delete'.
I do the latter via a so-called announcement.
These can be timed, though, so there might be a solution.
What is strange is that when I switch VoiceOver to German, the automatic calling of characters does not happen.
I will have to test this for other languages, as well.

From my typing experience I have to say that switching to upper case mode and then back to lower case mode takes quite long.
I am currently working on improving this by adding a two finger tap for the function keys.
This gives me more options so I can add a 'upper case for one letter only' mode.
It makes function keys a bit more complicated, but I hope it is worth it.

Looking forward to new feedback.

Thanks again for the effort!

André

By André on Saturday, July 23, 2016 - 02:14

In reply to by André

Hello everyone!

As mentioned before, I was experimenting with the system automatically calling out typed letters like 's'.
It took some over night thinking, but I thought about Jake's initial impressions and about the key echoes he mentioned.

At first I was not sure what he was talking about.
But in combination with my problem, it occurred to me:
There must be an option to tell iOS to speak out characters.
It is what Jake meant with key echoes.
I went through the VoiceOver settings and discovered 'typing feedback'.
It can be set to 'none', 'words' and 'characters'.
I tried the former two and, sure enough, when I typed a letter with Zwei, it was not called out anymore.

I have to find out if I can ask programatically, which mode is activated.

I think for Zwei users, the current letter should always be called out in some way.

What do you think?

Best regards,

André

By Jake on Saturday, July 23, 2016 - 02:14

In reply to by André

Hi,
I think Zwei should obey the keyboard echo setting. If we're designing the ability to type on a touch screen without having to look, then we're designing also for the ability to not have to hear the letters. I type with key echo off on a typical keyboard, as I neither want nor need confirmation of what I've typed. Once I'm good enough with Zwei, I believe it has this same potential and that is exactly what I'm looking for: the ability to type without needing to check what I've typed. So, when I have key echo off, I don't think Zwei should say anything at all in regards to the typed characters.

By Jake on Saturday, July 23, 2016 - 02:14

Hello,
Some more detailed impressions.
First, I'm getting better with Zwei as I use it. One thing I have to ask, and that messes with my mind every time I use it, is the way T breaks the pattern. Why T? I know the st combination is quite common in German (and English too) but designing for consistency and designing for frequent letter combinations are entirely different goals, to my way of thinking anyway. I keep wanting to type a T according to the rest of Zwei's pattern and getting a U instead, and that throws off the rest of the alphabet at least according to the English ordering of letters.
Second, I agree with your post above where by you mentioned having both an all upper case mode (like now) and a shift mode. I need to type just one capital letter more than I need to type all caps, though I most certainly would appreciate both abilities.
Third, I think we need a hide keyboard function. If we have one, I've missed it. When I'm done typing, sometimes I want to hide the keyboard so that the rest of the window returns to normal. I'd suggest a swipe down on both the left and right keys simultaneously, though I'm not partial to any particular gesture.
Fourth, and this is more a question, do you intend different layouts for different languages? I'm familiar with, and sometimes communicate in, German and Swedish. That upper case accent mode is lengthy indeed. Not a complaint; I'm just curious what direction you might plan to take when it comes to languages.
One possible bug, though I've not been able to reproduce it consistently. Sometimes Zwei seems to stop typing. That is, I can type, VoiceOver will speak as if I'm typing, but the letters aren't appearing in the text field as they should. So far I only saw this in the Notes app, and only a couple of times. I've yet to find a consistent way of reproducing it. If I do, I'll give a more detailed, step-by-step report.
That's about it, for now. I'll write more as I find more. Keep up the exciting work.

Hey Jake,

the motivation behind the letter 't' and why it breaks the pattern is that it is the most common letter in English as well as in many other languages.
So I wanted to have it accessible without swipe gesture.
An earlier version did this with the letter 'r' as well.
So on the right side I had 'r' for a one finger tap, 's' for a two fingers and 't' for three.
On the left it works out in alphabetic order as I added the periode and the comma.
Now the most common letters 'a', 'e' and 'i' are accessible without swiping.
In the end I changed the 'r' back to alphabetic order as 'n' is also a frequent letter and because single finger swipes are hopefully easy enough.
So now we only have the 't' left out of order.
Would be nice to hear what other people think, as well.

As for your second, third and forth point:
I think you will enjoy the update I am working on.
As mentioned before, I will extend the function keys with a two finger tap.
This way I can make single and all caps, the minimize function and diacritics/accents always available.
I hope I can finish the this over the weekend.

Last, but not least: The bug you mentioned.
I have not seen this behaviour, but I will look for it.

Best regards,

André

By TheBlindDragon on Saturday, July 23, 2016 - 02:14

Hello there, I would love to beta test.

By André on Saturday, July 23, 2016 - 02:14

Hello again,

on Monday I released an update for Zwei and I just wanted to check if anybody had the chance to test it out, yet?

As mentioned before, the big change was adding a two finger tap to the function key.
This made it possible to add some functionality (entering a tab, minimising the keyboard) and modes (single upper case mode, always available diacritics/accent mode).
Hope these changes were welcome.

I have not encountered any major problems and I think the biggest issue is the "out-of-order" letter 't', that I previously discussed with Jake.
I would still hope for some feedback on this topic.
What do you think: Is the reachability of frequent letters more important than an alphabetic order?

Best regards,

André

By Jake on Saturday, July 23, 2016 - 02:14

In reply to by André

Hi Andre,
Could you document what the various two finger function key combinations do? I didn't see mention of it in the update description. I'll be doing some more testing. It's been a busier week than I'd anticipated. I did install the update however and have used it a bit.
I'd not say frequent or alphabetical is better, but rather that consistency is important. If you're going for frequent letters, then you have to consider each language in turn and the letter combinations you'd need. Alphabetical is quick to design and quick to master, but may not be as straight-up efficient as an optimized layout. In the end, either have their merits, but I'm not sure they should be mixed as their ultimate goal is very different.

By André on Saturday, July 23, 2016 - 02:14

Hey Jake,

I had to rewrite the information about the new functionality in the app description itself and thus did not go into details in the update form. So here it is:
<cite>
Standard iOS keyboard operations such as 'Delete', 'Space' or 'Enter' are also implemented, but through a different approach:
While one of the two keys is being pressed, the other one changes into a function key.
So to delete the last character, hold down on the right key and then tap the left, both with one finger.
To enter a space, hold down the left key and then tap the right, again both with one finger.
If you use two fingers to tap the left while holding down the right key with one finger, this will correspond with the Enter function.
And a two finger tap on the right key, while the left is being pushed down with one finger will input a tab.
Function keys will only listen for either one or two finger taps and no swipe gestures.
The remaining combinations, a two or three finger hold of the first key plus a one or two finger tap on the function key will activate different modes or states:
In its standard mode, Zwei is set for lower case letter input as well as some standard punctuation (period, comma, exclamation and question mark).
The other modes are "single upper case", "continuous upper case", "numbers and more", "numbers and maths", "single diacritics" and "single upper case diacritic". The mandatory 'next keyboard' is also available through one of the before mentioned combinations, as is the functionality to minimize the keyboard.
Feel free to try how to change between these.
</cite>

Best regards,

André

By André on Saturday, July 23, 2016 - 02:14

Me again,

today I released another update in preparation to try to contact some review sites and committing the app for review by Apple.

To make the initial message as easy as possible, I changed to a complete alphabetic order.
This means the letter 't' is now where it belongs.

I still think I might go back to a layout based on letter frequency in the future.
But as learning a new keyboard itself is enough of a challenge, things should not be more complicated for this first release.

Best regards,

André

By Jake on Saturday, July 23, 2016 - 02:14

In reply to by André

Hi, did some more testing. I really like the new function key combinations. In particular, it helps to have a tab function (something I forgot to ask for) particularly when I'm doing remote computer administration through LogMeIn or SSH. Will get Build 15 installed tonight and test some more. I haven't found any major issues at this point, and the issue of keys not typing hasn't surfaced again; that one was probably operator error :). I'm gradually getting faster, and should Zwei get approved by Apple (and it had better) I, at least, will be buying it.

By Jake on Saturday, July 23, 2016 - 02:14

A slight issue, relating to VoiceOver's speech queues again. Your mode switch functions such as single upper case mode speak the mode they are switching to. This is fine, except that the description of what is on the keyboard is silencing the name of the mode. I'd suggest merging these before speaking them.

By Jake on Saturday, July 23, 2016 - 02:14

Hello,
Zwei has stopped working properly with VoiceOver as soon as I installed the iOS 9.3.3 update. The individual keys and gestures will work, but I can no longer use the two keys together for extra functions. Only the first key I press registers so, e.g. if I want to delete the previous character I get an N instead.

By André on Saturday, July 23, 2016 - 02:14

Hey Jake,

sorry for the late reply, but regular work kind of came in the way recently..
As for your problem after updating:
I have now tried it on both an old iPad 2 and my iPad Air 2, both on 9.3.3, and was not able to reproduce the bug.
The function keys are still accessible the intended way.
So I wanted to ask if others observed this, as well?
Or did you, Jake, change other accessibility settings (e.g. "typing style" or "phonetic feedback") and these are leading to the strange behaviour?

Best regards,

André

By Jake on Saturday, July 23, 2016 - 02:14

In reply to by André

Nope, I hadn't changed any settings. It worked, then it didn't. I'll try a few troubleshooting steps now that I know this isn't happening to you.

Hey Jake,

I just wanted to check if you had any success at fixing the issue with the function keys?

For starting the review process, I had to re-build the app.
The previous versions were built with a Mac OS beta, which apparently was a problem.
Maybe that caused the error as well.

Well, by now the app has been approved and I could send you a promo code to see if that fixes the problem.
I have not yet released the app as I am trying to find some media outlets interested in writing about Zwei.
But the promo codes should work, already.

Best regards,

André