App concept: NVDA, remote, iOS

By DrewWeber, 19 March, 2021

Forum
iOS and iPadOS

I believe we all have found a lot of uses for NVDA remote under Microsoft Windows, to remote a computer from a far. But what if this was taken one step further? to remote a windows computer from an iOS device!
My concept would be an app that would use a terminal to send in real time the keystrokes from a Bluetooth keyboard, or the jestures from the iOS devices touch screen, to the NVDA remote server. Voiceover on the iOS device would read back NVDA's spoken text from the remote computer. An extra Audio stream option may be available to hear the audio that may playback from the remote computers audio, such as a virtual audio capture driver.
I understand this would be a crowdsourcing financial campaign, but one I would put money towards and I feel well worth!

Options

Comments

By Herbie Allen on Thursday, March 25, 2021 - 08:03

I would so use that.

By Nut on Thursday, March 25, 2021 - 08:03

This is indeed a very good concept.

By a king in the north on Thursday, March 25, 2021 - 08:03

A major flaw is the latency involved. You would have to deliver certain elements on windows to voiceover and real time. Along with delivering gestures and keystrokes to the other computer. That wouldn’t be a good user experience, even if you got around latency.

Thank you for the point, I did think about latency, but the way I see it, and of course I may be missing some thing, it’s just text going back-and-forth, generally speaking, and we’re not talking about the audio stream right now. of course there may be latency issues, I have experienced them as well when using the remote NVDA under windows, but generally speaking again, you’re sending text as far as keystrokes from the iOS device and the remote client to the remote server, And, you’re receiving text back from the remote server that voiceover will read. With today’s Internet connections, I really don’t see how there would be any further latency then there may be from windows to windows. yes, there may be an issue when voiceover is running that could interfere with keystrokes and screen gestures, of course. So, if we take voiceover out of this equation, and the text that comes back from the remote server to the iPhone is red bye iPhone TTS speech without voice over involved this in my opinion could still very much be doable.

By Khushi on Thursday, March 25, 2021 - 08:03

cool concept!
:)

By Chris on Thursday, March 25, 2021 - 08:03

This sounds very interesting. It could work like Narwhal on Android.

By Saltemohn on Thursday, March 25, 2021 - 08:03

Hello everyone,
Through a friend I started developing such an app a week ago. The app basically works, but so far only one way. That is, I can remotely control a server via an external keyboard that I connect to the iPhone or iPad. I'm still working on the other direction, that I can send out the answers from the server via TTS or something similar.

You are welcome to test the app already, but please keep in mind that it is still completely in the starting blocks and bugs may occur. This includes, for example, that the left Alt key is not yet sent. The link for testing: https://testflight.apple.com/join/oeq7LtnI

Of course I would be very happy about financial support, because it is quite a bit of work for me! You can do this e.g. via Ko-Fi https://ko-fi.com/mjssoft or via PayPal https://paypal.me/mjssoft

I hope to release the app soon! Best regards,
Saltemohn

By Kushal Solanki on Thursday, March 25, 2021 - 08:03

When you mean by text going backwards and forwards do you mean like more of a commandline interface?

By Austin Grace on Thursday, March 25, 2021 - 08:03

Hello. I have wished for something like this for years on end. I'll even pay for it if you need to charge. I miss narwal on android.

Sorry for the late reply. With that I meant the TTS function. If you make any actions, you get a speech feedback. This function didn't work as I posted it. Now its implemented, but still WIP.

By Pax on Thursday, March 25, 2021 - 08:03

Hi.
I tried, and it crasched while I clicked connect.

By Samuel on Sunday, July 25, 2021 - 08:03

Hello,

It's amazing! Only one thing I don't understand, if I test with a brail material a Esys12, the message of my computer apaired only 1 second as a notification of my iPhone.
Other thing, if I move with my computer, I have a vocal return which read me where I am on my iPhone but if I move with my brail material it only moves in the iPhone and no on the computer.

Thanks for that app!!

By Justin Harris on Sunday, July 25, 2021 - 08:03

Very nice. Will test this once it is further along. Thanks a million for your hard work. I've wanted something like this for a while. It would be awesome if this app also works on m1 macs, as it is not possible to bootcamp anymore, so that takes away the method I am currently using on my intel MacBook pro, but having said that, I really want the m1 MacBook air, and if I can use something on it that will interface with NVDA Remote, that would be amazing!

By Igna Triay on Sunday, July 25, 2021 - 08:03

I'll test this once its ferther along. Question though; will this eventually make it onto the appstore? So that users can use it normally?

By peter on Sunday, July 25, 2021 - 08:03

What are the privacy and security implications of using this app? When I type in my passwords or other info, where does this information go?

--Pete

By Ekaj on Sunday, July 25, 2021 - 08:03

I used NVDA and found it to be quite a good screen reader. I would therefore be very interested in trying this out, if only I could find the time. Even though Windows as a whole is rather sketchy I do miss those days, lol. Plus, I could get more voices on my current i-device or a future one and that would be sweet.

By Chris on Sunday, July 25, 2021 - 08:03

I'm glad this is finally in the App Store. There are a few minor problems remaining that need to be ironed out. The speaking rate when using the TTS engine is really fast and can't be adjusted with the speech rate slider in the app. This slider also seems to reset every time you open the app which is not the intended behavior I'm sure. There should be a way to select the voice to be used from the app as well. The only way to do this currently is to go to Accessibility settings and change the system voice options under Spoken Content. This however doesn't address the really fast speech rate either, even if I change the speech rate from here as well.

It also appears some items are not being spoken. For example, when opening dialogs, the app doesn't seem to be receiving and automatically reading the dialog information. Having said all that, this is fantastic and it can only get better from here.

By René Jaun on Sunday, July 25, 2021 - 08:03

Hello!
This is an awesome app. Thanks so much for developing this.
I only just had the time for a first test drive.
It works well in general.
My problem: Whenever I type something (in Notepad) for example, the letters are typed, but when I try to read what I wrote by using the arrow keys, I don‘t get any feedback. The cursor is moved however, and when I perform the same action in NVDA directly, NVDA reads what has been written.
Is there any chance for fixing this?
Also, so far I have not been able to execute commands requiring the NVDA-Key. Neither the Insert-key nor the Capslock key on my external keyboard seem to do the trick, but this may just be my keyboard being configured wrongly.

By Samuel on Sunday, July 25, 2021 - 08:03

Wonderfull!
Only one question more, I just don't success to connect using my IP adress, it is only working on nvdaremote.com.
A message told me to look for the settings of the app but I can't see them in my phone.

Thanks and congrats!
Samuel

By Samuel on Sunday, July 25, 2021 - 08:03

Hello,
An other question is about the utilisation of my arrows key because when I use them it does things on the phone and not on the computer.
Appart for that, there is also a problem for me, I use a azerty keyboard and the only way with the app is to use a qwerty keyboard.
Thanks,
Samuel

By neosonic2 on Sunday, July 25, 2021 - 08:03

For a much better supported, much more widely available, and, in certain cases, much simpler experience, just use an existing remote desktop application to establish a remote connection from an iOS device to another computer running macOS or Windows. The market for these kinds of apps is already well-developed and these apps allow one to do much more than what the original concept mentioned in this thread describes; namely they allow one to take full and complete control over a remote machine from any other device, even one that is not running the same operating system or networking stack as the target machine. Many such apps exist for Windows, macOS, and even iOS and Android.

Hello,
Interesting comment. As for me, I have not yet found any IOS app that allowed me to remote control my Windows computer, at least not one that was actually accessible with VoiceOver.
Could you name one, or a few, of them? I would love to try these.

By Justin Harris on Sunday, July 25, 2021 - 08:03

I tried some of these such as team viewer, but at least with the keyboard I had, they didn't work. Plus, the nice thing about NVDA Remote is there is much less latency. I remember trying to use team viewer to remote in from a laptop to a desktop at home, and with a bad connection, I could barely get anything done. Same connection, with NVDA Remote and it was like I was sitting right there at that desktop. I run an online radio station, and when traveling, I often have to remote in to get some things done.

By Samuel on Sunday, July 25, 2021 - 08:03

Hello,

I found answers to most of my questions and problems on the app.
The only which still exists is the keyboard which switch to a qwertz mod during the utilisation.
I don't understand why and it's very anoying!
Can you help me with that?

Appart from that, thank again for this app.

By Saltemohn on Wednesday, August 25, 2021 - 08:03

Hello,
Sorry, I haven't looked in here for a while as I didn't think there were any bug reports coming in here about my app. As I wrote above, please report these bugs via GitHub (https://github.com/EscolarProgramming/NVDARemotePublic)! There it's a lot clearer for me than here in the forum (where I won't check in the future either, I already have 4 sources to check daily, and I can't be active on even more platforms). If there is no other way, contact me via email: nvdaremote@mjs-soft.de. GitHub is still the preferred means though.

The already written bugs I will of course remember and work off, but future ones I will most likely not notice.
Thanks for your understanding!