TinyPilot using remote desktop and ipad with magic keyboard
I am using this awesome tool and I am loving it. when I want to access tiny pilot from outside my home network and I use my iPad browser (chrome, firefox or Safari) I experiance unbearable slowness, so I decided to connect to a virtual machine in my home network using a remote desktop and then use tinyPilot within that VM, and that works perfectly, however, I am facing an issue with keyboard keystrokes, TinyPilot doesn't take/recognize magic keyboard keystrokes (https://www.apple.com/ca/ipad-keyboards/), it shows error in the lower right corner of screen, it says 'Keystorke faield' and the key I press displayed inside a red square, see below screenshot.
for some reason, some keys work like new line (enter) and backspace.
it worth to mention that when I connect from ouside my network to the VM inside my home network using a PC or laptop, the keystroke work as expected.
one more note: keyboard keystrokes works fine with any other application (aside from TinyPilot) on my ipad when I connect to my home vm using remote desktop.
I pulled the logs from TinyPilot and here is the link:
Please let me know if something I can do on my end to fix this issue or if you can release an hot fix for it
- 6 replies
From the logs, it looks like the iPad sends keys in an unusual format:
Aug 28 19:51:33 raspberrypi python: [2021-08-28 19:51:33,110] WARNING in socket_api: Unrecognized key: d (keycode=)
It should show keycode=KeyD when you push the D key.
Can you try visiting https://keycode.info/ from your iPad browser and share a screenshot of what it looks like when you push the 'D' key?
- BIn reply toBelal⬆:Belal Salas @Belal
Thank you for your response, I thought it is will be more useful to provide all keys instead of D only, here is a video:
I have done the same while connected to the TinyPilot using RD and here is the logs:
I see Unrecognized key everywhere in the log
This is strange. I thought that there was something with Safari iPad not setting the
event.codefield, but it's clearly working in the video.
Just to summarize:
- When your connection is iPad -> TinyPilot - Win10, the keyboard works but the video is slow
- When your connection is iPad -> RDP -> VM server -> TinyPilot -> Win10, the video is responsive, but certain keys fail to forward
Is that correct?
It might be that Remote Desktop doesn't emit keystrokes in the same way as a real device. The video above was you directly visiting keycode.info from your iPad, right? Can you try the same setup where you're seeing the TinyPilot failure, so iPad -> RDP -> VM server -> keycode.info? Video like before works, but all I need is a screenshot of a single screen where you're pressing the 'D' key.
Edit: Have you tried tuning the video settings?
"When your connection is iPad -> TinyPilot - Win10, the keyboard works but the video is slow
When your connection is iPad -> RDP -> VM server -> TinyPilot -> Win10, the video is responsive, but certain keys fail to forward"
above statements are correct.
I guess your assumption is correct, I have two screenshots, one for D char from my Mac using RD:
and one from my iPad using RD:
do you think we can do anything to fix this issue, or it is Microsoft RD issue and they have to fix it?
Yes I tried tuning video settings, it doesn't help with keystrokes.
That's strange. I can't reproduce it. I'm hitting RDP from Windows rather than an iPad, so that might be the difference.
Could you try RDPing from a non-iPad device to see if the keystrokes go through that way?
On the Windows VM, have you tried other browsers? It looks like you're running Firefox on the Windows VM, but I'm curious if Edge or Chrome behave differently.
Another workaround might be to replace RDP with another remote access tool like TeamViewer or VNC.
- BIn reply toBelal⬆:Belal Salas @Belal
Hi,Please update this thread if something new come up regarding same topic.
I tried it on no-ipad and it worked as expected.
I tried connecting using TeamViewer and it worked just fine.
I tried Edge as well on RDP through the ipad and it didn't work, here is a screenshot: