No internet connection
  1. Home
  2. Technical Support

Mouse HID gets interpreted as touch on Chromebook

By Vincent Mo @vmo
    2021-08-04 05:30:08.842Z

    I'm controlling a Chromebook remotely from a Windows 10 computer through Chrome (holy inception, Batman!).

    The Chromebook seems to interpret the TinyPilot HID device (/dev/hidg1) as touch input. When I wave my mouse around on the Windows 10 computer, the Chromebook's cursor doesn't follow. When I click, the Chromebook treats it as a touch. Long-left-click does what a long tap would normally do, pulling up the Android-like context menu. The scroll wheel doesn't work. Instead, scrolling is done by left-clicking and dragging - like you would do with a touch interface.

    I was able to replicate this on two different Chromebooks. Everything works perfectly (no touch-like behavior) when I'm remotely controlling a Windows computer.

    Logs: https://logs.tinypilotkvm.com/q6Flf5S3

    Screen shot after long-left-clicking the word "easy":

    Reply
    • 5 replies
    1. Michael Lynch @michael2021-08-04 20:39:46.367Z

      Thanks for reporting this! I haven't heard of this before. I've filed a bug to track:

      https://github.com/tiny-pilot/tinypilot/issues/753

      If you plug a USB mouse into the Chromebook, does it behave like a normal mouse? I'm trying to think of a reason why ChromeOS would treat TinyPilot differently.

      Reply
      1. V
        In reply tovmo⬆:
        Vincent Mo @vmo
          2021-08-05 01:11:32.472Z2021-08-05 01:35:53.760Z

          Yep! Plugging the mouse in directly works normally.

          I was wondering if it might have something to do with the report descriptor used for the mouse hid... maybe there's something about it that makes ChromeOS think it's a touch device rather than a normal mouse? I don't know anything about report descriptors though, and I didn't get to the point of trying to modify that on my Tiny Pilot to see if a different configuration would work. I just got to the point of discovering that you can't just mv the file and replace it with a different one but it has to be written at OS startup? Happy to try stuff if you have ideas. I'm comfortable with Linux (though I know Ubuntu best), and I've worked with Flask before too.

          Reply
          1. Michael Lynch @michael2021-08-05 20:20:09.325Z

            It's possible to test different descriptors if you're interested. The report descriptors are in /opt/tinypilot-privileged/init-usb-gadget. You can reload the virtual HID mouse without a reboot by running sudo service usb-gadget restart. And you can see errors with sudo journalctl -u usb-gadget.

            Note that if you run a TinyPilot update, it will overwrite the init-usb-gadget file with the standard version, but if you find a different descriptor that's ChromeOS-compatible, we can work on integrating it into a future release.

            Reply
            1. VVincent Mo @vmo
                2021-08-05 22:04:19.328Z

                Oh, thanks! I'll poke around there when I've got a spare moment.

                Reply
            2. Progress
              with handling this problem
            3. V
              Vincent Mo @vmo
                2021-09-02 08:00:22.148Z

                @michael - Finally got a chance to poke around on this a bit. I think Chrome OS is treating the mouse inputs as touch events if the X/Y input is interpreted as absolute coordinates. When it's changed to relative (\\x81\\x06), the scroll wheel begins to work, which makes me think Chrome OS is now treating it as mouse input rather than touch input. Of course, then all the mouse coordinates are messed up.

                Reply
                Reply (discussion)Add progress note