tinypilot nvr integration

The AIMOS KVM works, but it's not an officially supported component. If we're trying to track down why the mouse isn't passing through, it'll be easier if we cut out the third-party components to see if it's an AIMOS problem or a TinyPilot problem.

Hi David,

Thanks for the quick reply. Here is some abbreviated and summarized output from my recent troubleshooting with ChatGPT. Take a look and let me know what you think. It does seem to lead to the same conclusion about supporting relative mouse positioning.

I did disable keyboard as the NVR does not accept keyboard input. To do this I did the following:

• added the following line in /home/tinypilot/settings.yml
  usb_keyboard_enabled: false
• Commented out the following line in /home/tinypilot/app_settings.cfg
  # KEYBOARD_PATH = '/dev/hidg0'

output from ChatGPT session:

Stage:

I told ChatGPT about the problem, that it is a python app capturing input from a web client and proxying to /dev/hidg1.

ChatGTP:

Testing the NVR by writing directly to /dev/hidg1. Try running the following script and observe the response on the NVR screen. Save as move_mouse.py and run as:

sudo python3 move_mouse.py

import struct
import time

# Function to write HID report to /dev/hidg1
def move_mouse_left(total_distance):
    # Each step will be -127 to stay within the byte range
    step = -127
    remaining_distance = total_distance
    
    with open('/dev/hidg1', 'wb') as f:
        while remaining_distance < 0:
            x_movement = max(step, remaining_distance)
            y_movement = 0  # No movement on Y-axis
            buttons = 0  # No buttons pressed
            wheel = 0  # No wheel movement

            # Construct HID report: 1-byte button, 1-byte x movement, 1-byte y movement, 1-byte wheel
            report = struct.pack('bbbb', buttons, x_movement, y_movement, wheel)
            f.write(report)
            
            remaining_distance -= x_movement
            # Optional sleep to simulate real mouse movement
            time.sleep(0.01)

# Move mouse left by -1600 pixels (in chunks of -127)
move_mouse_left(-1600)

Results:

it's not working the mouse pointer is not moving on the screen.

ChatGPT:

Try this one

import struct
import time

# Function to write HID report to /dev/hidg1
def move_mouse_left(total_distance):
    # Each step will be -127 to stay within the byte range
    step = -127
    remaining_distance = total_distance

    with open('/dev/hidg1', 'wb') as f:
        while remaining_distance < 0:
            x_movement = max(step, remaining_distance)
            y_movement = 0  # No movement on Y-axis
            buttons = 0  # No buttons pressed
            wheel = 0  # No wheel movement

            # Construct HID report: 1-byte button, 1-byte x movement, 1-byte y movement, 1-byte wheel
            report = struct.pack('bbbb', buttons, x_movement, y_movement, wheel)
            
            # Debugging output
            print(f"Sending report: {report}")

            f.write(report)
            
            remaining_distance -= x_movement
            # Optional sleep to simulate real mouse movement
            time.sleep(0.01)

# Move mouse left by -1600 pixels (in chunks of -127)
move_mouse_left(-1600)

Results:

Still not working. Can we test sending a right mouse click?

ChatGPT:

Sure, try this one

import struct
import time

# Function to write HID report to /dev/hidg1
def send_right_click():
    with open('/dev/hidg1', 'wb') as f:
        # Construct HID report: 1-byte button, 1-byte x movement, 1-byte y movement, 1-byte wheel
        # Button byte: 0x02 (right button pressed), x movement: 0, y movement: 0, wheel: 0
        right_click_down = struct.pack('bbbb', 0x02, 0x00, 0x00, 0x00)
        right_click_up = struct.pack('bbbb', 0x00, 0x00, 0x00, 0x00)
        
        # Send right button down
        f.write(right_click_down)
        print("Right button down")
        
        # Short delay to simulate a click
        time.sleep(0.1)
        
        # Send right button up
        f.write(right_click_up)
        print("Right button up")

# Send a right mouse click
send_right_click()

Results:

so the right click did not show up on the screen. However, when I went to the web interface and did a physical right click the menu popped up in the top left corner. Though the mouse courser was still displayed in the center of the screen. This is after all the test runs of move_mouse.py

ChatGPT:

It sounds like there might be an issue with how the mouse movements and clicks are being interpreted by the NVR. Given that the right click registers in the top left corner while the cursor remains in the center, there might be an issue with the relative/absolute positioning or how the mouse coordinates are being sent.