Source of video latency
Aside from network, is the TinyPilo video latency primarily a product of the Pi CPU limits or protocol/software?
David @david2023-11-03 12:51:42.646ZHi @sr800, thanks for your great question!
Like you pointed out, the network is often the primary source of latency. Using your TinyPilot locally over ethernet offers much lower latency compared to using Wi-Fi over the internet half way across the world. Reducing the bandwidth your TinyPilot uses (by adjusting the video settings) is one of the best ways to improve latency since it reduces the amount of data sent over the network.
The Raspberry Pi 4B's processing power doesn't seem to be a bottleneck. For instance, if you check resource usage while running the TinyPilot software, you'll see that there's still headroom for additional processing (i.e., the CPU isn't 'maxed out').
The protocol you use with your TinyPilot (MJPEG or H.264) definitely has a significant impact on latency. So we suggest using H.264 where possible.
We've noticed that when TinyPilot's video streaming service receives updates, latency can improve, suggesting that additional improvements could reduce latency even further.
I hope that answers your question!