TinyPilot occationally freezes, then audio is distorted after restarting TinyPilot
- @cghague
I have been using the new H.264 with audio support on my TinyPilot Voyager 2a and it works well until TinyPilot freezes. When the remote screen freezes, I refresh my web browser (FireFox) and the video feed won't come back. Even MJPEG fails. The only way to get the video feed working again is to go to the TinyPilot "System" -> "Power" -> "Restart" button option.
However, after TinyPilot restarts, the audio portion of the video feed is loud and distorted--like a loud buzzing sound. The only way I can fix this is to completely power down TinyPilot and pull the power from TinyPilot. Once I have powered TinyPilot back up, the video feed and the audio work fine again. However, this is not a practical, long term solution.
- CCharles Hague @cghague2023-02-17 19:37:47.928Z
Hi Aaron, thanks for reaching out. I’m sorry to hear you’re having problems with your TinyPilot device. I’ve not encountered this issue before so we’ll need to investigate further. Can you please send me a link to your logs the next time the device freezes, after restarting it from the web interface and after power cycling it? You can share your logs by going to System, then Logs and then Get Sharable URL.
- CIn reply toAaronP⬆:Charles Hague @cghague2023-02-23 12:28:54.865Z
Thanks for getting back to me. The links in your post contained potentially sensitive data so I’ve hidden them from the public forum. I’ve read through the logs and noticed that it looks like you are logging in from two different networks (one is 10.x.x.x and one is 172.x.x.x). This could be due to a VPN or similar service being active but it could also point to a network configuration issue. Can you please check if the issue happens when you are directly connected to the same local network as your TinyPilot device?
- AAaron @AaronP
Hi Charles, Yes, the issue happens both remotely and also when I am directly connected to the same local network as the TinyPilot. I first noticed the problem when I was connected on the same local network. FYI, the issue of freezing and requiring a reboot never happened locally or remotely when using the previous version of the TinyPilot software.
- In reply tocghague⬆:AAaron @AaronP
FYI: Here is the log for when I shut down TinyPilot, unplugged it and then plugged it back in. TinyPilot is running with perfect sound support again:
- CIn reply toAaronP⬆:Charles Hague @cghague2023-02-24 11:18:12.631Z
Thanks for confirming that both local and remote connections are affected. I might need to escalate this to our development team, but I’d like to confirm a few more details first to ensure I have the full picture. Could you please check if sounds from the target computer can still be heard when the audio is distorted? Can you please also let me know if the freezing and/or audio distortion continue if you switch to a private window or a different browser?
- AAaron @AaronP
Hi Charles, yes sound from the target computer can still be heard clearly and cleanly--all while the audio is distorted and screeching when coming through the TinyPilot web interface on both Firefox and Chrome. Again, this only occurs after TinyPilot freezes and requires a restart. Once I initiate a TinyPilot restart after a freeze, H.264 video is restored, but audio coming through the feed is distorted and screeching: no matter if I use a different web browser or even a complete different computer on the same local network. The only solution is to power down TinyPilot and and unplug it and boot it back up. Once TinyPilot boots back up, H.264 audio is clean again. All I have to do is refresh my TinyPilot web browsing window and all is good again. The problem is that I don't want to have to power down the device every day to resolve this.
- CCharles Hague @cghague2023-02-24 19:05:24.859Z
Thanks for the detailed response, that’s really helpful information. It’s unusual for a power cycle to have a different effect to a restart, so that could certainly help us to narrow down the problem. If possible, the next time the audio issue happens, could you please try detaching and then re-connecting the HDMI cable and confirming if the distortion continues?
- AAaron @AaronP
Yes, I have already tried disconnecting the HDMI cable and reconnecting the HDMI cable after the distortion happened with the TinyPilot restart, because I thought I may have had a bad HDMI cable. I even went so far as to replacing the HDMI cable with another known good cable and the distortion still persisted. I even then went further to order a brand new 4star rated HDMI cable from Amazon, and the problem still persisted as described above. My last resort was to power down and unplug the TinyPilot device. That’s what ended up prompting me to post the very first message on this issue.
- CCharles Hague @cghague2023-02-25 19:26:50.484Z
Thank you for confirming! I’ll share this information with the team and get back to you as soon as I have an update.
- RRicardo Sotolongo @rsotolongo
@cghague Any update with this issue? I'm having EXACTLY the same issue than @AaronP and I have tried EVERYTHING than @AaronP tried too. No matter what I do, the only thing that back the audio to normality, is disconnecting the TP from the power (PoE in my case). I have two TP Voyager 2a and both behave the same.
Michael Lynch @michael2023-03-24 18:28:49.559Z@rsotolongo - thanks for reporting this!
We received back @AaronP's device, but we haven't been able to reproduce the behavior in our environment.
Can you share a bit more about how you see this to help us investigate?
- How long does it take until you notice the effect?
- What kind of computer is the TinyPilot connected to?
- Is there anything between the device and the TinyPilot, or does the HDMI cable connect directly to the target device?
- RRicardo Sotolongo @rsotolongo
Hi, sorry for don't reply early but I was trying "new" things. I continue with the same issue. After power the TinyPilot with the official USB-C adapter it took longer until TP freeze again, and sound started to be distorted. As I said before I tried EVERYTHING @AaronP tried following @michael instructions with the same result. I mean, after a while (not know exactly how long) the TP freeze and then after TP restart the sound started to be distorted, the only way sound back to be "normal" is unplug the ethernet cable (my previous power source) and plugged back again and refresh TP in the browser. After power the TP using the USB-C adapter I was able to restart TP and sound was OK until TP freeze and no matter what I do, sound always back distorted until power cables (Ethernet and USB-C) is disconnected and connected back. Now I'm trying two different things: 1- I change the microSD card with a super faster card instead the official one included with the TP purchased (of course with latest firmware flashed/installed). 2- Accessing the TP through HTTPS secure instead of the regular HTTP that I was using before. I will let you know the result of my tests once TP freeze again. I'm connecting TP directly to a laptop in front of me: HDMI to HDMI (no middle adapters) and USB-A to USB-A (no middle adapters). The laptop is always on and it is an Asus G14.
- Michael Lynch @michael2023-03-29 21:47:15.073Z
Thanks for the additional details, @rsotolongo!
We're trying to figure this out, but we're not sure what it is yet. It seems related to the environment, because we've found that a customer who replaced their TinyPilot Voyager 2a and all cables/power still saw this issue, so we're trying to figure out what in the environment or target computer triggers this.
- RRicardo Sotolongo @rsotolongo
@michael Thanks. My TPs are product of trade-ins my previous TP Voyager 2 but I have a friend that just bought a new one Voyager 2a (not trade-in) and he started to have the same issue. He is using a super-fast MicroSD card with latest firmware flashed and high-quality HDMI and USB cables. Should be something in the hardware because restarting the device by the menu option don't solve the issue, just disconnecting the power sources for few seconds.
- Michael Lynch @michael2023-03-30 19:51:16.232Z
@rsotolongo - Yeah, this is a tricky one. We're digging into this to try and figure it out.
- Can you share what OS you're running on ASUS G14?
- Can you try disabling the automatic screen sleep on the ASUS G14 to see if that affects the frequency of this bug?
- Next time you run into this, can you try going to System > Video Settings and setting the streaming mode to MJPEG?
Can you also ask your friend that's also hitting this to get in touch with us at support@tinypilotkvm.com? We're trying to see what's common across the customers who hit this so we can narrow in on the root cause.
- RRicardo Sotolongo @rsotolongo
Hi, the client laptop is running Windows 11 Home Edition with latest updates installed and with all drivers (BIOS included) updated to latest version too. Screen sleep has been disabled since day one, it is always ON laptop. If I put MJPEG as the video format I lost audio and audio is the key feature why I need Voyager 2a. I sent this thread to my friend to keep him in the loop.
- Michael Lynch @michael2023-03-31 15:27:48.514Z
If I put MJPEG as the video format I lost audio and audio is the key feature why I need Voyager 2a.
If you switch to MJPEG, does video work again? I ask only because it helps us understand what's causing the problem. If video is only frozen in H.264 mode but switching to MJPEG works, then it suggests a software problem in one of the components responsible for serving H.264 video. But if both H.264 and MJPEG are broken, it suggests a problem in the hardware or lower in the software.
- RRicardo Sotolongo @rsotolongo
Hi, once TP freeze, I change to MJPEG and video is back, but it is too slow compared with H.264 and eventually it freezes too. Example of a timeline on last Friday: TP was disconnected and connected at 11:30 AM EST, working fine (video and sound) until 09:30 PM EST (10 hours) when a black screen flashed and TP freeze. I refreshed the browser and video and sound get back perfect but 11:27 PM EST (almost 2 hours later) again a black screen flash taking out video and sound (freeze). This time no luck refreshing the browser, a restart was needed and video back to normal but not the sound. At this time, I change to MJPEG (slow, big lag) until next day (Saturday) at 10:13 AM EST that get freeze again and I change back to H.264 without sound until I disconnect and connect the power source.
- CCharles Hague @cghague2023-04-06 20:05:24.453Z
Hi Ricardo, thanks for getting back to us. I just wanted to give you an update as I appreciate how frustrating bugs like this can be. We're still looking into this within the team and we're working with our hardware partners to determine the next steps in the investigation. I'll get back to you as soon as we have more information or if we have any questions. I'm optimistic we'll be able to resolve this!
- PParker @parker
Was any progress made on this? I believe we're experiencing the same problem with a recently purchased refurb TinyPilot POE.
It works great for a while, then the image will freeze. We have it plugged into a KVM, and switching inputs, or unplugging the TP HDMI, will continue to show the same image. I can't speak to the audio, since we aren't using it.
The web interface still responds, but the video stream remains the same. Restarting the TP from the interface will help, sometimes, only briefly, before it freezes again. A hard restart will help for a longer amount of time, but inevitably will freeze again, sometimes pretty quickly.
- RRicardo Sotolongo @rsotolongo
Unfortunately, there was no work-around. My three TP PoE were, one by one, sent back to TinyPilot and replaced with new ones. Since then, I had no other issues with the image or with the sound. Very happy with the devices working 24/7.
- In reply toparker⬆:CCharles Hague @cghague2023-11-27 20:16:12.839Z
Hi @parker, thank you for your message. I'm sorry to hear you're having issues with video freezing.
I'd like to investigate this further, as there could be a few potential explanations. Could you please send me a link to your logs the next time the video freezes? You can do this by going to System, then Logs, then Get Shareable URL.
- PParker @parker
Will-do.
Is there a different place I should send logs rather than posting here?
For now, I have adjusted some of the encoding settings to see if it helps.
- CCharles Hague @cghague2023-11-27 23:07:37.921Z
Thanks for the quick response! The Log window will generate a URL that you can post on this thread. You can preview the information that will be shared, and an optional setting will hide potentially sensitive information.
- PParker @parker
We just had it happen again. I pulled the log link, with sensitive data hidden, but it appears there are still some domains in there I would rather not post publicly. Is there another option?
Thanks.
- CCharles Hague @cghague2023-12-05 04:11:52.571Z
Thanks for getting back to me! I completely understand and would be happy to continue this investigation over email. Can you please send the logs as an attachment to our support email address, mentioning this thread in the message?
- Progresswith handling this problem
- AAaron @AaronP
Video Feed Freezing and refreshing Window Fails:
Removed by TinyPilot teamAfter Restart of TinyPilot Device:
Removed by TinyPilot teamTinyPilot now has severely screeching and distorted audio coming through on H.264 feed:
Removed by TinyPilot teamThe only way to fix this will be for me to power down TinyPilot, pull power cable and then power back up.