No internet connection
  1. Home
  2. Technical Support

TinyPilot won't use attached USB Media for Virtual Media

By Paul Dunnells @pauldsolid
    2024-08-24 01:19:24.567Zassigned to
    • @david

    Installed 256GB USB Stick, ran move-virtual-media-to-disk --uuid ${UUID} script. Script failed after timeout because image transferred was too big. (11GB RHEL 9 Image). Deleted Image, re-ran script. Rebooted TinyPilot, checked mountpoints. All looks good....
    However, uploading any media does not go to the USB stick, it goes to the SD Card.... Re-running script does nothing other than report that the device, as sda1, is already mounted and configured.....
    I need a lot more space than the 24GB...... Each RHEL Image is about 13GB, and I have 2, 8 and 9. Plus the prep utility ISO, PartedMagic, is about 2.5GB. Not enough space.
    I could use some help here.

    Solved in post #20, click to view
    • 20 replies

    There are 20 replies. Estimated reading time: 15 minutes

    1. David @david2024-08-26 12:00:27.054Z

      Hi @pauldsolid, I'm sorry you're running into these virtual media issues.

      Based on your description, I think there could be a couple of issues happening simultaneously.

      Do you have the error message (image too large) from when you ran the move-virtual-media-to-disk script? That might help us find a bug in the script. Don't worry if not.

      If the move-virtual-media-to-disk script failed, it's likely your TinyPilot hasn't correctly set up your USB drive as the virtual media disk even if the disk remains mounted. That would explain why your uploads don't use your USB disk for storage.

      Could you run lsblk -o NAME,SIZE,FSTYPE,VENDOR,MODEL,LABEL,UUID and share the output for your USB disk? It should look something like this:

      NAME          SIZE FSTYPE VENDOR   MODEL            LABEL  UUID      MOUNTPOINT
      sdb         116.2G        Generic  MassStorageClass                                             
      └─sdb1      116.2G vfat                             test   A4A7-1D9D /media/tinypilot-mass-storage
      

      That info should show me the current state of your drive and allow me to suggest a method to get you back up and running using Virtual Media with it.

      Please let me know if you have any questions.

      1. PPaul Dunnells @pauldsolid
          2024-08-26 17:32:29.926Z

          Here is the output from the script, when an existing image is in place.

          UUID=858C-7143  /media/tinypilot-mass-storage   vfat    defaults,gid=tinypilot,uid=tinypilot    0       0
           sending incremental file list
          CentOS-Stream-9-latest-x86_64-dvd1.iso
            4,290,019,328  38%    2.49MB/s    0:45:38  rsync: [receiver] write failed on "/media/tinypilot-mass-storage/tinypilot-image-uploads/CentOS-Stream-9-latest-x86_64-dvd1.iso": File too large (27)
          rsync error: error in file IO (code 11) at receiver.c(378) [receiver=3.2.3]
          4,295,065,600  38%    2.51MB/s    0:45:18  rsync: [sender] write error: Broken pipe (32)
          
        • P
          In reply topauldsolid:
          Paul Dunnells @pauldsolid
            2024-08-26 17:34:26.890Z

            Here is the output of lsblk -o NAME,SIZE,VENDOR,MODEL,LABEL,UUID

            pilot@pauldtiny:~ $ lsblk -o NAME,SIZE,VENDOR,MODEL,LABEL,UUID
            NAME          SIZE VENDOR   MODEL        LABEL    UUID
            sda           231G          USB_DISK_3.0          
            └─sda1      230.9G                       USB DISK 858C-7143
            mmcblk0      29.1G                                
            ├─mmcblk0p1   256M                       bootfs   9E81-4F92
            ├─mmcblk0p2  28.8G                       rootfs   cf2895ca-6dc2-4797-8040-f76ba1508f41
            └─mmcblk0p3     4M 
            
            1. P
              In reply topauldsolid:
              Paul Dunnells @pauldsolid
                2024-08-26 17:35:54.888Z

                Here is the output of mount:

                pilot@pauldtiny:~ $ mount
                /dev/mmcblk0p2 on / type ext4 (rw,noatime)
                devtmpfs on /dev type devtmpfs (rw,relatime,size=592636k,nr_inodes=148159,mode=755)
                proc on /proc type proc (rw,relatime)
                sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,relatime)
                securityfs on /sys/kernel/security type securityfs (rw,nosuid,nodev,noexec,relatime)
                tmpfs on /dev/shm type tmpfs (rw,nosuid,nodev)
                devpts on /dev/pts type devpts (rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000)
                tmpfs on /run type tmpfs (rw,nosuid,nodev,size=342748k,nr_inodes=819200,mode=755)
                tmpfs on /run/lock type tmpfs (rw,nosuid,nodev,noexec,relatime,size=5120k)
                cgroup2 on /sys/fs/cgroup type cgroup2 (rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot)
                pstore on /sys/fs/pstore type pstore (rw,nosuid,nodev,noexec,relatime)
                bpf on /sys/fs/bpf type bpf (rw,nosuid,nodev,noexec,relatime,mode=700)
                systemd-1 on /proc/sys/fs/binfmt_misc type autofs (rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct)
                debugfs on /sys/kernel/debug type debugfs (rw,nosuid,nodev,noexec,relatime)
                mqueue on /dev/mqueue type mqueue (rw,nosuid,nodev,noexec,relatime)
                tracefs on /sys/kernel/tracing type tracefs (rw,nosuid,nodev,noexec,relatime)
                sunrpc on /run/rpc_pipefs type rpc_pipefs (rw,relatime)
                fusectl on /sys/fs/fuse/connections type fusectl (rw,nosuid,nodev,noexec,relatime)
                configfs on /sys/kernel/config type configfs (rw,nosuid,nodev,noexec,relatime)
                /dev/mmcblk0p1 on /boot type vfat (rw,relatime,fmask=0022,dmask=0022,codepage=437,iocharset=ascii,shortname=mixed,errors=remount-ro)
                /dev/sda1 on /media/tinypilot-mass-storage type vfat (rw,relatime,uid=111,gid=116,fmask=0022,dmask=0022,codepage=437,iocharset=ascii,shortname=mixed,errors=remount-ro)
                tmpfs on /run/user/1001 type tmpfs (rw,nosuid,nodev,relatime,size=171372k,nr_inodes=42843,mode=700,uid=1001,gid=1001)
                
                1. P
                  In reply topauldsolid:
                  Paul Dunnells @pauldsolid
                    2024-08-26 17:39:16.827Z

                    Output of the lsblk command as you requested:
                    lsblk -o NAME,SIZE,FSTYPE,VENDOR,MODEL,LABEL,UUID

                    pilot@pauldtiny:~ $ lsblk -o NAME,SIZE,FSTYPE,VENDOR,MODEL,LABEL,UUID
                    NAME          SIZE FSTYPE VENDOR   MODEL        LABEL    UUID
                    sda           231G                 USB_DISK_3.0          
                    └─sda1      230.9G vfat                         USB DISK 858C-7143
                    mmcblk0      29.1G                                       
                    ├─mmcblk0p1   256M vfat                         bootfs   9E81-4F92
                    ├─mmcblk0p2  28.8G ext4                         rootfs   cf2895ca-6dc2-4797-8040-f76ba1508f41
                    └─mmcblk0p3     4M                                       
                    
                    1. P
                      In reply topauldsolid:
                      Paul Dunnells @pauldsolid
                        2024-08-26 17:40:28.547Z

                        Let me know what other information you need.

                        1. P
                          In reply topauldsolid:
                          Paul Dunnells @pauldsolid
                            2024-08-26 18:02:26.198Z

                            Ok, I see the problem.
                            VFAT
                            VFAT is limited to 4GB files, which is too small for the ISOs I'm working with. Too small for most modern deployment Images.
                            Is there a way to reformat and mount within TinyPilot to either xfs or ext4 and get this working.
                            Ah, filesystem limitations......

                            1. PPaul Dunnells @pauldsolid
                                2024-08-26 19:06:11.618Z

                                The majority of the images I'm using, mainly Linux Deployment ISOs, are at least 13.6 GiB

                              • P
                                In reply topauldsolid:
                                Paul Dunnells @pauldsolid
                                  2024-08-27 00:37:25.752Z

                                  Ok, i effectively killed my tinypilot :(
                                  O reformatted my usb device to ext4 and rebooted the tinypilot for another reason, and now it won't boot.

                                  1. P
                                    In reply topauldsolid:
                                    Paul Dunnells @pauldsolid
                                      2024-08-27 01:52:02.685Z

                                      Restored the usb to vfat, but no joy. Now what do I do? I do have local access.

                                      1. David @david2024-08-27 11:56:42.192Z

                                        Thanks for all those updates, @pauldsolid!

                                        I can confirm that the script will fail if you use a FAT32 formatted drive and attempt to move an image >4GB. In this instance, the set up won't complete - so that explains the initial issue.

                                        There may be a way we can avoid this error within the script itself (i.e., by checking the filesystem type and the sizes of images that the script will move), so I've created an internal ticket to look into fixing this.

                                        now it won't boot

                                        Are you experiencing this error?:

                                        If so, I ran into this issue as well, so it looks like there's a bug somewhere. I was able to restore access by restoring the default virtual media location.

                                        sudo /opt/tinypilot-privileged/scripts/restore-default-virtual-media-location
                                        

                                        I'm currently looking into why this error occurred and how we can fix it. For now, I'd recommend against trying to use external storage for TinyPilot's virtual media. As a workaround, would it be possible for you to acquire and use a larger microSD card (e.g., 256GB) in place of the TinyPilot default microSD card?

                                        1. PPaul Dunnells @pauldsolid
                                            2024-08-27 15:57:25.868Z

                                            Perhaps we need a new ticket for this issue.
                                            I can't connect to the TinyPilot at all after rebooting. No ssh, no Webpage.
                                            I'll try to see if I can scan the network and see if the hostname just no longer works. These are different issues.

                                            1. PPaul Dunnells @pauldsolid
                                                2024-08-27 16:16:47.481Z

                                                Forked the no-boot issue to "TinyPilot will not boot after using removable media and changing Filesystem" to not muddy the USB Media Issue.

                                          • P
                                            In reply topauldsolid:
                                            Paul Dunnells @pauldsolid
                                              2024-08-27 16:19:21.918Z

                                              It is very possible for me to use a larger microSD card.
                                              What is the process to apply a new boot and software image to the microSD card?
                                              Perhaps that will also restore my TinyPilot to health too.

                                              1. David @david2024-08-27 18:10:42.302Z

                                                I can't connect to the TinyPilot at all after rebooting. No ssh, no Webpage.

                                                It could be that the TinyPilot rebooted in a bad state (like you mention in the other issue you created).

                                                Could you try performing a factory reset? That should return your TinyPilot to a working state, and if you perform the factory reset on a larger microSD card, you can start to freely use more space without worrying about the storage limit for virtual media.

                                                Since a factory reset covers your other open issue, I'm going to close that issue for now - I'm don't think there's much benefit in exploring that particular issue any further.

                                                I hope that helps! Please let me know if you have any questions.

                                                1. PPaul Dunnells @pauldsolid
                                                    2024-08-27 18:18:43.994Z

                                                    Thanks @david , I agree.
                                                    I found the procedure for the Factory Reset (Which I overlooked thinking it was a button) and am getting what I need to re-image the SD Card. I've got a much larger card (128GiB) on order and is expected to be here tomorrow. I may re-flash the existing card just to get my tinypilot up and running and get in some practice on the whole process.
                                                    Looking at a few other issues relating to using external USB drives for this sort of thing, I think a larger SD card is the better approach for now until the devs can get it sorted.
                                                    Cheers

                                                    1. David @david2024-08-28 11:18:22.719Z

                                                      Thanks for the update! I agree that using only a microSD is the best way forward for now.

                                                      Feel free to reach out if you have any other questions or issues.

                                                2. P
                                                  In reply topauldsolid:
                                                  Paul Dunnells @pauldsolid
                                                    2024-08-29 17:53:25.984Z

                                                    This issue is resolved.
                                                    Points of note:
                                                    External USB Devices, being formatted VFAT, only support an image size of 4GiB. I've found most O/S deployment ISOs to be >8Gib and about 10GiB in size on average.
                                                    TinyPilot does not currently have a mechanism to reformat an external USB Device to use XFS or Ext4, the scripts only support VFAT.
                                                    In order to support these larger ISOs, using the TinyPilot Media feature, it is necessary to re-image using a large enough micro HCSD to accommodate the required repository size.

                                                    ReplySolution
                                                    1. Progress
                                                      with handling this problem
                                                    2. P
                                                      Paul Dunnells @pauldsolid
                                                        2024-08-29 17:48:34.744Z

                                                        Successfully re-imaged the system using a 128GiB SanDisk MicroSD
                                                        Was able to get the system back up and running, with the web page at least.
                                                        Set Time (You need to add instructions for this, or a WebPage Control Knob), configured network, and set up host name.
                                                        Video is not working. Will open a new ticket on this.

                                                        1. David @david2024-08-30 11:26:19.722Zreplies topauldsolid:

                                                          Thanks for those updates, @pauldsolid!

                                                          I'm glad you're back online with a larger microSD.

                                                          Regarding vfat, I've created some internal tickets to look into whether we can improve TinyPilot's functionality around this.

                                                          Set Time (You need to add instructions for this, or a WebPage Control Knob)

                                                          Can you explain what the issue was here? TinyPilot should automatically set its own time using NTP.

                                                          Video is not working. Will open a new ticket on this.

                                                          Are you still experiencing this issue? If so, when you create a new ticket, could you include a log from your TinyPilot? On your TinyPilot web interface, go to System > Logs and click Get Shareable URL. Then paste the link in your reply.