How to fsck the root filesystem on a tinypilot bought from the website?
I noticed the root filesystem on our tinypilot has errors, and it seems it's not been fixed at reboot. I had a look using dump2fs, and this is the result:
root@tinypilot:~# dumpe2fs -h /dev/mmcblk0p2
dumpe2fs 1.44.5 (15-Dec-2018)
Filesystem volume name: rootfs
Last mounted on: /
Filesystem UUID: 7295bbc3-bbc2-4267-9fa0-099e10ef5bf0
Filesystem magic number: 0xEF53
Filesystem revision #: 1 (dynamic)
Filesystem features: has_journal ext_attr resize_inode dir_index filetype needs_recovery extent flex_bg sparse_super large_file dir_nlink extra_isize
Filesystem flags: unsigned_directory_hash
Default mount options: user_xattr acl
Filesystem state: clean with errors
Errors behavior: Continue
Filesystem OS type: Linux
Inode count: 1884960
Block count: 7542016
Reserved block count: 307604
Free blocks: 6255817
Free inodes: 1653384
First block: 0
Block size: 4096
Fragment size: 4096
Reserved GDT blocks: 94
Blocks per group: 32768
Fragments per group: 32768
Inodes per group: 8160
Inode blocks per group: 510
Flex block group size: 16
Filesystem created: Fri May 7 14:59:42 2021
Last mount time: Fri Jul 29 17:09:24 2022
Last write time: Mon Aug 1 19:17:01 2022
Mount count: 1
Maximum mount count: -1
Last checked: Fri Jul 29 17:08:49 2022
Check interval: 0 (<none>)
Lifetime writes: 34 GB
Reserved blocks uid: 0 (user root)
Reserved blocks gid: 0 (group root)
First inode: 11
Inode size: 256
Required extra isize: 32
Desired extra isize: 32
Journal inode: 8
Default directory hash: half_md4
Directory Hash Seed: 1009f32c-cdf8-4a01-8dbc-5aaf74bf0132
Journal backup: inode blocks
FS Error count: 3
First error time: Mon Aug 1 17:17:01 2022
First error function: htree_dirblock_to_tree
First error line #: 1030
First error inode #: 46
First error block #: 6277
Last error time: Mon Aug 1 19:17:01 2022
Last error function: htree_dirblock_to_tree
Last error line #: 1030
Last error inode #: 46
Last error block #: 6277
Journal features: journal_incompat_revoke
Journal size: 32M
Journal length: 8192
Journal sequence: 0x0001b2b5
Journal start: 2065
As you can see, Maximum mount count is -1, which indicates fsck at mount is completely disable. I was surprise to notice the root is also mounted as RW instead of RO, since raspberry pi's are famous for corrupting RW sd cards when they are power down unexpectedly. (which occurred more than once for us)
I'm also running into stuck problems rebooting the tinypilot, which is a huge issue since I'm using it remotely, so I can't physically turn it off/on again. I guess those rebooting getting stuck is related to the "broken" filesystem.
Right now It's running, but I'm afraid of enabling check at mount using tune2fs, and bricking it if it needs manual intervention to fix the filesystem. It will essentially brick the tinypilot since there's no one at the office that can do it manually.
Is there any way that I can fix the root filesystem without running into a "brick state" where it will go into maintenance mode at boot if the filesystem check needs manual intervention and can't be fixed?
Michael Lynch @michael2022-08-01 21:18:36.097ZThanks for reporting this, Roberto! Sorry to hear about the filesystem corruption.
We're looking into microSD alternatives, but changing hardware right now is difficult due to the chip crisis.
I was surprise to notice the root is also mounted as RW instead of RO, since raspberry pi's are famous for corrupting RW sd cards when they are power down unexpectedly. (which occurred more than once for us)
We ship them RW by default because it otherwise causes a lot of user confusion to have changes disappear on reboot, but it is possible to enable the overlay filesystem, which avoids writes to the physical microSD. Of course that doesn't help you in this case, as the filesystem is already corrupted.
We're going to investigate options for fixing the root filesystem and follow up.
As an alternative, is it possible for someone with physical access to the TinyPilot to reflash the microSD? That's the lowest-risk option, though I recognize it is inconvenient.
- RRoberto Hradec @rhradec
I spoke with one person at the office in Spain, and he should be back at work next week. So he will actually take the sdcard from the tinypilot and plug into a linux pc I use and I can do the fsck there.
How can I enable the overlay, btw? I think that would be a perfect solution for our case, since I rely on smart plugs to cold reset computers at the office, and that's how we do with the tinypilot too.
Diego @diegoHello @rhradec - To enable the overlay filesystem, please see our FAQ: How do I enable the read-only filesystem?