OS-8368: lstat in Ubuntu 20.04 LX zones reporting incorrect values


Issue Type:Bug
Priority:4 - Normal
Created at:2022-03-22T19:57:00.001Z
Updated at:2022-05-16T17:55:46.583Z


Created by:Dan McDonald
Reported by:Dan McDonald
Assigned to:Dan McDonald


From filer Ian Collins in illumos 14582:

I noticed that my build zones that use ccache were filling up despite there being a configured limit to the cache size. Despite the filesystem being full, ccache reporting was showing a much smaller size.

Ccache is using lstat to get the block count for the cache files (st_blocks) which is multiplied by 512 to approximate the size.

Picking a random cache file I see the following on a 16.04 zone:

# stat /home/.ccache/7/5/e446e54e52e11df06f2f588aae4c35-2994365.o
File: '/home/.ccache/7/5/e446e54e52e11df06f2f588aae4c35-2994365.o'
Size: 602072 Blocks: 1285 IO Block: 131072 regular file

which would give the size as 657920.

However, on a 20.04 zone:

# stat /home/.ccache/e/5/6a9c5k5jfcncsj97gmtmr4f6vc2vgeqR
File: /home/.ccache/e/5/6a9c5k5jfcncsj97gmtmr4f6vc2vgeqR
Size: 1130311 Blocks: 662 IO Block: 131072 regular file

which would give the size as 318464, about 28% of the actual size!

Changing the ccache code to just use st_size, still gives accurate reporting elsewhere but now over reports the space used by ~3.5 (1/0.28)

This also breaks du:

# ls -lh ccache
-rwxr-xr-x 1 root root 1.8M Mar 21 02:43 ccache

# du -sh ccache
971K ccache

So, it looks like lstat has a problem in Ubuntu 20.04 LX zones.


Comment by Dan McDonald
Created at 2022-03-22T20:21:28.667Z

A quick glance at the usr/src/uts/common/os/brand/lx/syscall/lx_stat.c file suggests nothing has changed here since 2016.

I don't know enough about the differences between Ubuntu 16 and Ubuntu 20, but I wonder if they are 32-vs-64 bit programs, and enter the lx_stat.c functions at different points (lx_lstat32 vs. lx_lstat64, e.g.).  Also, did du(1) perhaps change semantics between the two?  Does this problem exist in BHYVE or bare metal?  Also, does du(1) have an equivalent of the illumos "-A" which uses the same field (st_size) that ls(1) does?

Knowing those differences, if any, would help.

Comment by Dan McDonald
Created at 2022-04-19T20:56:44.712Z

I can’t seem to close this issue, but I think he figured out that actual vs. used disk space (the du -A in illumos) matters.

Comment by Dan McDonald
Created at 2022-05-16T17:55:46.583Z

Marking as closed, since du -A appears to be correct.