OS-3891: stale v_path slows vfs lookups


Issue Type:Bug
Priority:4 - Normal
Created at:2015-02-20T21:37:45.000Z
Updated at:2015-03-05T16:33:57.000Z


Created by:Former user
Reported by:Former user
Assigned to:Former user


Fixed: A fix for this issue is checked into the tree and tested.
(Resolution Date: 2015-03-05T16:33:57.000Z)

Fix Versions

2015-03-05 JavaRing (Release Date: 2015-03-05)


I was observing unusually slow (200ms/call) getcwd performance while perusing the illumos sources on my build box. (Vim was the canary in the coal mine, calling getcwd numerous times during initialization.)

Tracing dnlc_lookup activity yielded interesting results. Getcwd performed well when residing in the top of the sources tree with expected dnlc_lookup activity:

  4348 pwd              1   Y /zones/a1616d4f-c402-4060-c262-cccaa29d35cf/root
  4348 pwd              1   Y /zones/a1616d4f-c402-4060-c262-cccaa29d35cf/root/root
  4348 pwd              0   Y /zones/a1616d4f-c402-4060-c262-cccaa29d35cf/root/root/git
  4348 pwd              1   Y /zones/a1616d4f-c402-4060-c262-cccaa29d35cf/root/root/git/smartos-live
  4348 pwd              1   Y /zones/a1616d4f-c402-4060-c262-cccaa29d35cf/root/root/git/smartos-live/projects
  4348 pwd              1   Y /zones/a1616d4f-c402-4060-c262-cccaa29d35cf/root/root/git/smartos-live/projects/illumos

Going one level deeper, getcwd performance dropped significantly and the dnlc_lookup queries became strange:

  4373 pwd              1   Y /zones/a1616d4f-c402-4060-c262-cccaa29d35cf/root/root/git/smartos-live/projects
  4373 pwd              1   Y /zones/a1616d4f-c402-4060-c262-cccaa29d35cf/root/root/git/smartos-live/projects/other
  4373 pwd              1   Y /zones/a1616d4f-c402-4060-c262-cccaa29d35cf/root/zones
  4373 pwd              2   N /zones/a1616d4f-c402-4060-c262-cccaa29d35cf/root/root/git/smartos-live/projects/other/usr/..
  4373 pwd              2   Y /zones/a1616d4f-c402-4060-c262-cccaa29d35cf/root/zones
  4373 pwd              1   N /zones/a1616d4f-c402-4060-c262-cccaa29d35cf/root/root/git/smartos-live/projects/illumos/..

As it turns out, I had cloned the sources into a directory named 'other' during some maintenance, later renaming it to the standard 'illumos'. That 'other' name is persisting in the v_path of child vnodes since they are not touched by the rename operation at the top level.

Looking through the codepaths used by getcwd, it appears that both vnodetopath_common and lookuppnvp attempt to use v_path, but confirm its validity before doing so. Neither clear or correct v_path if it is found to be invalid, causing this poor performance to persist as long as those child vnodes are active.


Comment by Former user
Created at 2015-03-03T23:07:50.000Z

illumos-joyent commit 1635185 (branch master, by Patrick Mooney)

OS-3891 stale v_path slows vfs lookups
Reviewed by: Jerry Jelinek <jerry.jelinek@joyent.com>
Reviewed by: Bryan Cantrill <bryan@joyent.com>