OS-8150: lx should not report unlimited for max open files

Details

Issue Type:Bug
Priority:4 - Normal
Status:Resolved
Created at:2020-03-31T19:09:01.660Z
Updated at:2020-03-31T22:41:46.708Z

People

Created by:Michael Zeller
Reported by:Michael Zeller
Assigned to:Michael Zeller

Resolution

Fixed: A fix for this issue is checked into the tree and tested.
(Resolution Date: 2020-03-31T22:41:46.700Z)

Fix Versions

2020-04-09 Prognosis Negative (Release Date: 2020-04-09)

Related Links

Description

procfs(5) states the following:
       /proc/sys/fs/nr_open (since Linux 2.6.25)
              This file imposes ceiling on the value to which the RLIMIT_NOFILE resource limit can be
              raised  (see  getrlimit(2)).  This ceiling is enforced for both unprivileged and privi‐
              leged process.  The default value in this file is 1048576.  (Before Linux  2.6.25,  the
              ceiling for RLIMIT_NOFILE was hard-coded to the same value.)

On lx we see the following:
link@plex:~$ ulimit -Sn 10 && grep "Max open files" /proc/$$/limits
Max open files            10                   1048576              files
link@plex:~$ ulimit -Sn unlimited && grep "Max open files" /proc/$$/limits
Max open files            unlimited            1048576              files

While on linux we see the following:
link - zebes ~ $ ulimit -Sn 10 && grep "Max open files" /proc/$$/limits
Max open files            10                   524288               files
link - zebes ~ $ ulimit -Sn unlimited && grep "Max open files" /proc/$$/limits
Max open files            524288               524288               files

Note that the lx source currently returns "unlimited" when the max-file-descriptor "basic" field is missing.

The issue first appeared when trying to run gethue/hue:latest as a triton docker container:
arch@9d46dd2d-1178-ce83-cbc4-d396e4a24060 ~ ❯❯❯ triton-docker run -d -m=4G --name=bruce-hue -it -p 8888:8888 gethue/hue:latest                                                                                                         ✘ 130


D[31/Mar/2020 14:36:02 ] settings     INFO     Welcome to Hue 4.6.0
Traceback (most recent call last):
  File "/usr/share/hue/build/env/bin/hue", line 11, in <module>
    load_entry_point('desktop', 'console_scripts', 'hue')()
  File "/usr/share/hue/desktop/core/src/desktop/manage_entry.py", line 225, in entry
    execute_from_command_line(sys.argv)
  File "/usr/share/hue/build/env/local/lib/python2.7/site-packages/Django-1.11.22-py2.7.egg/django/core/management/__init__.py", line 364, in execute_from_command_line
    utility.execute()
  File "/usr/share/hue/build/env/local/lib/python2.7/site-packages/Django-1.11.22-py2.7.egg/django/core/management/__init__.py", line 338, in execute
    django.setup()
_  File "/usr/share/hue/build/env/local/lib/python2.7/site-packages/Django-1.11.22-py2.7.egg/django/__init__.py", line 27, in setup
    apps.populate(settings.INSTALLED_APPS)
  File "/usr/share/hue/build/env/local/lib/python2.7/site-packages/Django-1.11.22-py2.7.egg/django/apps/registry.py", line 85, in populate
    app_config = AppConfig.create(entry)
  File "/usr/share/hue/build/env/local/lib/python2.7/site-packages/Django-1.11.22-py2.7.egg/django/apps/config.py", line 94, in create
    module = import_module(entry)
  File "/usr/lib/python2.7/importlib/__init__.py", line 37, in import_module
G    __import__(name)
  File "/usr/share/hue/build/env/local/lib/python2.7/site-packages/django_prometheus-1.0.15-py2.7.egg/django_prometheus/__init__.py", line 9, in <module>
    import django_prometheus.middleware
  File "/usr/share/hue/build/env/local/lib/python2.7/site-packages/django_prometheus-1.0.15-py2.7.egg/django_prometheus/middleware.py", line 1, in <module>
    from prometheus_client import Counter, Histogram
  File "/usr/share/hue/build/env/local/lib/python2.7/site-packages/prometheus_client-0.7.1-py2.7.egg/prometheus_client/__init__.py", line 8, in <module>
Q    from . import process_collector
  File "/usr/share/hue/build/env/local/lib/python2.7/site-packages/prometheus_client-0.7.1-py2.7.egg/prometheus_client/process_collector.py", line 97, in <module>
    PROCESS_COLLECTOR = ProcessCollector()
  File "/usr/share/hue/build/env/local/lib/python2.7/site-packages/prometheus_client-0.7.1-py2.7.egg/prometheus_client/process_collector.py", line 43, in __init__
    registry.register(self)
  File "/usr/share/hue/build/env/local/lib/python2.7/site-packages/prometheus_client-0.7.1-py2.7.egg/prometheus_client/registry.py", line 24, in register
    names = self._get_names(collector)
  File "/usr/share/hue/build/env/local/lib/python2.7/site-packages/prometheus_client-0.7.1-py2.7.egg/prometheus_client/registry.py", line 64, in _get_names
    for metric in desc_func():
  File "/usr/share/hue/build/env/local/lib/python2.7/site-packages/prometheus_client-0.7.1-py2.7.egg/prometheus_client/process_collector.py", line 85, in collect
    value=float(line.split()[3]))
ValueError: could not convert string to float: unlimited
l[31/Mar/2020 14:36:02 ] supervisor   WARNING  Exit code for /usr/share/hue/build/env/bin/hue kt_renewer: 1
[31/Mar/2020 14:36:02 ] supervisor   ERROR    Process /usr/share/hue/build/env/bin/hue kt_renewer has restarted more than 3 times in the last 4 seconds
I[31/Mar/2020 14:36:02 ] supervisor   WARNING  Supervisor shutting down!
][31/Mar/2020 14:36:02 ] supervisor   WARNING  Waiting for children to exit for 5 seconds...
eTraceback (most recent call last):
  File "/usr/share/hue/build/env/bin/hue", line 11, in <module>
    load_entry_point('desktop', 'console_scripts', 'hue')()
  File "/usr/share/hue/desktop/core/src/desktop/manage_entry.py", line 225, in entry
    execute_from_command_line(sys.argv)
  File "/usr/share/hue/build/env/local/lib/python2.7/site-packages/Django-1.11.22-py2.7.egg/django/core/management/__init__.py", line 364, in execute_from_command_line
    utility.execute()
  File "/usr/share/hue/build/env/local/lib/python2.7/site-packages/Django-1.11.22-py2.7.egg/django/core/management/__init__.py", line 308, in execute
    settings.INSTALLED_APPS
  File "/usr/share/hue/build/env/local/lib/python2.7/site-packages/Django-1.11.22-py2.7.egg/django/conf/__init__.py", line 56, in __getattr__
]    self._setup(name)
  File "/usr/share/hue/build/env/local/lib/python2.7/site-packages/Django-1.11.22-py2.7.egg/django/conf/__init__.py", line 41, in _setup
    self._wrapped = Settings(settings_module)
  File "/usr/share/hue/build/env/local/lib/python2.7/site-packages/Django-1.11.22-py2.7.egg/django/conf/__init__.py", line 110, in __init__
    mod = importlib.import_module(self.SETTINGS_MODULE)
  File "/usr/lib/python2.7/importlib/__init__.py", line 37, in import_module
    __import__(name)
  File "/usr/share/hue/desktop/core/src/desktop/settings.py", line 483, in <module>
    EMAIL_HOST_PASSWORD = desktop.conf.get_smtp_password()
  File "/usr/share/hue/desktop/core/src/desktop/conf.py", line 2215, in get_smtp_password
{    password = SMTP.PASSWORD_SCRIPT.get()
  File "/usr/share/hue/desktop/core/src/desktop/lib/conf.py", line 159, in get
    return self.config.get_value(data, present=present, prefix=self.prefix, coerce_type=True)
  File "/usr/share/hue/desktop/core/src/desktop/lib/conf.py", line 273, in get_value

    return self._coerce_type(raw_val, prefix)
  File "/usr/share/hue/desktop/core/src/desktop/lib/conf.py", line 293, in _coerce_type
    return self.type(raw)
  File "/usr/share/hue/desktop/core/src/desktop/lib/conf.py", line 718, in coerce_password_from_script
k    stdout, stderr = p.communicate()
  File "/usr/lib/python2.7/subprocess.py", line 480, in communicate
P    self.wait()
  File "/usr/lib/python2.7/subprocess.py", line 1099, in wait
    pid, sts = _eintr_retry_call(os.waitpid, self.pid, 0)
  File "/usr/lib/python2.7/subprocess.py", line 125, in _eintr_retry_call
+    return func(*args)
KeyboardInterrupt
m[31/Mar/2020 14:36:02 ] supervisor   WARNING  Exit code for /usr/share/hue/build/env/bin/hue runcpserver: 1
[31/Mar/2020 14:36:02 ] supervisor   INFO     Stopping /usr/share/hue/build/env/bin/hue runcpserver because supervisor exiting
[31/Mar/2020 14:36:02 ] supervisor   ERROR    Exception in supervisor main loop
Traceback (most recent call last):
  File "/usr/share/hue/desktop/core/src/desktop/supervisor.py", line 379, in main
    wait_loop(sups, options)
  File "/usr/share/hue/desktop/core/src/desktop/supervisor.py", line 396, in wait_loop
    shutdown(sups)  # shutdown() exits the process
  File "/usr/share/hue/desktop/core/src/desktop/supervisor.py", line 218, in shutdown
    sys.exit(1)
SystemExit: 1
I[31/Mar/2020 14:36:02 ] supervisor   WARNING  Supervisor shutting down!
][31/Mar/2020 14:36:02 ] supervisor   WARNING  Waiting for children to exit for 5 seconds...
^C
arch@9d46dd2d-1178-ce83-cbc4-d396e4a24060 ~ ❯❯❯ triton-docker run -d -m=4G --name=bruce-hue -it -p 8888:8888 gethue/hue:latest                                                                                                         ✘ 130
arch@9d46dd2d-1178-ce83-cbc4-d396e4a24060 ~ ❯❯❯                                                                                                                                                                                        ✘ 130
arch@9d46dd2d-1178-ce83-cbc4-d396e4a24060 ~ ❯❯❯                                                                                                                                                                                        ✘ 130
arch@9d46dd2d-1178-ce83-cbc4-d396e4a24060 ~ ❯❯❯ td logs --follow bruce-hue                                                                                                                                                             ✘ 130
arch@9d46dd2d-1178-ce83-cbc4-d396e4a24060 ~ ❯❯❯                                                                                                                                                                                        ✘ 130
arch@9d46dd2d-1178-ce83-cbc4-d396e4a24060 ~ ❯❯❯ triton-docker run -d -m=4G --name=bruce-hue -it -p 8888:8888 gethue/hue:latest                                                                                                         ✘ 130
arch@9d46dd2d-1178-ce83-cbc4-d396e4a24060 ~ ❯❯❯ td logs --follow bruce-hue                                                                                                                                                             ✘ 130
k    stdout, stderr = p.communicate()
  File "/usr/lib/python2.7/subprocess.py", line 480, in communicate
P    self.wait()
  File "/usr/lib/python2.7/subprocess.py", line 1099, in wait
    pid, sts = _eintr_retry_call(os.waitpid, self.pid, 0)
  File "/usr/lib/python2.7/subprocess.py", line 125, in _eintr_retry_call
+    return func(*args)
KeyboardInterrupt
m[31/Mar/2020 14:36:02 ] supervisor   WARNING  Exit code for /usr/share/hue/build/env/bin/hue runcpserver: 1
[31/Mar/2020 14:36:02 ] supervisor   INFO     Stopping /usr/share/hue/build/env/bin/hue runcpserver because supervisor exiting
[31/Mar/2020 14:36:02 ] supervisor   ERROR    Exception in supervisor main loop
Traceback (most recent call last):
  File "/usr/share/hue/desktop/core/src/desktop/supervisor.py", line 379, in main
    wait_loop(sups, options)
  File "/usr/share/hue/desktop/core/src/desktop/supervisor.py", line 396, in wait_loop
    shutdown(sups)  # shutdown() exits the process
  File "/usr/share/hue/desktop/core/src/desktop/supervisor.py", line 218, in shutdown
    sys.exit(1)
SystemExit: 1
I[31/Mar/2020 14:36:02 ] supervisor   WARNING  Supervisor shutting down!
][31/Mar/2020 14:36:02 ] supervisor   WARNING  Waiting for children to exit for 5 seconds...

D[31/Mar/2020 14:40:18 ] settings     INFO     Welcome to Hue 4.6.0
Traceback (most recent call last):
  File "./build/env/bin/hue", line 11, in <module>
    load_entry_point('desktop', 'console_scripts', 'hue')()
  File "/usr/share/hue/desktop/core/src/desktop/manage_entry.py", line 225, in entry
    execute_from_command_line(sys.argv)
  File "/usr/share/hue/build/env/local/lib/python2.7/site-packages/Django-1.11.22-py2.7.egg/django/core/management/__init__.py", line 364, in execute_from_command_line
    utility.execute()
  File "/usr/share/hue/build/env/local/lib/python2.7/site-packages/Django-1.11.22-py2.7.egg/django/core/management/__init__.py", line 338, in execute
%    django.setup()
  File "/usr/share/hue/build/env/local/lib/python2.7/site-packages/Django-1.11.22-py2.7.egg/django/__init__.py", line 27, in setup
    apps.populate(settings.INSTALLED_APPS)
  File "/usr/share/hue/build/env/local/lib/python2.7/site-packages/Django-1.11.22-py2.7.egg/django/apps/registry.py", line 85, in populate
    app_config = AppConfig.create(entry)
  File "/usr/share/hue/build/env/local/lib/python2.7/site-packages/Django-1.11.22-py2.7.egg/django/apps/config.py", line 94, in create
    module = import_module(entry)
  File "/usr/lib/python2.7/importlib/__init__.py", line 37, in import_module
    __import__(name)
  File "/usr/share/hue/build/env/local/lib/python2.7/site-packages/django_prometheus-1.0.15-py2.7.egg/django_prometheus/__init__.py", line 9, in <module>
    import django_prometheus.middleware
  File "/usr/share/hue/build/env/local/lib/python2.7/site-packages/django_prometheus-1.0.15-py2.7.egg/django_prometheus/middleware.py", line 1, in <module>
    from prometheus_client import Counter, Histogram
  File "/usr/share/hue/build/env/local/lib/python2.7/site-packages/prometheus_client-0.7.1-py2.7.egg/prometheus_client/__init__.py", line 8, in <module>
Q    from . import process_collector
  File "/usr/share/hue/build/env/local/lib/python2.7/site-packages/prometheus_client-0.7.1-py2.7.egg/prometheus_client/process_collector.py", line 97, in <module>
    PROCESS_COLLECTOR = ProcessCollector()
  File "/usr/share/hue/build/env/local/lib/python2.7/site-packages/prometheus_client-0.7.1-py2.7.egg/prometheus_client/process_collector.py", line 43, in __init__
    registry.register(self)
  File "/usr/share/hue/build/env/local/lib/python2.7/site-packages/prometheus_client-0.7.1-py2.7.egg/prometheus_client/registry.py", line 24, in register
    names = self._get_names(collector)
  File "/usr/share/hue/build/env/local/lib/python2.7/site-packages/prometheus_client-0.7.1-py2.7.egg/prometheus_client/registry.py", line 64, in _get_names
    for metric in desc_func():
  File "/usr/share/hue/build/env/local/lib/python2.7/site-packages/prometheus_client-0.7.1-py2.7.egg/prometheus_client/process_collector.py", line 85, in collect
    value=float(line.split()[3]))
ValueError: could not convert string to float: unlimited
l[31/Mar/2020 14:40:19 ] supervisor   INFO     Starting process /usr/share/hue/build/env/bin/hue kt_renewer
m[31/Mar/2020 14:40:19 ] supervisor   INFO     Starting process /usr/share/hue/build/env/bin/hue runcpserver
y[31/Mar/2020 14:40:19 ] supervisor   INFO     Started proceses (pid 591271) /usr/share/hue/build/env/bin/hue kt_renewer
z[31/Mar/2020 14:40:19 ] supervisor   INFO     Started proceses (pid 591272) /usr/share/hue/build/env/bin/hue runcpserver
D[31/Mar/2020 14:40:20 ] settings     INFO     Welcome to Hue 4.6.0
D[31/Mar/2020 14:40:20 ] settings     INFO     Welcome to Hue 4.6.0
eTraceback (most recent call last):
  File "/usr/share/hue/build/env/bin/hue", line 11, in <module>
e    load_entry_point('desktop', 'console_scripts', 'hue')()
  File "/usr/share/hue/desktop/core/src/desktop/manage_entry.py", line 225, in entry
    execute_from_command_line(sys.argv)
  File "/usr/share/hue/build/env/local/lib/python2.7/site-packages/Django-1.11.22-py2.7.egg/django/core/management/__init__.py", line 364, in execute_from_command_line
    utility.execute()
  File "/usr/share/hue/build/env/local/lib/python2.7/site-packages/Django-1.11.22-py2.7.egg/django/core/management/__init__.py", line 338, in execute
    django.setup()
  File "/usr/share/hue/build/env/local/lib/python2.7/site-packages/Django-1.11.22-py2.7.egg/django/__init__.py", line 27, in setup
    apps.populate(settings.INSTALLED_APPS)
  File "/usr/share/hue/build/env/local/lib/python2.7/site-packages/Django-1.11.22-py2.7.egg/django/apps/registry.py", line 85, in populate
    app_config = AppConfig.create(entry)
  File "/usr/share/hue/build/env/local/lib/python2.7/site-packages/Django-1.11.22-py2.7.egg/django/apps/config.py", line 94, in create
    module = import_module(entry)
  File "/usr/lib/python2.7/importlib/__init__.py", line 37, in import_module
    __import__(name)
  File "/usr/share/hue/build/env/local/lib/python2.7/site-packages/django_prometheus-1.0.15-py2.7.egg/django_prometheus/__init__.py", line 9, in <module>
    import django_prometheus.middleware
  File "/usr/share/hue/build/env/local/lib/python2.7/site-packages/django_prometheus-1.0.15-py2.7.egg/django_prometheus/middleware.py", line 1, in <module>
    from prometheus_client import Counter, Histogram
  File "/usr/share/hue/build/env/local/lib/python2.7/site-packages/prometheus_client-0.7.1-py2.7.egg/prometheus_client/__init__.py", line 8, in <module>
    from . import process_collector
  File "/usr/share/hue/build/env/local/lib/python2.7/site-packages/prometheus_client-0.7.1-py2.7.egg/prometheus_client/process_collector.py", line 97, in <module>
    PROCESS_COLLECTOR = ProcessCollector()
  File "/usr/share/hue/build/env/local/lib/python2.7/site-packages/prometheus_client-0.7.1-py2.7.egg/prometheus_client/process_collector.py", line 43, in __init__
}    registry.register(self)
  File "/usr/share/hue/build/env/local/lib/python2.7/site-packages/prometheus_client-0.7.1-py2.7.egg/prometheus_client/registry.py", line 24, in register
    names = self._get_names(collector)
  File "/usr/share/hue/build/env/local/lib/python2.7/site-packages/prometheus_client-0.7.1-py2.7.egg/prometheus_client/registry.py", line 64, in _get_names
    for metric in desc_func():
  File "/usr/share/hue/build/env/local/lib/python2.7/site-packages/prometheus_client-0.7.1-py2.7.egg/prometheus_client/process_collector.py", line 85, in collect
    Yvalue=float(line.split()[3]))
ValueError: could not convert string to float: unlimited
eTraceback (most recent call last):
  File "/usr/share/hue/build/env/bin/hue", line 11, in <module>
e    load_entry_point('desktop', 'console_scripts', 'hue')()
  File "/usr/share/hue/desktop/core/src/desktop/manage_entry.py", line 225, in entry
    execute_from_command_line(sys.argv)
  File "/usr/share/hue/build/env/local/lib/python2.7/site-packages/Django-1.11.22-py2.7.egg/django/core/management/__init__.py", line 364, in execute_from_command_line
    utility.execute()
  File "/usr/share/hue/build/env/local/lib/python2.7/site-packages/Django-1.11.22-py2.7.egg/django/core/management/__init__.py", line 338, in execute
    django.setup()
  File "/usr/share/hue/build/env/local/lib/python2.7/site-packages/Django-1.11.22-py2.7.egg/django/__init__.py", line 27, in setup
    apps.populate(settings.INSTALLED_APPS)
  File "/usr/share/hue/build/env/local/lib/python2.7/site-packages/Django-1.11.22-py2.7.egg/django/apps/registry.py", line 85, in populate
    app_config = AppConfig.create(entry)
  File "/usr/share/hue/build/env/local/lib/python2.7/site-packages/Django-1.11.22-py2.7.egg/django/apps/config.py", line 94, in create
"    module = import_module(entry)
  File "/usr/lib/python2.7/importlib/__init__.py", line 37, in import_module
    __import__(name)
  File "/usr/share/hue/build/env/local/lib/python2.7/site-packages/django_prometheus-1.0.15-py2.7.egg/django_prometheus/__init__.py", line 9, in <module>
    import django_prometheus.middleware
  File "/usr/share/hue/build/env/local/lib/python2.7/site-packages/django_prometheus-1.0.15-py2.7.egg/django_prometheus/middleware.py", line 1, in <module>
    from prometheus_client import Counter, Histogram
  File "/usr/share/hue/build/env/local/lib/python2.7/site-packages/prometheus_client-0.7.1-py2.7.egg/prometheus_client/__init__.py", line 8, in <module>
%    from . import process_collector
  File "/usr/share/hue/build/env/local/lib/python2.7/site-packages/prometheus_client-0.7.1-py2.7.egg/prometheus_client/process_collector.py", line 97, in <module>
    PROCESS_COLLECTOR = ProcessCollector()
  File "/usr/share/hue/build/env/local/lib/python2.7/site-packages/prometheus_client-0.7.1-py2.7.egg/prometheus_client/process_collector.py", line 43, in __init__
}    registry.register(self)
  File "/usr/share/hue/build/env/local/lib/python2.7/site-packages/prometheus_client-0.7.1-py2.7.egg/prometheus_client/registry.py", line 24, in register
    names = self._get_names(collector)
  File "/usr/share/hue/build/env/local/lib/python2.7/site-packages/prometheus_client-0.7.1-py2.7.egg/prometheus_client/registry.py", line 64, in _get_names
    for metric in desc_func():
  File "/usr/share/hue/build/env/local/lib/python2.7/site-packages/prometheus_client-0.7.1-py2.7.egg/prometheus_client/process_collector.py", line 85, in collect
#    value=float(line.split()[3]))
:ValueError: could not convert string to float: unlimited
l[31/Mar/2020 14:40:20 ] supervisor   WARNING  Exit code for /usr/share/hue/build/env/bin/hue kt_renewer: 1
[31/Mar/2020 14:40:20 ] supervisor   ERROR    Process /usr/share/hue/build/env/bin/hue kt_renewer exited abnormally. Restarting it.
l[31/Mar/2020 14:40:20 ] supervisor   INFO     Starting process /usr/share/hue/build/env/bin/hue kt_renewer
m[31/Mar/2020 14:40:20 ] supervisor   WARNING  Exit code for /usr/share/hue/build/env/bin/hue runcpserver: 1
[31/Mar/2020 14:40:20 ] supervisor   ERROR    Process /usr/share/hue/build/env/bin/hue runcpserver exited abnormally. Restarting it.
m[31/Mar/2020 14:40:20 ] supervisor   INFO     Starting process /usr/share/hue/build/env/bin/hue runcpserver
y[31/Mar/2020 14:40:20 ] supervisor   INFO     Started proceses (pid 591288) /usr/share/hue/build/env/bin/hue kt_renewer
z[31/Mar/2020 14:40:20 ] supervisor   INFO     Started proceses (pid 591289) /usr/share/hue/build/env/bin/hue runcpserver
D[31/Mar/2020 14:40:21 ] settings     INFO     Welcome to Hue 4.6.0
D[31/Mar/2020 14:40:21 ] settings     INFO     Welcome to Hue 4.6.0
Traceback (most recent call last):
  File "/usr/share/hue/build/env/bin/hue", line 11, in <module>
    load_entry_point('desktop', 'console_scripts', 'hue')()
  File "/usr/share/hue/desktop/core/src/desktop/manage_entry.py", line 225, in entry
    execute_from_command_line(sys.argv)
  File "/usr/share/hue/build/env/local/lib/python2.7/site-packages/Django-1.11.22-py2.7.egg/django/core/management/__init__.py", line 364, in execute_from_command_line
    utility.execute()
  File "/usr/share/hue/build/env/local/lib/python2.7/site-packages/Django-1.11.22-py2.7.egg/django/core/management/__init__.py", line 338, in execute
    django.setup()
  File "/usr/share/hue/build/env/local/lib/python2.7/site-packages/Django-1.11.22-py2.7.egg/django/__init__.py", line 27, in setup
    apps.populate(settings.INSTALLED_APPS)
  File "/usr/share/hue/build/env/local/lib/python2.7/site-packages/Django-1.11.22-py2.7.egg/django/apps/registry.py", line 85, in populate
    app_config = AppConfig.create(entry)
  File "/usr/share/hue/build/env/local/lib/python2.7/site-packages/Django-1.11.22-py2.7.egg/django/apps/config.py", line 94, in create
    module = import_module(entry)
  File "/usr/lib/python2.7/importlib/__init__.py", line 37, in import_module
    __import__(name)
  File "/usr/share/hue/build/env/local/lib/python2.7/site-packages/django_prometheus-1.0.15-py2.7.egg/django_prometheus/__init__.py", line 9, in <module>
_    import django_prometheus.middleware
  File "/usr/share/hue/build/env/local/lib/python2.7/site-packages/django_prometheus-1.0.15-py2.7.egg/django_prometheus/middleware.py", line 1, in <module>
    from prometheus_client import Counter, Histogram
  File "/usr/share/hue/build/env/local/lib/python2.7/site-packages/prometheus_client-0.7.1-py2.7.egg/prometheus_client/__init__.py", line 8, in <module>
    from . import process_collector
  File "/usr/share/hue/build/env/local/lib/python2.7/site-packages/prometheus_client-0.7.1-py2.7.egg/prometheus_client/process_collector.py", line 97, in <module>
M    PROCESS_COLLECTOR = ProcessCollector()
  File "/usr/share/hue/build/env/local/lib/python2.7/site-packages/prometheus_client-0.7.1-py2.7.egg/prometheus_client/process_collector.py", line 43, in __init__
    registry.register(self)
  File "/usr/share/hue/build/env/local/lib/python2.7/site-packages/prometheus_client-0.7.1-py2.7.egg/prometheus_client/registry.py", line 24, in register
    names = self._get_names(collector)
  File "/usr/share/hue/build/env/local/lib/python2.7/site-packages/prometheus_client-0.7.1-py2.7.egg/prometheus_client/registry.py", line 64, in _get_names
     for metric in desc_func():
  File "/usr/share/hue/build/env/local/lib/python2.7/site-packages/prometheus_client-0.7.1-py2.7.egg/prometheus_client/process_collector.py", line 85, in collect
    value=float(line.split()[3]))
ValueError: could not convert string to float: unlimited
Traceback (most recent call last):
  File "/usr/share/hue/build/env/bin/hue", line 11, in <module>
    load_entry_point('desktop', 'console_scripts', 'hue')()
  File "/usr/share/hue/desktop/core/src/desktop/manage_entry.py", line 225, in entry
    execute_from_command_line(sys.argv)
  File "/usr/share/hue/build/env/local/lib/python2.7/site-packages/Django-1.11.22-py2.7.egg/django/core/management/__init__.py", line 364, in execute_from_command_line
    utility.execute()
  File "/usr/share/hue/build/env/local/lib/python2.7/site-packages/Django-1.11.22-py2.7.egg/django/core/management/__init__.py", line 338, in execute
    django.setup()
  File "/usr/share/hue/build/env/local/lib/python2.7/site-packages/Django-1.11.22-py2.7.egg/django/__init__.py", line 27, in setup
    apps.populate(settings.INSTALLED_APPS)
  File "/usr/share/hue/build/env/local/lib/python2.7/site-packages/Django-1.11.22-py2.7.egg/django/apps/registry.py", line 85, in populate
    app_config = AppConfig.create(entry)
  File "/usr/share/hue/build/env/local/lib/python2.7/site-packages/Django-1.11.22-py2.7.egg/django/apps/config.py", line 94, in create
    module = import_module(entry)
  File "/usr/lib/python2.7/importlib/__init__.py", line 37, in import_module
    __import__(name)
  File "/usr/share/hue/build/env/local/lib/python2.7/site-packages/django_prometheus-1.0.15-py2.7.egg/django_prometheus/__init__.py", line 9, in <module>
    import django_prometheus.middleware
  File "/usr/share/hue/build/env/local/lib/python2.7/site-packages/django_prometheus-1.0.15-py2.7.egg/django_prometheus/middleware.py", line 1, in <module>
    from prometheus_client import Counter, Histogram
3  File "/usr/share/hue/build/env/local/lib/python2.7/site-packages/prometheus_client-0.7.1-py2.7.egg/prometheus_client/__init__.py", line 8, in <module>
    from . import process_collector
  File "/usr/share/hue/build/env/local/lib/python2.7/site-packages/prometheus_client-0.7.1-py2.7.egg/prometheus_client/process_collector.py", line 97, in <module>
    PROCESS_COLLECTOR = ProcessCollector()
  File "/usr/share/hue/build/env/local/lib/python2.7/site-packages/prometheus_client-0.7.1-py2.7.egg/prometheus_client/process_collector.py", line 43, in __init__
@    registry.register(self)
  File "/usr/share/hue/build/env/local/lib/python2.7/site-packages/prometheus_client-0.7.1-py2.7.egg/prometheus_client/registry.py", line 24, in register
    names = self._get_names(collector)
  File "/usr/share/hue/build/env/local/lib/python2.7/site-packages/prometheus_client-0.7.1-py2.7.egg/prometheus_client/registry.py", line 64, in _get_names
    for metric in desc_func():
  File "/usr/share/hue/build/env/local/lib/python2.7/site-packages/prometheus_client-0.7.1-py2.7.egg/prometheus_client/process_collector.py", line 85, in collect
]    value=float(line.split()[3]))
ValueError: could not convert string to float: unlimited
l[31/Mar/2020 14:40:22 ] supervisor   WARNING  Exit code for /usr/share/hue/build/env/bin/hue kt_renewer: 1
[31/Mar/2020 14:40:22 ] supervisor   ERROR    Process /usr/share/hue/build/env/bin/hue kt_renewer exited abnormally. Restarting it.
l[31/Mar/2020 14:40:22 ] supervisor   INFO     Starting process /usr/share/hue/build/env/bin/hue kt_renewer
m[31/Mar/2020 14:40:22 ] supervisor   WARNING  Exit code for /usr/share/hue/build/env/bin/hue runcpserver: 1
[31/Mar/2020 14:40:22 ] supervisor   ERROR    Process /usr/share/hue/build/env/bin/hue runcpserver exited abnormally. Restarting it.
m[31/Mar/2020 14:40:22 ] supervisor   INFO     Starting process /usr/share/hue/build/env/bin/hue runcpserver
y[31/Mar/2020 14:40:22 ] supervisor   INFO     Started proceses (pid 591304) /usr/share/hue/build/env/bin/hue kt_renewer
z[31/Mar/2020 14:40:22 ] supervisor   INFO     Started proceses (pid 591305) /usr/share/hue/build/env/bin/hue runcpserver
D[31/Mar/2020 14:40:23 ] settings     INFO     Welcome to Hue 4.6.0
D[31/Mar/2020 14:40:23 ] settings     INFO     Welcome to Hue 4.6.0
iTraceback (most recent call last):
  File "/usr/share/hue/build/env/bin/hue", line 11, in <module>
    load_entry_point('desktop', 'console_scripts', 'hue')()
  File "/usr/share/hue/desktop/core/src/desktop/manage_entry.py", line 225, in entry
    execute_from_command_line(sys.argv)
  File "/usr/share/hue/build/env/local/lib/python2.7/site-packages/Django-1.11.22-py2.7.egg/django/core/management/__init__.py", line 364, in execute_from_command_line
    utility.execute()
  File "/usr/share/hue/build/env/local/lib/python2.7/site-packages/Django-1.11.22-py2.7.egg/django/core/management/__init__.py", line 338, in execute
%    django.setup()
  File "/usr/share/hue/build/env/local/lib/python2.7/site-packages/Django-1.11.22-py2.7.egg/django/__init__.py", line 27, in setup
    apps.populate(settings.INSTALLED_APPS)
  File "/usr/share/hue/build/env/local/lib/python2.7/site-packages/Django-1.11.22-py2.7.egg/django/apps/registry.py", line 85, in populate
    app_config = AppConfig.create(entry)
  File "/usr/share/hue/build/env/local/lib/python2.7/site-packages/Django-1.11.22-py2.7.egg/django/apps/config.py", line 94, in create
    module = import_module(entry)
  File "/usr/lib/python2.7/importlib/__init__.py", line 37, in import_module
    __import__(name)
  File "/usr/share/hue/build/env/local/lib/python2.7/site-packages/django_prometheus-1.0.15-py2.7.egg/django_prometheus/__init__.py", line 9, in <module>
    import django_prometheus.middleware
  File "/usr/share/hue/build/env/local/lib/python2.7/site-packages/django_prometheus-1.0.15-py2.7.egg/django_prometheus/middleware.py", line 1, in <module>
    from prometheus_client import Counter, Histogram
  File "/usr/share/hue/build/env/local/lib/python2.7/site-packages/prometheus_client-0.7.1-py2.7.egg/prometheus_client/__init__.py", line 8, in <module>
Q    from . import process_collector
  File "/usr/share/hue/build/env/local/lib/python2.7/site-packages/prometheus_client-0.7.1-py2.7.egg/prometheus_client/process_collector.py", line 97, in <module>
    PROCESS_COLLECTOR = ProcessCollector()
  File "/usr/share/hue/build/env/local/lib/python2.7/site-packages/prometheus_client-0.7.1-py2.7.egg/prometheus_client/process_collector.py", line 43, in __init__
    registry.register(self)
  File "/usr/share/hue/build/env/local/lib/python2.7/site-packages/prometheus_client-0.7.1-py2.7.egg/prometheus_client/registry.py", line 24, in register
    names = self._get_names(collector)
  File "/usr/share/hue/build/env/local/lib/python2.7/site-packages/prometheus_client-0.7.1-py2.7.egg/prometheus_client/registry.py", line 64, in _get_names
    for metric in desc_func():
  File "/usr/share/hue/build/env/local/lib/python2.7/site-packages/prometheus_client-0.7.1-py2.7.egg/prometheus_client/process_collector.py", line 85, in collect
    value=float(line.split()[3]))
ValueError: could not convert string to float: unlimited
iTraceback (most recent call last):
  File "/usr/share/hue/build/env/bin/hue", line 11, in <module>
    aload_entry_point('desktop', 'console_scripts', 'hue')()
  File "/usr/share/hue/desktop/core/src/desktop/manage_entry.py", line 225, in entry
    execute_from_command_line(sys.argv)
  File "/usr/share/hue/build/env/local/lib/python2.7/site-packages/Django-1.11.22-py2.7.egg/django/core/management/__init__.py", line 364, in execute_from_command_line
    utility.execute()
  File "/usr/share/hue/build/env/local/lib/python2.7/site-packages/Django-1.11.22-py2.7.egg/django/core/management/__init__.py", line 338, in execute
P    django.setup()
  File "/usr/share/hue/build/env/local/lib/python2.7/site-packages/Django-1.11.22-py2.7.egg/django/__init__.py", line 27, in setup
    apps.populate(settings.INSTALLED_APPS)
  File "/usr/share/hue/build/env/local/lib/python2.7/site-packages/Django-1.11.22-py2.7.egg/django/apps/registry.py", line 85, in populate
#    app_config = AppConfig.create(entry)
  File "/usr/share/hue/build/env/local/lib/python2.7/site-packages/Django-1.11.22-py2.7.egg/django/apps/config.py", line 94, in create
    module = import_module(entry)
  File "/usr/lib/python2.7/importlib/__init__.py", line 37, in import_module
G    __import__(name)
  File "/usr/share/hue/build/env/local/lib/python2.7/site-packages/django_prometheus-1.0.15-py

Comments

Comment by Michael Zeller
Created at 2020-03-31T20:10:07.568Z

To test the change I booted a platform with the patch and ran the following to confirm "unlimited" wasn't being displayed for the softlimit

link@ubuntu:~$ ulimit -Sn 10 && grep "Max open files" /proc/$$/limits
Max open files 10 65536 files
link@ubuntu:~$ ulimit -Sn unlimited && grep "Max open files" /proc/$$/limits
Max open files 65536 65536 files

Comment by Michael Zeller
Created at 2020-03-31T22:23:42.477Z

I had the original reporter run the docker container again to ensure that the fix worked correctly:

CONTAINER ID        IMAGE               COMMAND             CREATED             STATUS              PORTS                    NAMES
2e94cab37b8d        gethue/hue:latest   "./startup.sh"      10 minutes ago      Up 10 minutes       0.0.0.0:8888->8888/tcp   bruce-hue

Comment by Jira Bot
Created at 2020-03-31T22:40:01.425Z

illumos-joyent commit 88606cbe3645730adccc57e29f4b829af51c05f4 (branch master, by Michael Zeller)

OS-8150 lx should not report unlimited for max open files (#279)

Reviewed by: Jason King <jason.king@joyent.com>
Approved by: Jason King <jason.king@joyent.com>