Resolve "investigate slow startup of bec_lib"
Description
Thanks to profiling of module import time, it was determined which modules were taking time when doing import bec_lib
.
The strategy to reduce import time is to replace the module with a placeholder, which will become the real module at the moment it is really used ("proxy" pattern applied to do lazy module loading). Like this, no change is done in the logic of the code and objects are still imported at top of the file etc. Of course, the import call itself changes to use the lazy import feature.
Changes:
- new
proxy.py
module withProxy
class- code taken from BLISS (which was already taken from ActiveState recipe), this allows to have a proxy for any Python object
- new
lazy_import
andlazy_import_from
functions- substitute real import with a proxy object, calls
importlib.import_module
only when needed
- substitute real import with a proxy object, calls
- reorganization of
bec_lib.utils
as a set of modules within a directory instead of a single file - tracked what was taking time, and replaced with lazy imports
- some little changes to prevent early init of the proxies with real objects
- 3 bigger changes:
- all modules loaded in
bec_lib.__init__.py
have been cleared exceptbec_logger
(because it makes kind of sense? It does not increase startup time thanks for other changes inbec_lib.logger
) -
threadlocked
decorator usage is removed (not the decorator itself, but now it is for nothing hopefully...) -
logger.__init__
not longer set level, this is done inconfigure()
(otherwise IPython is called since the beginning)
- all modules loaded in
I decided to keep numpy
import because it does not add too much overhead and I didn't want to patch many modules with a numpy proxy just for a few milliseconds.
Result: bec
IPython client startup passes from 1.5 second to 0.6
Closes #283 (closed)