GPflow has a config file,
gpflowrc which allows the user to change
the default behavious in GPflow. GPflow searches for the file in the
following order: 1. In the working directory 2. In the user’s home
directory 3. In the GPflow directory (revert to default)
You can also make
gpflowrc a hidden file, if you don’t want it
clutting your home directory, by renaming as
By default, the configuration looks like this:
[verbosity] tf_compile_verb = False hmc_verb = True optimisation_verb = False [dtypes] float_type = float64 int_type = int32 [numerics] jitter_level = 1e-6 [profiling] dump_timeline = False dump_tensorboard = False [session] intra_op_parallelism_threads = 0 inter_op_parallelism_threads = 0
You can access the settings as
gpflow.settings, and the different
options are nested under the headings in the file. For example, to see
how much jitter is added before attempting Cholesky decomposition:
import gpflow print(gpflow.settings.jitter)
Settings can be modified for an entire session, or for a limited set of statements, using a Python context manager. It is recommeded to use the context manager, as this prevents the change of state unintentionally spilling into other parts of the program.
By default, verbose compiling is switched off:
import numpy as np import numpy.random as rnd X = rnd.randn(100, 1) Y = np.sin(X) + np.sin(1.5*X) + 0.3 * rnd.randn(*X.shape) m = gpflow.models.SGPR(X, Y, gpflow.kernels.RBF(1), Z=X.copy())
This can be switched on. First make a copy of the current settings using
get_settings(), then modify and set using the context manager
temp_settings. Finally, we see the compilation message showing up.
custom_config = gpflow.settings.get_settings() custom_config.verbosity.tf_compile_verb = True opt = gpflow.train.ScipyOptimizer() with gpflow.settings.temp_settings(custom_config), gpflow.session_manager.get_session().as_default(): m = gpflow.models.SGPR(X, Y, gpflow.kernels.RBF(1), Z=X.copy()) opt.minimize(m)
INFO:tensorflow:Optimization terminated with: Message: b'CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH' Objective function value: 27.154970 Number of iterations: 14 Number of functions evaluations: 19
Change TensorFlow session settings¶
GPflow may create multiple tensorflow sessions for a single model; for example a separate session is created for each autoflow method. To control the session parameters change the [session] section of the settings. This section may contain any valid TensorFlow ConfigProto setting.
For instance to ensure all tensorflow graphs are run serially set
[session] intra_op_parallelism_threads = 1 inter_op_parallelism_threads = 1
As per the TensorFlow documentation, a setting of 0 means the system picks an appropriate number of cores to use.
It’s important to note that for some cases, a re-compilation of the model is necessary. For example, if we change the jitter level and optimise, the hyperparameters won’t change unless we explicitly recompile the model. Additionally, state defined inside the context manager will be carried over to outside the context manager, until the next recompile.
Essentially, to be safe, if a model is to be used inside a context manager, everything should be done within the context manager.
We first look at the kernel hyperparameters from the previous optimisation. Those inside the context manager will be the same, despite the drastically increased jitter.
custom_config.numerics.jitter_level = 10e-0 with gpflow.settings.temp_settings(custom_config): opt.minimize(m) print(m.kern.lengthscales.read_value())
INFO:tensorflow:Optimization terminated with: Message: b'CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH' Objective function value: 27.154962 Number of iterations: 8 Number of functions evaluations: 10 1.2315556419374536
When the model is re-compiled, the modified jitter is taken into account in the TensorFlow graph, and the resulting hyperparameters are very different.
m.clear() with gpflow.settings.temp_settings(custom_config), gpflow.session_manager.get_session().as_default(): m.compile() opt.minimize(m) print(m.kern.lengthscales.read_value())
INFO:tensorflow:Optimization terminated with: Message: b'CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH' Objective function value: 93.107377 Number of iterations: 167 Number of functions evaluations: 184 1.0021331815790675