Settings

GPflow has a config file, gpflowrc which allows the user to change the default behavious in GPflow. GPflow searches for the file in the following order: 1. In the working directory 2. In the user’s home directory 3. In the GPflow directory (revert to default)

You can also make gpflowrc a hidden file, if you don’t want it clutting your home directory, by renaming as .gpflowrc.

By default, the configuration looks like this:

[verbosity]
tf_compile_verb = False
hmc_verb = True
optimisation_verb = False

[dtypes]
float_type = float64
int_type = int32

[numerics]
jitter_level = 1e-6

[profiling]
dump_timeline = False
dump_tensorboard = False

[session]
intra_op_parallelism_threads = 0
inter_op_parallelism_threads = 0

Accessing settings

You can access the settings as gpflow.settings, and the different options are nested under the headings in the file. For example, to see how much jitter is added before attempting Cholesky decomposition:

In [2]:
import gpflow
print(gpflow.settings.jitter)
1e-06

Modifying settings

Settings can be modified for an entire session, or for a limited set of statements, using a Python context manager. It is recommeded to use the context manager, as this prevents the change of state unintentionally spilling into other parts of the program.

Example

By default, verbose compiling is switched off:

In [3]:
import numpy as np
import numpy.random as rnd
X = rnd.randn(100, 1)
Y = np.sin(X) + np.sin(1.5*X) + 0.3 * rnd.randn(*X.shape)

m = gpflow.models.SGPR(X, Y, gpflow.kernels.RBF(1), Z=X.copy())

This can be switched on. First make a copy of the current settings using get_settings(), then modify and set using the context manager temp_settings. Finally, we see the compilation message showing up.

In [5]:
custom_config = gpflow.settings.get_settings()
custom_config.verbosity.tf_compile_verb = True
opt = gpflow.train.ScipyOptimizer()
with gpflow.settings.temp_settings(custom_config), gpflow.session_manager.get_session().as_default():
    m = gpflow.models.SGPR(X, Y, gpflow.kernels.RBF(1), Z=X.copy())
    opt.minimize(m)
INFO:tensorflow:Optimization terminated with:
  Message: b'CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH'
  Objective function value: 27.154970
  Number of iterations: 14
  Number of functions evaluations: 19

Change TensorFlow session settings

GPflow may create multiple tensorflow sessions for a single model; for example a separate session is created for each autoflow method. To control the session parameters change the [session] section of the settings. This section may contain any valid TensorFlow ConfigProto setting.

For instance to ensure all tensorflow graphs are run serially set

[session]
intra_op_parallelism_threads = 1
inter_op_parallelism_threads = 1

As per the TensorFlow documentation, a setting of 0 means the system picks an appropriate number of cores to use.

Notes

It’s important to note that for some cases, a re-compilation of the model is necessary. For example, if we change the jitter level and optimise, the hyperparameters won’t change unless we explicitly recompile the model. Additionally, state defined inside the context manager will be carried over to outside the context manager, until the next recompile.

Essentially, to be safe, if a model is to be used inside a context manager, everything should be done within the context manager.

Example

We first look at the kernel hyperparameters from the previous optimisation. Those inside the context manager will be the same, despite the drastically increased jitter.

In [6]:
m.kern.lengthscales.read_value()
Out[6]:
array(1.231800598892151)
In [7]:
custom_config.numerics.jitter_level = 10e-0
with gpflow.settings.temp_settings(custom_config):
    opt.minimize(m)
    print(m.kern.lengthscales.read_value())
INFO:tensorflow:Optimization terminated with:
  Message: b'CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH'
  Objective function value: 27.154962
  Number of iterations: 8
  Number of functions evaluations: 10
1.2315556419374536

When the model is re-compiled, the modified jitter is taken into account in the TensorFlow graph, and the resulting hyperparameters are very different.

In [9]:
m.clear()
with gpflow.settings.temp_settings(custom_config), gpflow.session_manager.get_session().as_default():
    m.compile()
    opt.minimize(m)
    print(m.kern.lengthscales.read_value())
INFO:tensorflow:Optimization terminated with:
  Message: b'CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH'
  Objective function value: 93.107377
  Number of iterations: 167
  Number of functions evaluations: 184
1.0021331815790675