Layers: keras_proba.layers¶
Uncertainty-aware Keras 3 layers. Every layer works with the TensorFlow, JAX and PyTorch backends and is serializable with model.save.
Distance-aware (SNGP)¶
RandomFourierFeatures ¶
RandomFourierFeatures(output_dim, kernel_initializer='gaussian', scale=None, trainable=False, name=None, **kwargs)
Bases: keras.src.layers.layer.Layer
Projects inputs into a random feature space approximating an RBF kernel.
Typically used to "kernelize" a linear model: a linear layer trained on top of these features is (up to approximation) a kernel machine, e.g. kernel logistic regression or the random feature Gaussian process used in SNGP.
Example:
model = keras.Sequential([
keras.Input(shape=(784,)),
RandomFourierFeatures(output_dim=4096, scale=10.0),
keras.layers.Dense(10, activation="softmax"),
])
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
output_dim
|
Positive integer, the number of random features |
required | |
kernel_initializer
|
|
'gaussian'
|
|
scale
|
Positive float, the kernel length scale. If |
None
|
|
trainable
|
Whether |
False
|
|
name
|
String, name of the layer. |
None
|
Input shape
(..., input_dim) with a defined last dimension. Unlike the reference,
inputs of rank greater than 2 are supported.
Output shape
(..., output_dim).
__doc__
class-attribute
¶
__doc__ = 'Projects inputs into a random feature space approximating an RBF kernel.\n\n Typically used to "kernelize" a linear model: a linear layer trained on top\n of these features is (up to approximation) a kernel machine, e.g. kernel\n logistic regression or the random feature Gaussian process used in SNGP.\n\n Example:\n\n ```python\n model = keras.Sequential([\n keras.Input(shape=(784,)),\n RandomFourierFeatures(output_dim=4096, scale=10.0),\n keras.layers.Dense(10, activation="softmax"),\n ])\n ```\n\n Args:\n output_dim: Positive integer, the number of random features `D`.\n kernel_initializer: `"gaussian"`, `"laplacian"` (case insensitive), or a\n Keras initializer for the random feature kernel `W`. The kernel is\n never trainable.\n scale: Positive float, the kernel length scale. If `None`, defaults to\n `sqrt(input_dim / 2)` for `"gaussian"` and `1.0` otherwise. If\n `trainable=True`, it is learned and this is its initial value.\n trainable: Whether `scale` is trainable. Defaults to `False`.\n name: String, name of the layer.\n\n Input shape:\n `(..., input_dim)` with a defined last dimension. Unlike the reference,\n inputs of rank greater than 2 are supported.\n\n Output shape:\n `(..., output_dim)`.\n '
str(object='') -> str str(bytes_or_buffer[, encoding[, errors]]) -> str
Create a new string object from the given object. If encoding or errors is specified, then the object must expose a data buffer that will be decoded using the given encoding and error handler. Otherwise, returns the result of object.str() (if defined) or repr(object). encoding defaults to sys.getdefaultencoding(). errors defaults to 'strict'.
__module__
class-attribute
¶
str(object='') -> str str(bytes_or_buffer[, encoding[, errors]]) -> str
Create a new string object from the given object. If encoding or errors is specified, then the object must expose a data buffer that will be decoded using the given encoding and error handler. Otherwise, returns the result of object.str() (if defined) or repr(object). encoding defaults to sys.getdefaultencoding(). errors defaults to 'strict'.
RandomFeatureGaussianProcess ¶
RandomFeatureGaussianProcess(units, num_inducing=1024, gp_kernel_type='gaussian', gp_kernel_scale=1.0, gp_output_bias=0.0, normalize_input=False, gp_kernel_scale_trainable=False, gp_output_bias_trainable=False, gp_cov_momentum=0.999, gp_cov_ridge_penalty=1.0, scale_random_features=True, use_custom_random_features=True, custom_random_features_initializer=None, custom_random_features_activation=None, l2_regularization=1e-06, gp_cov_likelihood='gaussian', return_gp_cov=True, return_random_features=False, dtype=None, name=None, trainable=True, **gp_output_kwargs)
Bases: keras.src.layers.layer.Layer
Gaussian process layer with random feature approximation [1].
During training, the layer updates the maximum a posteriori (MAP) logits
and the posterior precision matrix using minibatch statistics. During
inference, it also returns the predictive covariance, which can be used to
compute posterior mean probabilities via mean_field_logits.
Different random features can be used by setting
use_custom_random_features=True and changing the initializer and
activation of the custom random features. For example:
MLP kernel: initializer="random_normal", activation="relu"
RBF kernel: initializer="random_normal", activation=keras.ops.cos
A linear kernel can be used with gp_kernel_type="linear" and
use_custom_random_features=True.
Example:
inputs = keras.Input(shape=(128,))
logits, covmat = RandomFeatureGaussianProcess(units=10)(inputs)
probs = keras.ops.softmax(mean_field_logits(logits, covmat, math.pi / 8))
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
units
|
Number of output units. |
required | |
num_inducing
|
Number of random Fourier features. |
1024
|
|
gp_kernel_type
|
Kernel type. |
'gaussian'
|
|
gp_kernel_scale
|
Length scale of the kernel, i.e. for the RBF kernel
|
1.0
|
|
gp_output_bias
|
Initial value of the output bias. |
0.0
|
|
normalize_input
|
Whether to apply layer normalization to the input. |
False
|
|
gp_kernel_scale_trainable
|
Whether the length scale is trainable. Only
used when |
False
|
|
gp_output_bias_trainable
|
Whether the output bias is trainable. |
False
|
|
gp_cov_momentum
|
Discount factor of the moving average of the precision matrix. If not positive, minibatch precision matrices are summed instead, which gives the exact precision after one pass over the data. |
0.999
|
|
gp_cov_ridge_penalty
|
Ridge penalty of the initial precision matrix. |
1.0
|
|
scale_random_features
|
Whether to scale random features by
|
True
|
|
use_custom_random_features
|
Whether to use a frozen |
True
|
|
custom_random_features_initializer
|
Initializer for the custom random
features. Defaults to |
None
|
|
custom_random_features_activation
|
Activation for the custom random
features. Defaults to |
None
|
|
l2_regularization
|
L2 regularization strength on the output weights. |
1e-06
|
|
gp_cov_likelihood
|
Likelihood used for the Laplace approximation, one of
|
'gaussian'
|
|
return_gp_cov
|
Whether to also return the GP covariance matrix. If
|
True
|
|
return_random_features
|
Whether to also return the random features. |
False
|
|
**gp_output_kwargs
|
Additional keyword arguments to the output |
required |
Call arguments
inputs: Tensor of shape (batch_size, input_dim).
training: Whether to update the precision matrix.
Returns:
| Type | Description |
|---|---|
|
|
|
|
|
|
|
covariance of shape |
|
|
training) and the random features of shape |
|
|
|
__doc__
class-attribute
¶
__doc__ = 'Gaussian process layer with random feature approximation [1].\n\n During training, the layer updates the maximum a posteriori (MAP) logits\n and the posterior precision matrix using minibatch statistics. During\n inference, it also returns the predictive covariance, which can be used to\n compute posterior mean probabilities via `mean_field_logits`.\n\n Different random features can be used by setting\n `use_custom_random_features=True` and changing the initializer and\n activation of the custom random features. For example:\n\n MLP kernel: initializer="random_normal", activation="relu"\n RBF kernel: initializer="random_normal", activation=keras.ops.cos\n\n A linear kernel can be used with `gp_kernel_type="linear"` and\n `use_custom_random_features=True`.\n\n Example:\n\n ```python\n inputs = keras.Input(shape=(128,))\n logits, covmat = RandomFeatureGaussianProcess(units=10)(inputs)\n probs = keras.ops.softmax(mean_field_logits(logits, covmat, math.pi / 8))\n ```\n\n Args:\n units: Number of output units.\n num_inducing: Number of random Fourier features.\n gp_kernel_type: Kernel type. `"gaussian"` (RBF) by default, or\n `"linear"` when `use_custom_random_features=True`.\n gp_kernel_scale: Length scale of the kernel, i.e. for the RBF kernel\n `exp(-|x1 - x2|**2 / gp_kernel_scale)`.\n gp_output_bias: Initial value of the output bias.\n normalize_input: Whether to apply layer normalization to the input.\n gp_kernel_scale_trainable: Whether the length scale is trainable. Only\n used when `use_custom_random_features=False`.\n gp_output_bias_trainable: Whether the output bias is trainable.\n gp_cov_momentum: Discount factor of the moving average of the precision\n matrix. If not positive, minibatch precision matrices are summed\n instead, which gives the exact precision after one pass over the\n data.\n gp_cov_ridge_penalty: Ridge penalty of the initial precision matrix.\n scale_random_features: Whether to scale random features by\n `sqrt(2 / num_inducing)`.\n use_custom_random_features: Whether to use a frozen `Dense` layer for\n the random features instead of `RandomFourierFeatures`.\n custom_random_features_initializer: Initializer for the custom random\n features. Defaults to `RandomNormal(stddev=1.0)`.\n custom_random_features_activation: Activation for the custom random\n features. Defaults to `keras.ops.cos`.\n l2_regularization: L2 regularization strength on the output weights.\n gp_cov_likelihood: Likelihood used for the Laplace approximation, one of\n `"gaussian"`, `"binary_logistic"` or `"poisson"`.\n return_gp_cov: Whether to also return the GP covariance matrix. If\n `False`, no covariance is learned.\n return_random_features: Whether to also return the random features.\n **gp_output_kwargs: Additional keyword arguments to the output `Dense`\n layer.\n\n Call arguments:\n inputs: Tensor of shape `(batch_size, input_dim)`.\n training: Whether to update the precision matrix.\n\n Returns:\n `logits` of shape `(batch_size, units)` if neither `return_gp_cov` nor\n `return_random_features` is set. Otherwise a tuple of `logits`, the\n covariance of shape `(batch_size, batch_size)` (identity during\n training) and the random features of shape\n `(batch_size, num_inducing)`, in that order.\n '
str(object='') -> str str(bytes_or_buffer[, encoding[, errors]]) -> str
Create a new string object from the given object. If encoding or errors is specified, then the object must expose a data buffer that will be decoded using the given encoding and error handler. Otherwise, returns the result of object.str() (if defined) or repr(object). encoding defaults to sys.getdefaultencoding(). errors defaults to 'strict'.
__module__
class-attribute
¶
str(object='') -> str str(bytes_or_buffer[, encoding[, errors]]) -> str
Create a new string object from the given object. If encoding or errors is specified, then the object must expose a data buffer that will be decoded using the given encoding and error handler. Otherwise, returns the result of object.str() (if defined) or repr(object). encoding defaults to sys.getdefaultencoding(). errors defaults to 'strict'.
reset_covariance_matrix ¶
Resets the covariance matrix of the GP layer.
Useful for resetting the covariance at the beginning of a new epoch.
LaplaceRandomFeatureCovariance ¶
LaplaceRandomFeatureCovariance(momentum=0.999, ridge_penalty=1.0, likelihood='gaussian', dtype=None, name=None, **kwargs)
Bases: keras.src.layers.layer.Layer
Computes the Gaussian process covariance using the Laplace method.
At training time, this layer updates the Gaussian process posterior using model features in minibatches.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
momentum
|
Discount factor of the moving average of the posterior precision matrix, analogous to batch normalization momentum. If not positive, minibatch precision matrices are summed instead, which gives the exact precision matrix after one pass over the data (e.g. in the final epoch). |
0.999
|
|
ridge_penalty
|
Ridge penalty of the initial precision matrix. It
stabilizes the inverse |
1.0
|
|
likelihood
|
Likelihood used for the Laplace approximation, one of
|
'gaussian'
|
Call arguments
inputs: GP random features of shape (batch_size, num_features).
logits: Pre-activation output of shape (batch_size, 1). Required for
non-Gaussian likelihoods.
training: Whether to update the precision matrix.
Returns:
| Type | Description |
|---|---|
|
Identity of shape |
|
|
predictive covariance of the same shape during inference. |
__doc__
class-attribute
¶
__doc__ = 'Computes the Gaussian process covariance using the Laplace method.\n\n At training time, this layer updates the Gaussian process posterior using\n model features in minibatches.\n\n Args:\n momentum: Discount factor of the moving average of the posterior\n precision matrix, analogous to batch normalization momentum. If not\n positive, minibatch precision matrices are summed instead, which\n gives the exact precision matrix after one pass over the data (e.g.\n in the final epoch).\n ridge_penalty: Ridge penalty of the initial precision matrix. It\n stabilizes the inverse `inv(t(X) X + s I)`, but should not be so\n large that it dominates `t(X) X`.\n likelihood: Likelihood used for the Laplace approximation, one of\n `"gaussian"`, `"binary_logistic"` or `"poisson"`.\n\n Call arguments:\n inputs: GP random features of shape `(batch_size, num_features)`.\n logits: Pre-activation output of shape `(batch_size, 1)`. Required for\n non-Gaussian likelihoods.\n training: Whether to update the precision matrix.\n\n Returns:\n Identity of shape `(batch_size, batch_size)` during training, and the\n predictive covariance of the same shape during inference.\n '
str(object='') -> str str(bytes_or_buffer[, encoding[, errors]]) -> str
Create a new string object from the given object. If encoding or errors is specified, then the object must expose a data buffer that will be decoded using the given encoding and error handler. Otherwise, returns the result of object.str() (if defined) or repr(object). encoding defaults to sys.getdefaultencoding(). errors defaults to 'strict'.
__module__
class-attribute
¶
str(object='') -> str str(bytes_or_buffer[, encoding[, errors]]) -> str
Create a new string object from the given object. If encoding or errors is specified, then the object must expose a data buffer that will be decoded using the given encoding and error handler. Otherwise, returns the result of object.str() (if defined) or repr(object). encoding defaults to sys.getdefaultencoding(). errors defaults to 'strict'.
compute_predictive_covariance ¶
Computes the posterior predictive covariance.
Given training random features Phi_tr of shape (num_train, num_hidden) and test random features Phi_ts of shape (batch_size, num_hidden), the predictive covariance (assuming a Gaussian likelihood) is:
s * Phi_ts @ inv(t(Phi_tr) * Phi_tr + s * I) @ t(Phi_ts),
where s is the ridge penalty used to stabilize the inverse.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
gp_feature
|
Random features of the test data, shape
|
required |
Returns:
| Type | Description |
|---|---|
|
Predictive covariance matrix of shape |
make_precision_matrix_update_op ¶
Updates the precision matrix of feature weights.
reset_precision_matrix ¶
Resets the precision matrix to its initial value.
Useful for resetting the covariance at the beginning of a new epoch.
SpectralNormalization ¶
SpectralNormalization(layer, iteration=1, norm_multiplier=0.95, training=True, aggregation='mean', inhere_layer_name=False, mode='projection', **kwargs)
Bases: keras.src.layers.core.wrapper.Wrapper
Implements spectral normalization for a Dense layer.
Example:
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
layer
|
A Keras layer with a |
required | |
iteration
|
Number of power iterations per call to estimate the kernel's largest singular value. |
1
|
|
norm_multiplier
|
Upper bound of the spectral norm. Under normalization, the singular value usually converges to this value. |
0.95
|
|
training
|
Whether to perform power iteration when the layer is called
with |
True
|
|
aggregation
|
How the |
'mean'
|
|
inhere_layer_name
|
Whether to use the name of the wrapped layer. |
False
|
|
mode
|
|
'projection'
|
|
**kwargs
|
Other keyword arguments for |
required |
__doc__
class-attribute
¶
__doc__ = 'Implements spectral normalization for a Dense layer.\n\n Example:\n\n ```python\n dense = SpectralNormalization(keras.layers.Dense(64), norm_multiplier=0.9)\n ```\n\n Args:\n layer: A Keras layer with a `kernel`, typically `Dense`.\n iteration: Number of power iterations per call to estimate the\n kernel\'s largest singular value.\n norm_multiplier: Upper bound of the spectral norm. Under normalization,\n the singular value usually converges to this value.\n training: Whether to perform power iteration when the layer is called\n with `training=None`.\n aggregation: How the `u` and `v` vectors are aggregated across replicas\n in distributed training, e.g. `"mean"`.\n inhere_layer_name: Whether to use the name of the wrapped layer.\n mode: `"projection"` (as in the reference) or `"reparameterization"`\n (identical on all backends). See the module docstring.\n **kwargs: Other keyword arguments for `keras.layers.Wrapper`.\n '
str(object='') -> str str(bytes_or_buffer[, encoding[, errors]]) -> str
Create a new string object from the given object. If encoding or errors is specified, then the object must expose a data buffer that will be decoded using the given encoding and error handler. Otherwise, returns the result of object.str() (if defined) or repr(object). encoding defaults to sys.getdefaultencoding(). errors defaults to 'strict'.
__module__
class-attribute
¶
str(object='') -> str str(bytes_or_buffer[, encoding[, errors]]) -> str
Create a new string object from the given object. If encoding or errors is specified, then the object must expose a data buffer that will be decoded using the given encoding and error handler. Otherwise, returns the result of object.str() (if defined) or repr(object). encoding defaults to sys.getdefaultencoding(). errors defaults to 'strict'.
update_weights ¶
Runs power iteration and bounds the kernel's spectral norm.
With training=True, u and v are updated by power iteration first.
Otherwise the current u and v are used as is.
In "projection" mode, the normalized kernel is assigned to the wrapped
layer. Unlike the reference, which uses the raw kernel at inference,
this keeps inference normalized on JAX; elsewhere it is a no-op unless
the kernel changed since the last training call. In
"reparameterization" mode, the kernel is left unchanged.
Returns:
| Type | Description |
|---|---|
|
The normalized kernel. |
SpectralNormalizationConv2D ¶
SpectralNormalizationConv2D(layer, iteration=1, norm_multiplier=0.95, training=True, aggregation='mean', legacy_mode=False, mode='projection', **kwargs)
Bases: keras.src.layers.core.wrapper.Wrapper
Implements spectral normalization for a Conv2D layer based on [3].
The spectral norm of the convolution as a linear operator is estimated
by power iteration with conv and conv_transpose, so the input height
and width must be known and divisible by the strides. Inputs are expected
in channels_last format.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
layer
|
A |
required | |
iteration
|
Number of power iterations per call. |
1
|
|
norm_multiplier
|
Upper bound of the spectral norm. |
0.95
|
|
training
|
Whether to perform power iteration on each call. |
True
|
|
aggregation
|
How the |
'mean'
|
|
legacy_mode
|
Whether to set the leading dimension of |
False
|
|
mode
|
|
'projection'
|
|
**kwargs
|
Other keyword arguments for |
required |
__doc__
class-attribute
¶
__doc__ = 'Implements spectral normalization for a Conv2D layer based on [3].\n\n The spectral norm of the convolution as a linear operator is estimated\n by power iteration with `conv` and `conv_transpose`, so the input height\n and width must be known and divisible by the strides. Inputs are expected\n in `channels_last` format.\n\n Args:\n layer: A `keras.layers.Conv2D` layer.\n iteration: Number of power iterations per call.\n norm_multiplier: Upper bound of the spectral norm.\n training: Whether to perform power iteration on each call.\n aggregation: How the `u` and `v` vectors are aggregated across replicas\n in distributed training, e.g. `"mean"`.\n legacy_mode: Whether to set the leading dimension of `u` and `v` to the\n batch size. Only for backward compatibility.\n mode: `"projection"` (as in the reference) or `"reparameterization"`\n (identical on all backends). See the module docstring.\n **kwargs: Other keyword arguments for `keras.layers.Wrapper`.\n '
str(object='') -> str str(bytes_or_buffer[, encoding[, errors]]) -> str
Create a new string object from the given object. If encoding or errors is specified, then the object must expose a data buffer that will be decoded using the given encoding and error handler. Otherwise, returns the result of object.str() (if defined) or repr(object). encoding defaults to sys.getdefaultencoding(). errors defaults to 'strict'.
__module__
class-attribute
¶
str(object='') -> str str(bytes_or_buffer[, encoding[, errors]]) -> str
Create a new string object from the given object. If encoding or errors is specified, then the object must expose a data buffer that will be decoded using the given encoding and error handler. Otherwise, returns the result of object.str() (if defined) or repr(object). encoding defaults to sys.getdefaultencoding(). errors defaults to 'strict'.
update_weights ¶
Computes power iteration for convolutional filters based on [3].
Returns:
| Type | Description |
|---|---|
|
The normalized kernel. It is also assigned to the wrapped layer in |
|
|
|
ClassificationHead ¶
ClassificationHead(inner_dim, num_classes, cls_token_idx=0, activation='tanh', dropout_rate=0.0, initializer='glorot_uniform', **kwargs)
Bases: keras.src.layers.layer.Layer
Pooling head for classification tasks.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
inner_dim
|
Dimensionality of the inner projection layer. If 0 or
|
required | |
num_classes
|
Number of output classes. |
required | |
cls_token_idx
|
Index of the sequence position to pool when the input
has rank 3, e.g. the |
0
|
|
activation
|
Activation of the inner projection layer. |
'tanh'
|
|
dropout_rate
|
Dropout probability before the output projection. |
0.0
|
|
initializer
|
Initializer for the dense layer kernels. |
'glorot_uniform'
|
|
**kwargs
|
Keyword arguments for |
required |
Call arguments
features: Tensor of shape (batch_size, seq_len, hidden_size), which is
pooled at cls_token_idx, or (batch_size, hidden_size).
only_project: If True, returns the inner projection instead of the
logits.
training: Whether the layer is in training mode.
__doc__
class-attribute
¶
__doc__ = 'Pooling head for classification tasks.\n\n Args:\n inner_dim: Dimensionality of the inner projection layer. If 0 or\n `None`, only the output projection layer is created.\n num_classes: Number of output classes.\n cls_token_idx: Index of the sequence position to pool when the input\n has rank 3, e.g. the `<CLS>` token of a sequence encoder.\n activation: Activation of the inner projection layer.\n dropout_rate: Dropout probability before the output projection.\n initializer: Initializer for the dense layer kernels.\n **kwargs: Keyword arguments for `keras.layers.Layer`.\n\n Call arguments:\n features: Tensor of shape `(batch_size, seq_len, hidden_size)`, which is\n pooled at `cls_token_idx`, or `(batch_size, hidden_size)`.\n only_project: If `True`, returns the inner projection instead of the\n logits.\n training: Whether the layer is in training mode.\n '
str(object='') -> str str(bytes_or_buffer[, encoding[, errors]]) -> str
Create a new string object from the given object. If encoding or errors is specified, then the object must expose a data buffer that will be decoded using the given encoding and error handler. Otherwise, returns the result of object.str() (if defined) or repr(object). encoding defaults to sys.getdefaultencoding(). errors defaults to 'strict'.
__module__
class-attribute
¶
str(object='') -> str str(bytes_or_buffer[, encoding[, errors]]) -> str
Create a new string object from the given object. If encoding or errors is specified, then the object must expose a data buffer that will be decoded using the given encoding and error handler. Otherwise, returns the result of object.str() (if defined) or repr(object). encoding defaults to sys.getdefaultencoding(). errors defaults to 'strict'.
GaussianProcessClassificationHead ¶
GaussianProcessClassificationHead(inner_dim, num_classes, cls_token_idx=0, activation='tanh', dropout_rate=0.0, initializer='glorot_uniform', use_spec_norm=True, use_gp_layer=True, temperature=None, **kwargs)
Bases: keras_proba.layers.classification_head.ClassificationHead
Gaussian process-based pooling head for classification (SNGP) [1].
It applies spectral normalization to the hidden pooler layer, and replaces
the dense output layer with a random feature Gaussian process. During
training it returns the raw logits; at inference it returns logits adjusted
with mean_field_logits, and optionally the covariance matrix.
Remember to reset the GP covariance at the start of each epoch, e.g. with
keras_proba.callbacks.ResetCovarianceCallback.
Example:
inputs = keras.Input((128,))
hidden = keras.layers.Dense(128, activation="relu")(inputs)
head = GaussianProcessClassificationHead(inner_dim=128, num_classes=10, temperature=math.pi / 8)
model = keras.Model(inputs, head(hidden))
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
inner_dim
|
Dimensionality of the inner projection layer. If 0 or
|
required | |
num_classes
|
Number of output classes. |
required | |
cls_token_idx
|
Index of the sequence position to pool when the input has rank 3. |
0
|
|
activation
|
Activation of the inner projection layer. |
'tanh'
|
|
dropout_rate
|
Dropout probability before the output projection. |
0.0
|
|
initializer
|
Initializer for the dense layer kernels. |
'glorot_uniform'
|
|
use_spec_norm
|
Whether to apply spectral normalization to the pooler layer. |
True
|
|
use_gp_layer
|
Whether to use a Gaussian process as the output layer. |
True
|
|
temperature
|
Mean-field factor used at inference. If |
None
|
|
**kwargs
|
Spectral normalization arguments ( |
required |
Call arguments
features: Tensor of shape (batch_size, seq_len, hidden_size) or
(batch_size, hidden_size).
training: Whether the layer is in training mode.
return_covmat: Whether to also return the covariance matrix of shape
(batch_size, batch_size) when use_gp_layer=True. During
training, keep it False to be compatible with model.fit().
__doc__
class-attribute
¶
__doc__ = 'Gaussian process-based pooling head for classification (SNGP) [1].\n\n It applies spectral normalization to the hidden pooler layer, and replaces\n the dense output layer with a random feature Gaussian process. During\n training it returns the raw logits; at inference it returns logits adjusted\n with `mean_field_logits`, and optionally the covariance matrix.\n\n Remember to reset the GP covariance at the start of each epoch, e.g. with\n `keras_proba.callbacks.ResetCovarianceCallback`.\n\n Example:\n\n ```python\n inputs = keras.Input((128,))\n hidden = keras.layers.Dense(128, activation="relu")(inputs)\n head = GaussianProcessClassificationHead(inner_dim=128, num_classes=10, temperature=math.pi / 8)\n model = keras.Model(inputs, head(hidden))\n ```\n\n Args:\n inner_dim: Dimensionality of the inner projection layer. If 0 or\n `None`, only the output projection layer is created.\n num_classes: Number of output classes.\n cls_token_idx: Index of the sequence position to pool when the input\n has rank 3.\n activation: Activation of the inner projection layer.\n dropout_rate: Dropout probability before the output projection.\n initializer: Initializer for the dense layer kernels.\n use_spec_norm: Whether to apply spectral normalization to the pooler\n layer.\n use_gp_layer: Whether to use a Gaussian process as the output layer.\n temperature: Mean-field factor used at inference. If `None`, no\n mean-field adjustment is applied.\n **kwargs: Spectral normalization arguments (`iteration`,\n `norm_multiplier`, `mode`), `RandomFeatureGaussianProcess`\n arguments (see `extract_gp_layer_kwargs` for those supported and\n their defaults), and keyword arguments for `keras.layers.Layer`.\n\n Call arguments:\n features: Tensor of shape `(batch_size, seq_len, hidden_size)` or\n `(batch_size, hidden_size)`.\n training: Whether the layer is in training mode.\n return_covmat: Whether to also return the covariance matrix of shape\n `(batch_size, batch_size)` when `use_gp_layer=True`. During\n training, keep it `False` to be compatible with `model.fit()`.\n '
str(object='') -> str str(bytes_or_buffer[, encoding[, errors]]) -> str
Create a new string object from the given object. If encoding or errors is specified, then the object must expose a data buffer that will be decoded using the given encoding and error handler. Otherwise, returns the result of object.str() (if defined) or repr(object). encoding defaults to sys.getdefaultencoding(). errors defaults to 'strict'.
__module__
class-attribute
¶
str(object='') -> str str(bytes_or_buffer[, encoding[, errors]]) -> str
Create a new string object from the given object. If encoding or errors is specified, then the object must expose a data buffer that will be decoded using the given encoding and error handler. Otherwise, returns the result of object.str() (if defined) or repr(object). encoding defaults to sys.getdefaultencoding(). errors defaults to 'strict'.
reset_covariance_matrix ¶
Resets the covariance matrix of the Gaussian process layer.
Bayesian (variational)¶
DenseReparameterization ¶
DenseReparameterization(units, activation=None, use_bias=True, kernel_loc_initializer=None, kernel_untransformed_scale_initializer=None, bias_initializer=None, kernel_prior_scale=1.0, kl_weight=1.0, seed=None, activity_regularizer=None, **kwargs)
Bases: keras_proba.layers.variational._DenseVariational
Bayesian dense layer with the reparameterization estimator [1].
One kernel W ~ q(W) is sampled per call:
outputs = activation(inputs @ W + bias).
Example:
kl_weight = 1.0 / num_train_examples
model = keras.Sequential([
keras.Input((num_features,)),
DenseReparameterization(64, activation="relu", kl_weight=kl_weight),
DenseReparameterization(num_classes, kl_weight=kl_weight),
])
model.compile(optimizer="adam", loss=keras.losses.SparseCategoricalCrossentropy(from_logits=True))
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
units
|
Dimensionality of the output. |
required | |
activation
|
Activation function. |
None
|
|
use_bias
|
Whether to add a (deterministic) bias. |
True
|
|
kernel_loc_initializer
|
Initializer for the posterior mean of the
kernel. Defaults to |
None
|
|
kernel_untransformed_scale_initializer
|
Initializer for the posterior
scale before |
None
|
|
bias_initializer
|
Initializer for the bias. Defaults to
|
None
|
|
kernel_prior_scale
|
Standard deviation of the Normal kernel prior. |
1.0
|
|
kl_weight
|
Multiplier of the KL divergence added to the losses, e.g.
|
1.0
|
|
seed
|
Seed for the random noise. |
None
|
|
activity_regularizer
|
Regularizer for the output. |
None
|
__doc__
class-attribute
¶
__doc__ = 'Bayesian dense layer with the reparameterization estimator [1].\n\n One kernel `W ~ q(W)` is sampled per call:\n `outputs = activation(inputs @ W + bias)`.\n\n Example:\n\n ```python\n kl_weight = 1.0 / num_train_examples\n model = keras.Sequential([\n keras.Input((num_features,)),\n DenseReparameterization(64, activation="relu", kl_weight=kl_weight),\n DenseReparameterization(num_classes, kl_weight=kl_weight),\n ])\n model.compile(optimizer="adam", loss=keras.losses.SparseCategoricalCrossentropy(from_logits=True))\n ```\n\n Args:\n units: Dimensionality of the output.\n activation: Activation function.\n use_bias: Whether to add a (deterministic) bias.\n kernel_loc_initializer: Initializer for the posterior mean of the\n kernel. Defaults to `RandomNormal(stddev=0.1)`.\n kernel_untransformed_scale_initializer: Initializer for the posterior\n scale before `softplus`. Defaults to `RandomNormal(mean=-3,\n stddev=0.1)`, a posterior close to a point mass.\n bias_initializer: Initializer for the bias. Defaults to\n `RandomNormal(stddev=0.1)`.\n kernel_prior_scale: Standard deviation of the Normal kernel prior.\n kl_weight: Multiplier of the KL divergence added to the losses, e.g.\n `1 / num_train_examples`.\n seed: Seed for the random noise.\n activity_regularizer: Regularizer for the output.\n '
str(object='') -> str str(bytes_or_buffer[, encoding[, errors]]) -> str
Create a new string object from the given object. If encoding or errors is specified, then the object must expose a data buffer that will be decoded using the given encoding and error handler. Otherwise, returns the result of object.str() (if defined) or repr(object). encoding defaults to sys.getdefaultencoding(). errors defaults to 'strict'.
__module__
class-attribute
¶
str(object='') -> str str(bytes_or_buffer[, encoding[, errors]]) -> str
Create a new string object from the given object. If encoding or errors is specified, then the object must expose a data buffer that will be decoded using the given encoding and error handler. Otherwise, returns the result of object.str() (if defined) or repr(object). encoding defaults to sys.getdefaultencoding(). errors defaults to 'strict'.
DenseLocalReparameterization ¶
DenseLocalReparameterization(units, activation=None, use_bias=True, kernel_loc_initializer=None, kernel_untransformed_scale_initializer=None, bias_initializer=None, kernel_prior_scale=1.0, kl_weight=1.0, seed=None, activity_regularizer=None, **kwargs)
Bases: keras_proba.layers.variational._DenseVariational
Bayesian dense layer with the local reparameterization estimator [2].
Samples the pre-activations, whose distribution under q(W) is
N(inputs @ loc, (inputs^2) @ (scale^2)), independently per example.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
units
|
Dimensionality of the output. |
required | |
activation
|
Activation function. |
None
|
|
use_bias
|
Whether to add a (deterministic) bias. |
True
|
|
kernel_loc_initializer
|
Initializer for the posterior mean of the
kernel. Defaults to |
None
|
|
kernel_untransformed_scale_initializer
|
Initializer for the posterior
scale before |
None
|
|
bias_initializer
|
Initializer for the bias. Defaults to
|
None
|
|
kernel_prior_scale
|
Standard deviation of the Normal kernel prior. |
1.0
|
|
kl_weight
|
Multiplier of the KL divergence added to the losses, e.g.
|
1.0
|
|
seed
|
Seed for the random noise. |
None
|
|
activity_regularizer
|
Regularizer for the output. |
None
|
__doc__
class-attribute
¶
__doc__ = 'Bayesian dense layer with the local reparameterization estimator [2].\n\n Samples the pre-activations, whose distribution under `q(W)` is\n `N(inputs @ loc, (inputs^2) @ (scale^2))`, independently per example.\n\n Args:\n units: Dimensionality of the output.\n activation: Activation function.\n use_bias: Whether to add a (deterministic) bias.\n kernel_loc_initializer: Initializer for the posterior mean of the\n kernel. Defaults to `RandomNormal(stddev=0.1)`.\n kernel_untransformed_scale_initializer: Initializer for the posterior\n scale before `softplus`. Defaults to `RandomNormal(mean=-3,\n stddev=0.1)`, a posterior close to a point mass.\n bias_initializer: Initializer for the bias. Defaults to\n `RandomNormal(stddev=0.1)`.\n kernel_prior_scale: Standard deviation of the Normal kernel prior.\n kl_weight: Multiplier of the KL divergence added to the losses, e.g.\n `1 / num_train_examples`.\n seed: Seed for the random noise.\n activity_regularizer: Regularizer for the output.\n '
str(object='') -> str str(bytes_or_buffer[, encoding[, errors]]) -> str
Create a new string object from the given object. If encoding or errors is specified, then the object must expose a data buffer that will be decoded using the given encoding and error handler. Otherwise, returns the result of object.str() (if defined) or repr(object). encoding defaults to sys.getdefaultencoding(). errors defaults to 'strict'.
__module__
class-attribute
¶
str(object='') -> str str(bytes_or_buffer[, encoding[, errors]]) -> str
Create a new string object from the given object. If encoding or errors is specified, then the object must expose a data buffer that will be decoded using the given encoding and error handler. Otherwise, returns the result of object.str() (if defined) or repr(object). encoding defaults to sys.getdefaultencoding(). errors defaults to 'strict'.
DenseFlipout ¶
DenseFlipout(units, activation=None, use_bias=True, kernel_loc_initializer=None, kernel_untransformed_scale_initializer=None, bias_initializer=None, kernel_prior_scale=1.0, kl_weight=1.0, seed=None, activity_regularizer=None, **kwargs)
Bases: keras_proba.layers.variational._DenseVariational
Bayesian dense layer with the Flipout estimator [3].
Samples one kernel perturbation dW ~ N(0, scale^2) per call, and
decorrelates it across examples with random signs:
outputs = inputs @ loc + ((inputs * s_in) @ dW) * s_out.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
units
|
Dimensionality of the output. |
required | |
activation
|
Activation function. |
None
|
|
use_bias
|
Whether to add a (deterministic) bias. |
True
|
|
kernel_loc_initializer
|
Initializer for the posterior mean of the
kernel. Defaults to |
None
|
|
kernel_untransformed_scale_initializer
|
Initializer for the posterior
scale before |
None
|
|
bias_initializer
|
Initializer for the bias. Defaults to
|
None
|
|
kernel_prior_scale
|
Standard deviation of the Normal kernel prior. |
1.0
|
|
kl_weight
|
Multiplier of the KL divergence added to the losses, e.g.
|
1.0
|
|
seed
|
Seed for the random noise. |
None
|
|
activity_regularizer
|
Regularizer for the output. |
None
|
__doc__
class-attribute
¶
__doc__ = 'Bayesian dense layer with the Flipout estimator [3].\n\n Samples one kernel perturbation `dW ~ N(0, scale^2)` per call, and\n decorrelates it across examples with random signs:\n `outputs = inputs @ loc + ((inputs * s_in) @ dW) * s_out`.\n\n Args:\n units: Dimensionality of the output.\n activation: Activation function.\n use_bias: Whether to add a (deterministic) bias.\n kernel_loc_initializer: Initializer for the posterior mean of the\n kernel. Defaults to `RandomNormal(stddev=0.1)`.\n kernel_untransformed_scale_initializer: Initializer for the posterior\n scale before `softplus`. Defaults to `RandomNormal(mean=-3,\n stddev=0.1)`, a posterior close to a point mass.\n bias_initializer: Initializer for the bias. Defaults to\n `RandomNormal(stddev=0.1)`.\n kernel_prior_scale: Standard deviation of the Normal kernel prior.\n kl_weight: Multiplier of the KL divergence added to the losses, e.g.\n `1 / num_train_examples`.\n seed: Seed for the random noise.\n activity_regularizer: Regularizer for the output.\n '
str(object='') -> str str(bytes_or_buffer[, encoding[, errors]]) -> str
Create a new string object from the given object. If encoding or errors is specified, then the object must expose a data buffer that will be decoded using the given encoding and error handler. Otherwise, returns the result of object.str() (if defined) or repr(object). encoding defaults to sys.getdefaultencoding(). errors defaults to 'strict'.
__module__
class-attribute
¶
str(object='') -> str str(bytes_or_buffer[, encoding[, errors]]) -> str
Create a new string object from the given object. If encoding or errors is specified, then the object must expose a data buffer that will be decoded using the given encoding and error handler. Otherwise, returns the result of object.str() (if defined) or repr(object). encoding defaults to sys.getdefaultencoding(). errors defaults to 'strict'.
Conv1DReparameterization ¶
Conv1DReparameterization(filters, kernel_size, strides=1, padding='valid', data_format='channels_last', dilation_rate=1, activation=None, use_bias=True, kernel_loc_initializer=None, kernel_untransformed_scale_initializer=None, bias_initializer=None, kernel_prior_scale=1.0, kl_weight=1.0, seed=None, activity_regularizer=None, **kwargs)
Bases: keras_proba.layers.variational._ConvReparameterization
Bayesian 1D convolution (e.g. temporal convolution) with the reparameterization estimator [1].
One kernel W ~ q(W) is sampled per call.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
filters
|
Number of output channels. |
required | |
kernel_size
|
Int or tuple of ints, one per spatial dimension: the convolution window. |
required | |
strides
|
Int or tuple of ints, one per spatial dimension. |
1
|
|
padding
|
|
'valid'
|
|
data_format
|
|
'channels_last'
|
|
dilation_rate
|
Int or tuple of ints, one per spatial dimension. |
1
|
|
activation
|
Activation function. |
None
|
|
use_bias
|
Whether to add a (deterministic) bias. |
True
|
|
kernel_loc_initializer
|
Initializer for the posterior mean of the
kernel. Defaults to |
None
|
|
kernel_untransformed_scale_initializer
|
Initializer for the posterior
scale before |
None
|
|
bias_initializer
|
Initializer for the bias. Defaults to
|
None
|
|
kernel_prior_scale
|
Standard deviation of the Normal kernel prior. |
1.0
|
|
kl_weight
|
Multiplier of the KL divergence added to the losses, e.g.
|
1.0
|
|
seed
|
Seed for the random noise. |
None
|
|
activity_regularizer
|
Regularizer for the output. |
None
|
__doc__
class-attribute
¶
__doc__ = 'Bayesian 1D convolution (e.g. temporal convolution) with the reparameterization estimator [1].\n\n One kernel `W ~ q(W)` is sampled per call.\n\n Args:\n filters: Number of output channels.\n kernel_size: Int or tuple of ints, one per spatial dimension: the\n convolution window.\n strides: Int or tuple of ints, one per spatial dimension.\n padding: `"valid"` or `"same"`.\n data_format: `"channels_last"` or `"channels_first"`.\n dilation_rate: Int or tuple of ints, one per spatial dimension.\n activation: Activation function.\n use_bias: Whether to add a (deterministic) bias.\n kernel_loc_initializer: Initializer for the posterior mean of the\n kernel. Defaults to `RandomNormal(stddev=0.1)`.\n kernel_untransformed_scale_initializer: Initializer for the posterior\n scale before `softplus`. Defaults to `RandomNormal(mean=-3,\n stddev=0.1)`, a posterior close to a point mass.\n bias_initializer: Initializer for the bias. Defaults to\n `RandomNormal(stddev=0.1)`.\n kernel_prior_scale: Standard deviation of the Normal kernel prior.\n kl_weight: Multiplier of the KL divergence added to the losses, e.g.\n `1 / num_train_examples`.\n seed: Seed for the random noise.\n activity_regularizer: Regularizer for the output.\n '
str(object='') -> str str(bytes_or_buffer[, encoding[, errors]]) -> str
Create a new string object from the given object. If encoding or errors is specified, then the object must expose a data buffer that will be decoded using the given encoding and error handler. Otherwise, returns the result of object.str() (if defined) or repr(object). encoding defaults to sys.getdefaultencoding(). errors defaults to 'strict'.
__module__
class-attribute
¶
str(object='') -> str str(bytes_or_buffer[, encoding[, errors]]) -> str
Create a new string object from the given object. If encoding or errors is specified, then the object must expose a data buffer that will be decoded using the given encoding and error handler. Otherwise, returns the result of object.str() (if defined) or repr(object). encoding defaults to sys.getdefaultencoding(). errors defaults to 'strict'.
rank
class-attribute
¶
int([x]) -> integer int(x, base=10) -> integer
Convert a number or string to an integer, or return 0 if no arguments are given. If x is a number, return x.int(). For floating-point numbers, this truncates towards zero.
If x is not a number or if base is given, then x must be a string, bytes, or bytearray instance representing an integer literal in the given base. The literal can be preceded by '+' or '-' and be surrounded by whitespace. The base defaults to 10. Valid bases are 0 and 2-36. Base 0 means to interpret the base from the string as an integer literal.
int('0b100', base=0) 4
Conv1DFlipout ¶
Conv1DFlipout(filters, kernel_size, strides=1, padding='valid', data_format='channels_last', dilation_rate=1, activation=None, use_bias=True, kernel_loc_initializer=None, kernel_untransformed_scale_initializer=None, bias_initializer=None, kernel_prior_scale=1.0, kl_weight=1.0, seed=None, activity_regularizer=None, **kwargs)
Bases: keras_proba.layers.variational._ConvFlipout
Bayesian 1D convolution (e.g. temporal convolution) with the Flipout estimator [3].
Samples one kernel perturbation per call, decorrelated across examples with
random signs per example and channel:
outputs = conv(inputs, loc) + conv(inputs * s_in, dW) * s_out.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
filters
|
Number of output channels. |
required | |
kernel_size
|
Int or tuple of ints, one per spatial dimension: the convolution window. |
required | |
strides
|
Int or tuple of ints, one per spatial dimension. |
1
|
|
padding
|
|
'valid'
|
|
data_format
|
|
'channels_last'
|
|
dilation_rate
|
Int or tuple of ints, one per spatial dimension. |
1
|
|
activation
|
Activation function. |
None
|
|
use_bias
|
Whether to add a (deterministic) bias. |
True
|
|
kernel_loc_initializer
|
Initializer for the posterior mean of the
kernel. Defaults to |
None
|
|
kernel_untransformed_scale_initializer
|
Initializer for the posterior
scale before |
None
|
|
bias_initializer
|
Initializer for the bias. Defaults to
|
None
|
|
kernel_prior_scale
|
Standard deviation of the Normal kernel prior. |
1.0
|
|
kl_weight
|
Multiplier of the KL divergence added to the losses, e.g.
|
1.0
|
|
seed
|
Seed for the random noise. |
None
|
|
activity_regularizer
|
Regularizer for the output. |
None
|
__doc__
class-attribute
¶
__doc__ = 'Bayesian 1D convolution (e.g. temporal convolution) with the Flipout estimator [3].\n\n Samples one kernel perturbation per call, decorrelated across examples with\n random signs per example and channel:\n `outputs = conv(inputs, loc) + conv(inputs * s_in, dW) * s_out`.\n\n Args:\n filters: Number of output channels.\n kernel_size: Int or tuple of ints, one per spatial dimension: the\n convolution window.\n strides: Int or tuple of ints, one per spatial dimension.\n padding: `"valid"` or `"same"`.\n data_format: `"channels_last"` or `"channels_first"`.\n dilation_rate: Int or tuple of ints, one per spatial dimension.\n activation: Activation function.\n use_bias: Whether to add a (deterministic) bias.\n kernel_loc_initializer: Initializer for the posterior mean of the\n kernel. Defaults to `RandomNormal(stddev=0.1)`.\n kernel_untransformed_scale_initializer: Initializer for the posterior\n scale before `softplus`. Defaults to `RandomNormal(mean=-3,\n stddev=0.1)`, a posterior close to a point mass.\n bias_initializer: Initializer for the bias. Defaults to\n `RandomNormal(stddev=0.1)`.\n kernel_prior_scale: Standard deviation of the Normal kernel prior.\n kl_weight: Multiplier of the KL divergence added to the losses, e.g.\n `1 / num_train_examples`.\n seed: Seed for the random noise.\n activity_regularizer: Regularizer for the output.\n '
str(object='') -> str str(bytes_or_buffer[, encoding[, errors]]) -> str
Create a new string object from the given object. If encoding or errors is specified, then the object must expose a data buffer that will be decoded using the given encoding and error handler. Otherwise, returns the result of object.str() (if defined) or repr(object). encoding defaults to sys.getdefaultencoding(). errors defaults to 'strict'.
__module__
class-attribute
¶
str(object='') -> str str(bytes_or_buffer[, encoding[, errors]]) -> str
Create a new string object from the given object. If encoding or errors is specified, then the object must expose a data buffer that will be decoded using the given encoding and error handler. Otherwise, returns the result of object.str() (if defined) or repr(object). encoding defaults to sys.getdefaultencoding(). errors defaults to 'strict'.
rank
class-attribute
¶
int([x]) -> integer int(x, base=10) -> integer
Convert a number or string to an integer, or return 0 if no arguments are given. If x is a number, return x.int(). For floating-point numbers, this truncates towards zero.
If x is not a number or if base is given, then x must be a string, bytes, or bytearray instance representing an integer literal in the given base. The literal can be preceded by '+' or '-' and be surrounded by whitespace. The base defaults to 10. Valid bases are 0 and 2-36. Base 0 means to interpret the base from the string as an integer literal.
int('0b100', base=0) 4
Conv2DReparameterization ¶
Conv2DReparameterization(filters, kernel_size, strides=1, padding='valid', data_format='channels_last', dilation_rate=1, activation=None, use_bias=True, kernel_loc_initializer=None, kernel_untransformed_scale_initializer=None, bias_initializer=None, kernel_prior_scale=1.0, kl_weight=1.0, seed=None, activity_regularizer=None, **kwargs)
Bases: keras_proba.layers.variational._ConvReparameterization
Bayesian 2D convolution (e.g. spatial convolution over images) with the reparameterization estimator [1].
One kernel W ~ q(W) is sampled per call.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
filters
|
Number of output channels. |
required | |
kernel_size
|
Int or tuple of ints, one per spatial dimension: the convolution window. |
required | |
strides
|
Int or tuple of ints, one per spatial dimension. |
1
|
|
padding
|
|
'valid'
|
|
data_format
|
|
'channels_last'
|
|
dilation_rate
|
Int or tuple of ints, one per spatial dimension. |
1
|
|
activation
|
Activation function. |
None
|
|
use_bias
|
Whether to add a (deterministic) bias. |
True
|
|
kernel_loc_initializer
|
Initializer for the posterior mean of the
kernel. Defaults to |
None
|
|
kernel_untransformed_scale_initializer
|
Initializer for the posterior
scale before |
None
|
|
bias_initializer
|
Initializer for the bias. Defaults to
|
None
|
|
kernel_prior_scale
|
Standard deviation of the Normal kernel prior. |
1.0
|
|
kl_weight
|
Multiplier of the KL divergence added to the losses, e.g.
|
1.0
|
|
seed
|
Seed for the random noise. |
None
|
|
activity_regularizer
|
Regularizer for the output. |
None
|
__doc__
class-attribute
¶
__doc__ = 'Bayesian 2D convolution (e.g. spatial convolution over images) with the reparameterization estimator [1].\n\n One kernel `W ~ q(W)` is sampled per call.\n\n Args:\n filters: Number of output channels.\n kernel_size: Int or tuple of ints, one per spatial dimension: the\n convolution window.\n strides: Int or tuple of ints, one per spatial dimension.\n padding: `"valid"` or `"same"`.\n data_format: `"channels_last"` or `"channels_first"`.\n dilation_rate: Int or tuple of ints, one per spatial dimension.\n activation: Activation function.\n use_bias: Whether to add a (deterministic) bias.\n kernel_loc_initializer: Initializer for the posterior mean of the\n kernel. Defaults to `RandomNormal(stddev=0.1)`.\n kernel_untransformed_scale_initializer: Initializer for the posterior\n scale before `softplus`. Defaults to `RandomNormal(mean=-3,\n stddev=0.1)`, a posterior close to a point mass.\n bias_initializer: Initializer for the bias. Defaults to\n `RandomNormal(stddev=0.1)`.\n kernel_prior_scale: Standard deviation of the Normal kernel prior.\n kl_weight: Multiplier of the KL divergence added to the losses, e.g.\n `1 / num_train_examples`.\n seed: Seed for the random noise.\n activity_regularizer: Regularizer for the output.\n '
str(object='') -> str str(bytes_or_buffer[, encoding[, errors]]) -> str
Create a new string object from the given object. If encoding or errors is specified, then the object must expose a data buffer that will be decoded using the given encoding and error handler. Otherwise, returns the result of object.str() (if defined) or repr(object). encoding defaults to sys.getdefaultencoding(). errors defaults to 'strict'.
__module__
class-attribute
¶
str(object='') -> str str(bytes_or_buffer[, encoding[, errors]]) -> str
Create a new string object from the given object. If encoding or errors is specified, then the object must expose a data buffer that will be decoded using the given encoding and error handler. Otherwise, returns the result of object.str() (if defined) or repr(object). encoding defaults to sys.getdefaultencoding(). errors defaults to 'strict'.
rank
class-attribute
¶
int([x]) -> integer int(x, base=10) -> integer
Convert a number or string to an integer, or return 0 if no arguments are given. If x is a number, return x.int(). For floating-point numbers, this truncates towards zero.
If x is not a number or if base is given, then x must be a string, bytes, or bytearray instance representing an integer literal in the given base. The literal can be preceded by '+' or '-' and be surrounded by whitespace. The base defaults to 10. Valid bases are 0 and 2-36. Base 0 means to interpret the base from the string as an integer literal.
int('0b100', base=0) 4
Conv2DFlipout ¶
Conv2DFlipout(filters, kernel_size, strides=1, padding='valid', data_format='channels_last', dilation_rate=1, activation=None, use_bias=True, kernel_loc_initializer=None, kernel_untransformed_scale_initializer=None, bias_initializer=None, kernel_prior_scale=1.0, kl_weight=1.0, seed=None, activity_regularizer=None, **kwargs)
Bases: keras_proba.layers.variational._ConvFlipout
Bayesian 2D convolution (e.g. spatial convolution over images) with the Flipout estimator [3].
Samples one kernel perturbation per call, decorrelated across examples with
random signs per example and channel:
outputs = conv(inputs, loc) + conv(inputs * s_in, dW) * s_out.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
filters
|
Number of output channels. |
required | |
kernel_size
|
Int or tuple of ints, one per spatial dimension: the convolution window. |
required | |
strides
|
Int or tuple of ints, one per spatial dimension. |
1
|
|
padding
|
|
'valid'
|
|
data_format
|
|
'channels_last'
|
|
dilation_rate
|
Int or tuple of ints, one per spatial dimension. |
1
|
|
activation
|
Activation function. |
None
|
|
use_bias
|
Whether to add a (deterministic) bias. |
True
|
|
kernel_loc_initializer
|
Initializer for the posterior mean of the
kernel. Defaults to |
None
|
|
kernel_untransformed_scale_initializer
|
Initializer for the posterior
scale before |
None
|
|
bias_initializer
|
Initializer for the bias. Defaults to
|
None
|
|
kernel_prior_scale
|
Standard deviation of the Normal kernel prior. |
1.0
|
|
kl_weight
|
Multiplier of the KL divergence added to the losses, e.g.
|
1.0
|
|
seed
|
Seed for the random noise. |
None
|
|
activity_regularizer
|
Regularizer for the output. |
None
|
__doc__
class-attribute
¶
__doc__ = 'Bayesian 2D convolution (e.g. spatial convolution over images) with the Flipout estimator [3].\n\n Samples one kernel perturbation per call, decorrelated across examples with\n random signs per example and channel:\n `outputs = conv(inputs, loc) + conv(inputs * s_in, dW) * s_out`.\n\n Args:\n filters: Number of output channels.\n kernel_size: Int or tuple of ints, one per spatial dimension: the\n convolution window.\n strides: Int or tuple of ints, one per spatial dimension.\n padding: `"valid"` or `"same"`.\n data_format: `"channels_last"` or `"channels_first"`.\n dilation_rate: Int or tuple of ints, one per spatial dimension.\n activation: Activation function.\n use_bias: Whether to add a (deterministic) bias.\n kernel_loc_initializer: Initializer for the posterior mean of the\n kernel. Defaults to `RandomNormal(stddev=0.1)`.\n kernel_untransformed_scale_initializer: Initializer for the posterior\n scale before `softplus`. Defaults to `RandomNormal(mean=-3,\n stddev=0.1)`, a posterior close to a point mass.\n bias_initializer: Initializer for the bias. Defaults to\n `RandomNormal(stddev=0.1)`.\n kernel_prior_scale: Standard deviation of the Normal kernel prior.\n kl_weight: Multiplier of the KL divergence added to the losses, e.g.\n `1 / num_train_examples`.\n seed: Seed for the random noise.\n activity_regularizer: Regularizer for the output.\n '
str(object='') -> str str(bytes_or_buffer[, encoding[, errors]]) -> str
Create a new string object from the given object. If encoding or errors is specified, then the object must expose a data buffer that will be decoded using the given encoding and error handler. Otherwise, returns the result of object.str() (if defined) or repr(object). encoding defaults to sys.getdefaultencoding(). errors defaults to 'strict'.
__module__
class-attribute
¶
str(object='') -> str str(bytes_or_buffer[, encoding[, errors]]) -> str
Create a new string object from the given object. If encoding or errors is specified, then the object must expose a data buffer that will be decoded using the given encoding and error handler. Otherwise, returns the result of object.str() (if defined) or repr(object). encoding defaults to sys.getdefaultencoding(). errors defaults to 'strict'.
rank
class-attribute
¶
int([x]) -> integer int(x, base=10) -> integer
Convert a number or string to an integer, or return 0 if no arguments are given. If x is a number, return x.int(). For floating-point numbers, this truncates towards zero.
If x is not a number or if base is given, then x must be a string, bytes, or bytearray instance representing an integer literal in the given base. The literal can be preceded by '+' or '-' and be surrounded by whitespace. The base defaults to 10. Valid bases are 0 and 2-36. Base 0 means to interpret the base from the string as an integer literal.
int('0b100', base=0) 4
Conv3DReparameterization ¶
Conv3DReparameterization(filters, kernel_size, strides=1, padding='valid', data_format='channels_last', dilation_rate=1, activation=None, use_bias=True, kernel_loc_initializer=None, kernel_untransformed_scale_initializer=None, bias_initializer=None, kernel_prior_scale=1.0, kl_weight=1.0, seed=None, activity_regularizer=None, **kwargs)
Bases: keras_proba.layers.variational._ConvReparameterization
Bayesian 3D convolution (e.g. spatial convolution over volumes) with the reparameterization estimator [1].
One kernel W ~ q(W) is sampled per call.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
filters
|
Number of output channels. |
required | |
kernel_size
|
Int or tuple of ints, one per spatial dimension: the convolution window. |
required | |
strides
|
Int or tuple of ints, one per spatial dimension. |
1
|
|
padding
|
|
'valid'
|
|
data_format
|
|
'channels_last'
|
|
dilation_rate
|
Int or tuple of ints, one per spatial dimension. |
1
|
|
activation
|
Activation function. |
None
|
|
use_bias
|
Whether to add a (deterministic) bias. |
True
|
|
kernel_loc_initializer
|
Initializer for the posterior mean of the
kernel. Defaults to |
None
|
|
kernel_untransformed_scale_initializer
|
Initializer for the posterior
scale before |
None
|
|
bias_initializer
|
Initializer for the bias. Defaults to
|
None
|
|
kernel_prior_scale
|
Standard deviation of the Normal kernel prior. |
1.0
|
|
kl_weight
|
Multiplier of the KL divergence added to the losses, e.g.
|
1.0
|
|
seed
|
Seed for the random noise. |
None
|
|
activity_regularizer
|
Regularizer for the output. |
None
|
__doc__
class-attribute
¶
__doc__ = 'Bayesian 3D convolution (e.g. spatial convolution over volumes) with the reparameterization estimator [1].\n\n One kernel `W ~ q(W)` is sampled per call.\n\n Args:\n filters: Number of output channels.\n kernel_size: Int or tuple of ints, one per spatial dimension: the\n convolution window.\n strides: Int or tuple of ints, one per spatial dimension.\n padding: `"valid"` or `"same"`.\n data_format: `"channels_last"` or `"channels_first"`.\n dilation_rate: Int or tuple of ints, one per spatial dimension.\n activation: Activation function.\n use_bias: Whether to add a (deterministic) bias.\n kernel_loc_initializer: Initializer for the posterior mean of the\n kernel. Defaults to `RandomNormal(stddev=0.1)`.\n kernel_untransformed_scale_initializer: Initializer for the posterior\n scale before `softplus`. Defaults to `RandomNormal(mean=-3,\n stddev=0.1)`, a posterior close to a point mass.\n bias_initializer: Initializer for the bias. Defaults to\n `RandomNormal(stddev=0.1)`.\n kernel_prior_scale: Standard deviation of the Normal kernel prior.\n kl_weight: Multiplier of the KL divergence added to the losses, e.g.\n `1 / num_train_examples`.\n seed: Seed for the random noise.\n activity_regularizer: Regularizer for the output.\n '
str(object='') -> str str(bytes_or_buffer[, encoding[, errors]]) -> str
Create a new string object from the given object. If encoding or errors is specified, then the object must expose a data buffer that will be decoded using the given encoding and error handler. Otherwise, returns the result of object.str() (if defined) or repr(object). encoding defaults to sys.getdefaultencoding(). errors defaults to 'strict'.
__module__
class-attribute
¶
str(object='') -> str str(bytes_or_buffer[, encoding[, errors]]) -> str
Create a new string object from the given object. If encoding or errors is specified, then the object must expose a data buffer that will be decoded using the given encoding and error handler. Otherwise, returns the result of object.str() (if defined) or repr(object). encoding defaults to sys.getdefaultencoding(). errors defaults to 'strict'.
rank
class-attribute
¶
int([x]) -> integer int(x, base=10) -> integer
Convert a number or string to an integer, or return 0 if no arguments are given. If x is a number, return x.int(). For floating-point numbers, this truncates towards zero.
If x is not a number or if base is given, then x must be a string, bytes, or bytearray instance representing an integer literal in the given base. The literal can be preceded by '+' or '-' and be surrounded by whitespace. The base defaults to 10. Valid bases are 0 and 2-36. Base 0 means to interpret the base from the string as an integer literal.
int('0b100', base=0) 4
Conv3DFlipout ¶
Conv3DFlipout(filters, kernel_size, strides=1, padding='valid', data_format='channels_last', dilation_rate=1, activation=None, use_bias=True, kernel_loc_initializer=None, kernel_untransformed_scale_initializer=None, bias_initializer=None, kernel_prior_scale=1.0, kl_weight=1.0, seed=None, activity_regularizer=None, **kwargs)
Bases: keras_proba.layers.variational._ConvFlipout
Bayesian 3D convolution (e.g. spatial convolution over volumes) with the Flipout estimator [3].
Samples one kernel perturbation per call, decorrelated across examples with
random signs per example and channel:
outputs = conv(inputs, loc) + conv(inputs * s_in, dW) * s_out.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
filters
|
Number of output channels. |
required | |
kernel_size
|
Int or tuple of ints, one per spatial dimension: the convolution window. |
required | |
strides
|
Int or tuple of ints, one per spatial dimension. |
1
|
|
padding
|
|
'valid'
|
|
data_format
|
|
'channels_last'
|
|
dilation_rate
|
Int or tuple of ints, one per spatial dimension. |
1
|
|
activation
|
Activation function. |
None
|
|
use_bias
|
Whether to add a (deterministic) bias. |
True
|
|
kernel_loc_initializer
|
Initializer for the posterior mean of the
kernel. Defaults to |
None
|
|
kernel_untransformed_scale_initializer
|
Initializer for the posterior
scale before |
None
|
|
bias_initializer
|
Initializer for the bias. Defaults to
|
None
|
|
kernel_prior_scale
|
Standard deviation of the Normal kernel prior. |
1.0
|
|
kl_weight
|
Multiplier of the KL divergence added to the losses, e.g.
|
1.0
|
|
seed
|
Seed for the random noise. |
None
|
|
activity_regularizer
|
Regularizer for the output. |
None
|
__doc__
class-attribute
¶
__doc__ = 'Bayesian 3D convolution (e.g. spatial convolution over volumes) with the Flipout estimator [3].\n\n Samples one kernel perturbation per call, decorrelated across examples with\n random signs per example and channel:\n `outputs = conv(inputs, loc) + conv(inputs * s_in, dW) * s_out`.\n\n Args:\n filters: Number of output channels.\n kernel_size: Int or tuple of ints, one per spatial dimension: the\n convolution window.\n strides: Int or tuple of ints, one per spatial dimension.\n padding: `"valid"` or `"same"`.\n data_format: `"channels_last"` or `"channels_first"`.\n dilation_rate: Int or tuple of ints, one per spatial dimension.\n activation: Activation function.\n use_bias: Whether to add a (deterministic) bias.\n kernel_loc_initializer: Initializer for the posterior mean of the\n kernel. Defaults to `RandomNormal(stddev=0.1)`.\n kernel_untransformed_scale_initializer: Initializer for the posterior\n scale before `softplus`. Defaults to `RandomNormal(mean=-3,\n stddev=0.1)`, a posterior close to a point mass.\n bias_initializer: Initializer for the bias. Defaults to\n `RandomNormal(stddev=0.1)`.\n kernel_prior_scale: Standard deviation of the Normal kernel prior.\n kl_weight: Multiplier of the KL divergence added to the losses, e.g.\n `1 / num_train_examples`.\n seed: Seed for the random noise.\n activity_regularizer: Regularizer for the output.\n '
str(object='') -> str str(bytes_or_buffer[, encoding[, errors]]) -> str
Create a new string object from the given object. If encoding or errors is specified, then the object must expose a data buffer that will be decoded using the given encoding and error handler. Otherwise, returns the result of object.str() (if defined) or repr(object). encoding defaults to sys.getdefaultencoding(). errors defaults to 'strict'.
__module__
class-attribute
¶
str(object='') -> str str(bytes_or_buffer[, encoding[, errors]]) -> str
Create a new string object from the given object. If encoding or errors is specified, then the object must expose a data buffer that will be decoded using the given encoding and error handler. Otherwise, returns the result of object.str() (if defined) or repr(object). encoding defaults to sys.getdefaultencoding(). errors defaults to 'strict'.
rank
class-attribute
¶
int([x]) -> integer int(x, base=10) -> integer
Convert a number or string to an integer, or return 0 if no arguments are given. If x is a number, return x.int(). For floating-point numbers, this truncates towards zero.
If x is not a number or if base is given, then x must be a string, bytes, or bytearray instance representing an integer literal in the given base. The literal can be preceded by '+' or '-' and be surrounded by whitespace. The base defaults to 10. Valid bases are 0 and 2-36. Base 0 means to interpret the base from the string as an integer literal.
int('0b100', base=0) 4
Probabilistic regression heads¶
GaussianHead ¶
GaussianHead(units, min_scale=0.001, kernel_initializer='glorot_uniform', bias_initializer='zeros', scale_bias_initializer='zeros', kernel_regularizer=None, bias_regularizer=None, **kwargs)
Bases: keras.src.layers.layer.Layer
Predicts the mean and scale of a Gaussian for probabilistic regression.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
units
|
Number of regression targets. |
required | |
min_scale
|
Positive lower bound added to the predicted scale, for numerical stability of the likelihood. |
0.001
|
|
kernel_initializer
|
Initializer for both kernels. |
'glorot_uniform'
|
|
bias_initializer
|
Initializer for the mean bias. |
'zeros'
|
|
scale_bias_initializer
|
Initializer for the scale bias. The initial
scale is roughly |
'zeros'
|
|
kernel_regularizer
|
Regularizer for both kernels. |
None
|
|
bias_regularizer
|
Regularizer for both biases. |
None
|
Input shape
(..., input_dim).
Output shape
(..., 2 * units): the mean followed by the scale (standard deviation).
__doc__
class-attribute
¶
__doc__ = 'Predicts the mean and scale of a Gaussian for probabilistic regression.\n\n Args:\n units: Number of regression targets.\n min_scale: Positive lower bound added to the predicted scale, for\n numerical stability of the likelihood.\n kernel_initializer: Initializer for both kernels.\n bias_initializer: Initializer for the mean bias.\n scale_bias_initializer: Initializer for the scale bias. The initial\n scale is roughly `softplus(bias) + min_scale`.\n kernel_regularizer: Regularizer for both kernels.\n bias_regularizer: Regularizer for both biases.\n\n Input shape:\n `(..., input_dim)`.\n\n Output shape:\n `(..., 2 * units)`: the mean followed by the scale (standard deviation).\n '
str(object='') -> str str(bytes_or_buffer[, encoding[, errors]]) -> str
Create a new string object from the given object. If encoding or errors is specified, then the object must expose a data buffer that will be decoded using the given encoding and error handler. Otherwise, returns the result of object.str() (if defined) or repr(object). encoding defaults to sys.getdefaultencoding(). errors defaults to 'strict'.
__module__
class-attribute
¶
str(object='') -> str str(bytes_or_buffer[, encoding[, errors]]) -> str
Create a new string object from the given object. If encoding or errors is specified, then the object must expose a data buffer that will be decoded using the given encoding and error handler. Otherwise, returns the result of object.str() (if defined) or repr(object). encoding defaults to sys.getdefaultencoding(). errors defaults to 'strict'.
MixtureGaussianHead ¶
MixtureGaussianHead(num_components, units=1, min_scale=0.001, kernel_initializer='glorot_uniform', bias_initializer='zeros', scale_bias_initializer='zeros', kernel_regularizer=None, bias_regularizer=None, **kwargs)
Bases: keras.src.layers.layer.Layer
Predicts a mixture of Gaussians for probabilistic regression (mixture density network).
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
num_components
|
Number of mixture components |
required | |
units
|
Number of regression targets |
1
|
|
min_scale
|
Non-negative lower bound added to the component scales, for numerical stability of the likelihood. |
0.001
|
|
kernel_initializer
|
Initializer for the three kernels. |
'glorot_uniform'
|
|
bias_initializer
|
Initializer for the logit and mean biases. |
'zeros'
|
|
scale_bias_initializer
|
Initializer for the scale bias. Component
scales start near |
'zeros'
|
|
kernel_regularizer
|
Regularizer for the three kernels. |
None
|
|
bias_regularizer
|
Regularizer for the three biases. |
None
|
Input shape
(..., input_dim).
Output shape
(..., K * (1 + 2 * D)): mixture logits, then component means, then
component scales (standard deviations), each component's D values
contiguous.
__doc__
class-attribute
¶
__doc__ = "Predicts a mixture of Gaussians for probabilistic regression (mixture density network).\n\n Args:\n num_components: Number of mixture components `K`.\n units: Number of regression targets `D`.\n min_scale: Non-negative lower bound added to the component scales, for\n numerical stability of the likelihood.\n kernel_initializer: Initializer for the three kernels.\n bias_initializer: Initializer for the logit and mean biases.\n scale_bias_initializer: Initializer for the scale bias. Component\n scales start near `softplus(bias) + min_scale`.\n kernel_regularizer: Regularizer for the three kernels.\n bias_regularizer: Regularizer for the three biases.\n\n Input shape:\n `(..., input_dim)`.\n\n Output shape:\n `(..., K * (1 + 2 * D))`: mixture logits, then component means, then\n component scales (standard deviations), each component's `D` values\n contiguous.\n "
str(object='') -> str str(bytes_or_buffer[, encoding[, errors]]) -> str
Create a new string object from the given object. If encoding or errors is specified, then the object must expose a data buffer that will be decoded using the given encoding and error handler. Otherwise, returns the result of object.str() (if defined) or repr(object). encoding defaults to sys.getdefaultencoding(). errors defaults to 'strict'.
__module__
class-attribute
¶
str(object='') -> str str(bytes_or_buffer[, encoding[, errors]]) -> str
Create a new string object from the given object. If encoding or errors is specified, then the object must expose a data buffer that will be decoded using the given encoding and error handler. Otherwise, returns the result of object.str() (if defined) or repr(object). encoding defaults to sys.getdefaultencoding(). errors defaults to 'strict'.