HParams

class texar.HParams(hparams, default_hparams, allow_new_hparam=False)[source]

A class that maintains hyperparameters for configing Texar modules. The class has several useful features:

  • Auto-completion of missing values. Users can specify only a subset of hyperparameters they care about. Other hyperparameters will automatically take the default values. The auto-completion performs recursively so that hyperparameters taking dict values will also be auto-completed All Texar modules provide a default_hparams() containing allowed hyperparameters and their default values. For example

    ## Recursive auto-completion
    default_hparams = {"a": 1, "b": {"c": 2, "d": 3}}
    hparams = {"b": {"c": 22}}
    hparams_ = HParams(hparams, default_hparams)
    hparams_.todict() == {"a": 1, "b": {"c": 22, "d": 3}}
        # "a" and "d" are auto-completed
    
    ## All Texar modules have built-in `default_hparams`
    hparams = {"dropout_rate": 0.1}
    emb = tx.modules.WordEmbedder(hparams=hparams, ...)
    emb.hparams.todict() == {
        "dropout_rate": 0.1,  # provided value
        "dim": 100            # default value
        ...
    }
    
  • Automatic typecheck. For most hyperparameters, provided value must have the same or compatible dtype with the default value. HParams does necessary typecheck, and raises Error if improper dtype is provided. Also, hyperparameters not listed in default_hparams are not allowed, except for “kwargs” as detailed below.

  • Flexible dtype for specified hyperparameters. Some hyperparameters may allow different dtypes of values.

    • Hyperparameters named “type” are not typechecked. For example, in get_rnn_cell(), hyperparameter “type” can take value of an RNNCell class, its string name of module path, or an RNNCell class instance. (String name or module path is allowd so that users can specify the value in YAML config files.)
    • For other hyperparameters, list them in the “@no_typecheck” field in default_hparams to skip typecheck. For example, in get_rnn_cell(), hyperparameter “*_keep_prob” can be set to either a float or a tf.placeholder.
  • Special flexibility of keyword argument hyparameters. Hyperparameters named “kwargs” are used as keyword arguments for a class constructor or a function call. Such hyperparameters take a dict, and users can add arbitrary valid keyword arguments to the dict. For example:

    default_rnn_cell_hparams = {
        "type": "LSTMCell",
        "kwargs": {"num_units": 256}
        # Other hyperparameters
        ...
    }
    my_hparams = {
        "kwargs" {
            "num_units": 123,
            "forget_bias": 0.0         # Other valid keyword arguments
            "activation": "tf.nn.relu" # for LSTMCell constructor
        }
    }
    _ = HParams(my_hparams, default_rnn_cell_hparams)
    
  • Rich interfaces. An HParams instance provides rich interfaces for accessing, updating, or adding hyperparameters.

    hparams = HParams(my_hparams, default_hparams)
    # Access
    hparams.type == hparams["type"]
    # Update
    hparams.type = "GRUCell"
    hparams.kwargs = { "num_units": 100 }
    hparams.kwargs.num_units == 100
    # Add new
    hparams.add_hparam("index", 1)
    hparams.index == 1
    
    # Convert to `dict` (recursively)
    type(hparams.todic()) == dict
    
    # I/O
    pickle.dump(hparams, "hparams.dump")
    with open("hparams.dump", 'rb') as f:
        hparams_loaded = pickle.load(f)
    
Parameters:
  • hparams – A dict or an HParams instance containing hyperparameters. If None, all hyperparameters are set to default values.
  • default_hparams (dict) – Hyperparameters with default values. If None, Hyperparameters are fully defined by hparams.
  • allow_new_hparam (bool) – If False (default), hparams cannot contain hyperparameters that are not included in default_hparams, except for the case of "kwargs" as above.
items()[source]

Returns the list of hyperparam (name, value) pairs

keys()[source]

Returns the list of hyperparam names

get(name, default=None)[source]

Returns the hyperparameter value for the given name. If name is not available then returns default.

Parameters:
  • name (str) – the name of hyperparameter.
  • default – the value to be returned in case name does not exist.
add_hparam(name, value)[source]

Adds a new hyperparameter.

todict()[source]

Returns a copy of hyperparameters as a dictionary.