Utils¶

Frequent Use¶

AverageRecorder¶

class texar.tf.utils.AverageRecorder(size=None)[source]

Maintains the moving averages (i.e., the average of the latest N records) of (possibly multiple) fields.

Fields are determined by the first call of add().

Parameters:	size (int, optional) – The window size of moving average. If None, the average of all added records is maintained.

Example

## Use to maintain moving average of training loss
avg_rec = AverageRecorder(size=10) # average over latest 10 records
while training:
    loss_0, loss_1  = ...
    avg_rec.add([loss_0, loss_1])
    # avg_rec.avg() == [0.12343452, 0.567800323]
    # avg_rec.avg(0) == 0.12343452
    # avg_rec.to_str(precision=2, ) == '0.12 0.57'

## Use to maintain average of test metrics on the whole test set
avg_rec = AverageRecorder() # average over ALL records
while test:
    metric_0, metric_1  = ...
    avg_rec.add({'m0': metric_0, 'm1': metric_1}) # dict is allowed
print(avg_rec.to_str(precision=4, delimiter=' , '))
# 'm0: 0.1234 , m1: 0.5678'
#
# avg_rec.avg() == {'m0': 0.12343452, 'm1': 0.567800323}
# avg_rec.avg(0) == 0.12343452

add(record, weight=None)[source]

Appends a new record.

record can be a list, dict, or a single scalar. The record type is determined at the first time add() is called. All subsequent calls to add() must have the same type of record.

record in subsequent calls to add() can contain only a subset of fields than the first call to add().

Example

recorder.add({'1': 0.2, '2': 0.2}) # 1st call to `add`
x = recorder.add({'1': 0.4}) # 2nd call to `add`
# x == {'1': 0.3, '2': 0.2}

Parameters:	record – A single scalar, a list of scalars, or a dict of scalars. weight (optional) – A scalar, weight of the new record for calculating a weighted average. If None, weight is set to 1. For example, `weight` can be set to batch size and `record` the average value of certain metrics on the batch in order to calculate the average metric values on a whole dataset.
Returns:	The (moving) average after appending the record, with the same type as `record`.

avg(id_or_name=None)[source]

Returns the (moving) average.

Parameters:	id_or_name (optional) – A list of or a single element. Each element is the index (if the record type is list) or name (if the record type is dict) of the field for which the average is calculated. If not given, the average of all fields are returned.
Returns:	The average value(s). If `id_or_name` is a single element (not a list), then returns the average value of the corresponding field. Otherwise, if `id_or_name` is a list of element(s), then returns average value(s) in the same type as `record` of `add()`.

reset(id_or_name=None)[source]

Resets the record.

Parameters:	id_or_name (optional) – A list or a single element. Each element is the index (if the record type is list) or name (if the record type is dict) of the field to reset. If None, all fields are reset.

to_str(precision=None, delimiter=' ')[source]

Returns a string of the average values of the records.

Parameters:

precision (int, optional) – The number of decimal places to keep in the returned string. E.g., for an average value of 0.1234, precision = 2 leads to ‘0.12’.
delimiter (str) – The delimiter string that separates between fields.

Returns:

A string of the average values.

If record is of type dict, the string is a concatenation of ‘field_name: average_value’, delimited with delimiter. E.g., ‘field_name_1: 0.1234 field_name_2: 0.5678 …’.

Otherwise, the string is of a concatenation of ‘average_value’. E.g., ‘0.1234 0.5678 …’

collect_trainable_variables¶

texar.tf.utils.collect_trainable_variables(modules)[source]

Collects all trainable variables of modules.

Trainable variables included in multiple modules occur only once in the returned list.

Parameters:	modules – A (list of) instance of the subclasses of `ModuleBase`.
Returns:	A list of trainable variables in the modules.

compat_as_text¶

texar.tf.utils.compat_as_text(str_)[source]

Converts strings into unicode (Python 2) or str (Python 3).

Parameters:	str – A string or other data types convertible to string, or an n-D numpy array or (possibly nested) list of such elements.
Returns:	The converted strings of the same structure/shape as `str_`.

map_ids_to_strs¶

texar.tf.utils.map_ids_to_strs(ids, vocab, join=True, strip_pad='<PAD>', strip_bos='<BOS>', strip_eos='<EOS>', compat=True)[source]

Transforms int indexes to strings by mapping ids to tokens, concatenating tokens into sentences, and stripping special tokens, etc.

Parameters:

ids – An n-D numpy array or (possibly nested) list of int indexes.
vocab – An instance of Vocab.
join (bool) – Whether to concat along the last dimension of the the tokens into a string separated with a space character.
strip_pad (str) – The PAD token to strip from the strings (i.e., remove the leading and trailing PAD tokens of the strings). Default is ‘<PAD>’ as defined in SpecialTokens.PAD. Set to None or False to disable the stripping.
strip_bos (str) – The BOS token to strip from the strings (i.e., remove the leading BOS tokens of the strings). Default is ‘<BOS>’ as defined in SpecialTokens.BOS. Set to None or False to disable the stripping.
strip_eos (str) – The EOS token to strip from the strings (i.e., remove the EOS tokens and all subsequent tokens of the strings). Default is ‘<EOS>’ as defined in SpecialTokens.EOS. Set to None or False to disable the stripping.

Returns:

If join is True, returns a (n-1)-D numpy array (or list) of concatenated strings. If join is False, returns an n-D numpy array (or list) of str tokens.

Example

text_ids = [[1, 9, 6, 2, 0, 0], [1, 28, 7, 8, 2, 0]]

text = map_ids_to_strs(text_ids, data.vocab)
# text == ['a sentence', 'parsed from ids']

text = map_ids_to_strs(
    text_ids, data.vocab, join=False,
    strip_pad=None, strip_bos=None, strip_eos=None)
# text == [['<BOS>', 'a', 'sentence', '<EOS>', '<PAD>', '<PAD>'],
#          ['<BOS>', 'parsed', 'from', 'ids', '<EOS>', '<PAD>']]

write_paired_text¶

texar.tf.utils.write_paired_text(src, tgt, fname, append=False, mode='h', sep='\t', src_fname_suffix='src', tgt_fname_suffix='tgt')[source]

Writes paired text to a file.

Parameters:

src – A list (or array) of str source text.
tgt – A list (or array) of str target text.
fname (str) – The output filename.
append (bool) – Whether append content to the end of the file if exists.
mode (str) –
The mode of writing, with the following options:
- ’h’: The “horizontal” mode. Each source target pair is written in one line, intervened with sep, e.g.:
```
source_1 target_1
source_2 target_2
```
- ’v’: The “vertical” mode. Each source target pair is written in two consecutive lines, e.g:
```
source_1
target_1
source_2
target_2
```
- ’s’: The “separate” mode. Each source target pair is written in corresponding lines of two files named as “{fname}.{src_fname_suffix}” and “{fname}.{tgt_fname_suffix}”, respectively.
sep (str) – The string intervening between source and target. Used when mode is set to ‘h’.
src_fname_suffix (str) – Used when mode is ‘s’. The suffix to the source output filename. E.g., with (fname=’output’, src_fname_suffix=’src’), the output source file is named as output.src.
tgt_fname_suffix (str) – Used when mode is ‘s’. The suffix to the target output filename.

Returns:

The fileanme(s). If mode == ‘h’ or ‘v’, returns fname. If mode == ‘s’, returns a list of filenames [“{fname}.src”, “{fname}.tgt”].

straight_through¶

texar.tf.utils.straight_through(fw_tensor, bw_tensor)[source]

Use a tensor in forward pass while backpropagating gradient to another.

Parameters:	fw_tensor – A tensor to be used in the forward pass. bw_tensor – A tensor to which gradient is backpropagated. Must have the same shape and type with `fw_tensor`.
Returns:	A tensor of the same shape and value with `fw_tensor` but will direct gradient to bw_tensor.

Variables¶

collect_trainable_variables¶

texar.tf.utils.collect_trainable_variables(modules)[source]¶

Collects all trainable variables of modules.

Trainable variables included in multiple modules occur only once in the returned list.

Parameters:	modules – A (list of) instance of the subclasses of `ModuleBase`.
Returns:	A list of trainable variables in the modules.

get_unique_named_variable_scope¶

texar.tf.utils.get_unique_named_variable_scope(base_name)[source]¶

Returns a variable scope with a unique name.

Parameters:	base_name (str) – The base name to uniquified.
Returns:	An instance of variable_scope.

Example

vs = get_unique_named_variable_scope('base_name')
with tf.variable_scope(vs):
    ....

add_variable¶

texar.tf.utils.add_variable(variable, var_list)[source]¶

Adds variable to a given list.

Parameters:	variable – A (list of) variable(s). var_list (list) – The list where the `variable` are added to.

IO¶

write_paired_text¶

texar.tf.utils.write_paired_text(src, tgt, fname, append=False, mode='h', sep='\t', src_fname_suffix='src', tgt_fname_suffix='tgt')[source]¶

Writes paired text to a file.

Parameters:

src – A list (or array) of str source text.
tgt – A list (or array) of str target text.
fname (str) – The output filename.
append (bool) – Whether append content to the end of the file if exists.
mode (str) –
The mode of writing, with the following options:
- ’h’: The “horizontal” mode. Each source target pair is written in one line, intervened with sep, e.g.:
```
source_1 target_1
source_2 target_2
```
- ’v’: The “vertical” mode. Each source target pair is written in two consecutive lines, e.g:
```
source_1
target_1
source_2
target_2
```
- ’s’: The “separate” mode. Each source target pair is written in corresponding lines of two files named as “{fname}.{src_fname_suffix}” and “{fname}.{tgt_fname_suffix}”, respectively.
sep (str) – The string intervening between source and target. Used when mode is set to ‘h’.
src_fname_suffix (str) – Used when mode is ‘s’. The suffix to the source output filename. E.g., with (fname=’output’, src_fname_suffix=’src’), the output source file is named as output.src.
tgt_fname_suffix (str) – Used when mode is ‘s’. The suffix to the target output filename.

Returns:

The fileanme(s). If mode == ‘h’ or ‘v’, returns fname. If mode == ‘s’, returns a list of filenames [“{fname}.src”, “{fname}.tgt”].

load_config¶

texar.tf.utils.load_config(config_path, config=None)[source]¶

Loads configs from (possibly multiple) file(s).

A config file can be either a Python file (with suffix ‘.py’) or a YAML file. If the filename is not suffixed with ‘.py’, the file is parsed as YAML.

Parameters:	config_path – Paths to configuration files. This can be a list of config file names, or a path to a directory in which all files are loaded, or a string of multiple file names separated by commas. config (dict, optional) – A config dict to which new configurations are added. If None, a new config dict is created.
Returns:	A dict of configurations.

maybe_create_dir¶

texar.tf.utils.maybe_create_dir(dirname)[source]¶: Creates directory if doesn’t exist

get_files¶

texar.tf.utils.get_files(file_paths)[source]¶

Gets a list of file paths given possibly a pattern file_paths.

Adapted from tf.contrib.slim.data.parallel_reader.get_data_files.

Parameters:	file_paths – A (list of) path to the files. The path can be a pattern, e.g., /path/to/train*, /path/to/train[12]
Returns:	A list of file paths.
Raises:	`ValueError` – If no files are not found

DType¶

compat_as_text¶

texar.tf.utils.compat_as_text(str_)[source]¶

Converts strings into unicode (Python 2) or str (Python 3).

Parameters:	str – A string or other data types convertible to string, or an n-D numpy array or (possibly nested) list of such elements.
Returns:	The converted strings of the same structure/shape as `str_`.

get_tf_dtype¶

texar.tf.utils.get_tf_dtype(dtype)[source]¶

Returns equivalent tf dtype.

Parameters:	dtype – A str, python numeric or string type, numpy data type, or tf dtype.
Returns:	The corresponding tf dtype.

is_callable¶

texar.tf.utils.is_callable(x)[source]¶: Return True if x is callable.

is_str¶

texar.tf.utils.is_str(x)[source]¶: Returns True if x is either a str or unicode. Returns False otherwise.

is_placeholder¶

texar.tf.utils.is_placeholder(x)[source]¶: Returns True if x is a tf.placeholder or tf.placeholder_with_default.

maybe_hparams_to_dict¶

texar.tf.utils.maybe_hparams_to_dict(hparams)[source]¶: If hparams is an instance of HParams, converts it to a dict and returns. If hparams is a dict, returns as is.

Shape¶

mask_sequences¶

texar.tf.utils.mask_sequences(sequence, sequence_length, dtype=None, time_major=False, tensor_rank=2)[source]¶

Masks out sequence entries that are beyond the respective sequence lengths. Masks along the time dimension.

sequence and sequence_length can either be python arrays or Tensors, respectively. If both are python arrays (or None), the return will be a python array as well.

Parameters:

sequence – A Tensor or python array of sequence values. If time_major==False (default), this must be a Tensor of shape [batch_size, max_time, …]. The batch and time dimension is exchanged if time_major==True.
sequence_length – A Tensor or python array of shape [batch_size]. Time steps beyond the respective sequence lengths will be made zero.
dtype (dtype) – Type of sequence. If None, infer from sequence automatically.
time_major (bool) – The shape format of the inputs. If True, sequence must have shape [max_time, batch_size, …]. If False (default), sequence must have shape [batch_size, max_time, …].
tensor_rank (int) – The number of dimensions of sequence. Default is 2, i.e., sequence is a 2D Tensor consisting of batch and time dimensions. Ignored if both sequence and sequence_length are python arrays.

Returns:

The masked sequence, i.e., a Tensor or python array of the same shape as sequence but with masked-out entries (set to zero).

If both sequence and sequence_length are python arrays, the returned value is a python array as well.

transpose_batch_time¶

texar.tf.utils.transpose_batch_time(inputs)[source]¶

Transposes inputs between time-major and batch-major.

Parameters:	inputs – A Tensor of shape [batch_size, max_time, …] (batch-major) or [max_time, batch_size, …] (time-major), or a (possibly nested) tuple of such elements.
Returns:	A (possibly nested tuple of) Tensor with transposed batch and time dimensions of inputs.

get_batch_size¶

texar.tf.utils.get_batch_size(tensor)[source]¶: Returns a unit Tensor representing the batch size, i.e., the size of the 1st dimension of tensor.

get_rank¶

texar.tf.utils.get_rank(tensor)[source]¶

Returns the tensor rank as a python int. The input tensor can also be a python array.

Parameters:	tensor – A Tensor or python array.
Returns:	A python int representing the rank of `tensor`. Returns None if the rank cannot be determined.

shape_list¶

texar.tf.utils.shape_list(x)[source]¶

Returns static shape of the input Tensor whenever possible.

Parameters:	x – A Tensor.
Returns:	If the rank of x is unknown, returns the dynamic shape `tf.shape(x)` Otherwise, returns a list of dims, each of which is either an int whenever it can be statically determined, or a scalar Tensor otherwise.

pad_and_concat¶

texar.tf.utils.pad_and_concat(values, axis, rank=None, pad_axis=None, pad_constant_values=0)[source]¶

Concats tensors along one dimension. Pads each of other dimensions of the tensors to the corresponding maximum size if necessary.

Parameters:	values – A list of Tensors of the same rank. axis (int) – A Python int. Dimension along which to concatenate. rank (int, optional) – Rank of the tensors. If None, inferred automatically from `values`. pad_axis (int or list, optional) – A Python int or a list of int. Dimensions to pad. Paddings are only added to the end of corresponding dimensions. If None, all dimensions except the `axis` dimension are padded. pad_constant_values – The scalar pad value to use. Must be same type as the tensors.
Returns:	A Tensor resulting from padding and concatenation of the input tensors.
Raises:	`ValueError` – If `rank` is None and cannot be inferred from `values`.

Example

a = tf.ones([1, 2])
b = tf.ones([2, 3])

c = pad_and_concat([a,b], 0)
# c.shape == [3, 3]
# c == [[1, 1, 0],
#       [1, 1, 1],
#       [1, 1, 1]]

d = pad_and_concat([a,b], 1)
# d.shape == [2, 5]
# d == [[1, 1, 1, 1, 1]
#       [0, 0, 1, 1, 1]]

reduce_with_weights¶

texar.tf.utils.reduce_with_weights(tensor, weights=None, average_across_batch=True, average_across_remaining=False, sum_over_batch=False, sum_over_remaining=True, tensor_rank=None)[source]¶

Weights and reduces tensor.

Parameters:

tensor – A Tensor to weight and reduce, of shape [batch_size, …].
weights (optional) – A Tensor of the same shape and dtype with tensor. For example, this is can be a 0-1 tensor for masking values of tensor`.
average_across_batch (bool) – If set, average the tensor across the batch dimension. Must not set average_across_batch’ and sum_over_batch at the same time.
average_across_remaining (bool) – If set, average the tensor across the remaining dimensions. Must not set average_across_remaining’ and sum_over_remaining at the same time. If weights is given, this is a weighted average.
sum_over_batch (bool) – If set, sum the tensor across the batch dimension. Must not set average_across_batch and sum_over_batch at the same time.
sum_over_remaining (bool) – If set, sum the tensor across the remaining dimension. Must not set average_across_remaining and sum_over_remaining at the same time. If weights is given, this is a weighted sum.
tensor_rank (int, optional) – The number of dimensions of tensor. If not given, inferred from tensor automatically.

Returns:

A Tensor.

Example

x = tf.constant([[10, 10, 2, 2],
                 [20, 2, 2, 2]])
mask = tf.constant([[1, 1, 0, 0],
                    [1, 0, 0, 0]])

z = reduce_with_weights(x, weights=mask)
# z == 20
# (all 2 in x are masked)

flatten¶

texar.tf.utils.flatten(tensor, preserve_dims, flattened_dim=None)[source]¶

Flattens a tensor whiling keeping several leading dimensions.

preserve_dims must < tensor’s rank

Parameters:	tensor – A Tensor to flatten. preserve_dims (int) – The number of leading dimensions to preserve. flatterned_dim (int, optional) – The size of the resulting flattened dimension. If not given, infer automatically, which can cause a statically unknown dimension size.
Returns:	A Tensor with rank `perserve_dims` + 1.

Example

x = tf.ones(shape=[d_1, d_2, d_3, d_4])
y = flatten(x, 2) # y.shape == [d_1, d_2, d_3 * d_4]

varlength_concat¶

texar.tf.utils.varlength_concat(x, y, x_length, dtype=None, tensor_rank=None)[source]¶

Concatenates rows of x and y where each row of x has a variable length.

Both x and y are of numeric dtypes, such as tf.int32 and tf.float32, with mask value 0. The two tensors must be of the same dtype.

Parameters:

x – A tensor of shape [batch_size, x_dim_2, other_dims].
y – A tensor of shape [batch_size, y_dim_2, other_dims]. All dimensions except the 2nd dimension must be the same with those of x.
x_length – A 1D int tensor of shape [batch_size] containing the length of each x row. Elements beyond the respective lengths will be made zero.
dtype – Type of x. If None, inferred from x automatically.
tensor_rank (int, optional) – The number of dimensions of x. If not given, inferred from x automatically.

Returns:

A Tensor of shape [batch_size, x_dim_2 + y_dim_2, other_dims].

Example

x = tf.constant([[1, 1, 0, 0],
                 [1, 1, 1, 0]])
x_length = [2, 3]
y = tf.constant([[2, 2, 0],
                 [2, 2, 2]])

out = varlength_concat(x, y, x_length)
# out = [[1, 1, 2, 2, 0, 0, 0]
#        [1, 1, 1, 2, 2, 2, 0]]

varlength_concat_py¶

texar.tf.utils.varlength_concat_py(x, y, x_length, dtype=None)[source]¶

Concatenates rows of x and y where each row of x has a variable length.

The function has the same semantic as varlength_concat(), except that this function is for numpy arrays instead of TF tensors.

Both x and y are of numeric dtypes, such as int32 and float32, with mask value 0. The two arrays must be of the same dtype.

Parameters:

x – A array of shape [batch_size, x_dim_2, other_dims].
y – A array of shape [batch_size, y_dim_2, other_dims]. All dimensions except the 2nd dimension must be the same with those of x.
x_length – A 1D int array of shape [batch_size] containing the length of each x row. Elements beyond the respective lengths will be made zero.
dtype – Type of x. If None, inferred from x automatically.

Returns:

An array of shape [batch_size, x_dim_2 + y_dim_2, other_dims].

Example

x = np.asarray([[1, 1, 0, 0],
                [1, 1, 1, 0]])
x_length = [2, 3]
y = np.asarray([[2, 2, 0],
                [2, 2, 2]])

out = varlength_concat_py(x, y, x_length)
# out = [[1, 1, 2, 2, 0, 0, 0]
#        [1, 1, 1, 2, 2, 2, 0]]

varlength_roll¶

texar.tf.utils.varlength_roll(input, shift, axis=1, dtype=None)[source]¶

Rolls the elements of each row of a tensor along an axis for variable steps.

This is a tf.while_loop wrapper of tf.roll. Note the different definition of shift and axis here compared to tf.roll.

Parameters:

input – A tensor of shape [batch_size, other_dims] where other_dims can be multiple dimensions.
shift – A 1D int tensor of shape [batch_size] containing the steps for which each row in the batch are rolled. Positive shifts will roll towards larger indices, while negative shifts will roll towards smaller indices.
axis – A scalar int tensor > 0. The dimension that the roll should occur.
dtype – Type of input. If None, inferred from input automatically.

Returns:

A Tensor of the same shape/dtype as input.

Example

x = tf.constant([[0, 0, 1, 0],
                 [0, 1, 1, 1]])
shift = [-2, -1]

out = varlength_roll(x, shift)
# out = [[1, 0, 0, 0]
#        [1, 1, 1, 0]]

x = tf.constant([[1, 2, 3, 4],
                 [5, 6, 7, 8]])
shift = [1, -1]

out = varlength_roll(x, shift)
# out = [[4, 1, 2, 3]
#        [6, 7, 8, 5]]

Dictionary¶

dict_patch¶

texar.tf.utils.dict_patch(tgt_dict, src_dict)[source]¶

Recursively patch tgt_dict by adding items from src_dict that do not exist in tgt_dict.

If respective items in src_dict and tgt_dict are both dict, the tgt_dict item is patched recursively.

Parameters:	tgt_dict (dict) – Target dictionary to patch. src_dict (dict) – Source dictionary.
Returns:	The new `tgt_dict` that is patched.
Return type:	dict

dict_lookup¶

texar.tf.utils.dict_lookup(dict_, keys, default=None)[source]¶

Looks up keys in the dict, returns the corresponding values.

The default is used for keys not present in the dict.

Parameters:	dict (dict) – A dictionary for lookup. keys – A numpy array or a (possibly nested) list of keys. default (optional) – Value to be returned when a key is not in `dict_`. Error is raised if `default` is not given and key is not in the dict.
Returns:	A numpy array of values with the same structure as `keys`.
Raises:	`TypeError` – If key is not in `dict_` and `default` is None.

dict_fetch¶

texar.tf.utils.dict_fetch(src_dict, tgt_dict_or_keys)[source]¶

Fetches a sub dict of src_dict with the keys in tgt_dict_or_keys.

Parameters:	src_dict – A dict or instance of `HParams`. The source dict to fetch values from. tgt_dict_or_keys – A dict, instance of `HParams`, or a list (or a dict_keys) of keys to be included in the output dict.
Returns:	A new dict that is a subdict of `src_dict`.

dict_pop¶

texar.tf.utils.dict_pop(dict_, pop_keys, default=None)[source]¶

Removes keys from a dict and returns their values.

Parameters:	dict (dict) – A dictionary from which items are removed. pop_keys – A key or a list of keys to remove and return respective values or `default`. default (optional) – Value to be returned when a key is not in `dict_`. The default value is None.
Returns:	A dict of the items removed from `dict_`.

flatten_dict¶

texar.tf.utils.flatten_dict(dict_, parent_key='', sep='.')[source]¶

Flattens a nested dictionary. Namedtuples within the dictionary are converted to dicts.

Adapted from: https://github.com/google/seq2seq/blob/master/seq2seq/models/model_base.py

Parameters:	dict (dict) – The dictionary to flatten. parent_key (str) – A prefix to prepend to each key. sep (str) – Separator that intervenes between parent and child keys. E.g., if sep == ‘.’, then { “a”: { “b”: 3 } } is converted into { “a.b”: 3 }.
Returns:	A new flattened dict.

String¶

map_ids_to_strs¶

texar.tf.utils.map_ids_to_strs(ids, vocab, join=True, strip_pad='<PAD>', strip_bos='<BOS>', strip_eos='<EOS>', compat=True)[source]¶

Transforms int indexes to strings by mapping ids to tokens, concatenating tokens into sentences, and stripping special tokens, etc.

Parameters:

ids – An n-D numpy array or (possibly nested) list of int indexes.
vocab – An instance of Vocab.
join (bool) – Whether to concat along the last dimension of the the tokens into a string separated with a space character.
strip_pad (str) – The PAD token to strip from the strings (i.e., remove the leading and trailing PAD tokens of the strings). Default is ‘<PAD>’ as defined in SpecialTokens.PAD. Set to None or False to disable the stripping.
strip_bos (str) – The BOS token to strip from the strings (i.e., remove the leading BOS tokens of the strings). Default is ‘<BOS>’ as defined in SpecialTokens.BOS. Set to None or False to disable the stripping.
strip_eos (str) – The EOS token to strip from the strings (i.e., remove the EOS tokens and all subsequent tokens of the strings). Default is ‘<EOS>’ as defined in SpecialTokens.EOS. Set to None or False to disable the stripping.

Returns:

If join is True, returns a (n-1)-D numpy array (or list) of concatenated strings. If join is False, returns an n-D numpy array (or list) of str tokens.

Example

text_ids = [[1, 9, 6, 2, 0, 0], [1, 28, 7, 8, 2, 0]]

text = map_ids_to_strs(text_ids, data.vocab)
# text == ['a sentence', 'parsed from ids']

text = map_ids_to_strs(
    text_ids, data.vocab, join=False,
    strip_pad=None, strip_bos=None, strip_eos=None)
# text == [['<BOS>', 'a', 'sentence', '<EOS>', '<PAD>', '<PAD>'],
#          ['<BOS>', 'parsed', 'from', 'ids', '<EOS>', '<PAD>']]

strip_token¶

texar.tf.utils.strip_token(str_, token, is_token_list=False, compat=True)[source]¶

Returns a copy of strings with leading and trailing tokens removed.

Note that besides token, all leading and trailing whitespace characters are also removed.

If is_token_list is False, then the function assumes tokens in str_ are separated with whitespace character.

Parameters:

str – A str, or an n-D numpy array or (possibly nested) list of str.
token (str) – The token to strip, e.g., the ‘<PAD>’ token defined in SpecialTokens.PAD
is_token_list (bool) – Whether each sentence in str_ is a list of tokens. If False, each sentence in str_ is assumed to contain tokens separated with space character.
compat (bool) – Whether to convert tokens into unicode (Python 2) or str (Python 3).

Returns:

The stripped strings of the same structure/shape as str_.

Example

str_ = '<PAD> a sentence <PAD> <PAD>  '
str_stripped = strip_token(str_, '<PAD>')
# str_stripped == 'a sentence'

str_ = ['<PAD>', 'a', 'sentence', '<PAD>', '<PAD>', '', '']
str_stripped = strip_token(str_, '<PAD>', is_token_list=True)
# str_stripped == 'a sentence'

strip_eos¶

texar.tf.utils.strip_eos(str_, eos_token='<EOS>', is_token_list=False, compat=True)[source]¶

Remove the EOS token and all subsequent tokens.

If is_token_list is False, then the function assumes tokens in str_ are separated with whitespace character.

Parameters:

str – A str, or an n-D numpy array or (possibly nested) list of str.
eos_token (str) – The EOS token. Default is ‘<EOS>’ as defined in SpecialTokens.EOS
is_token_list (bool) – Whether each sentence in str_ is a list of tokens. If False, each sentence in str_ is assumed to contain tokens separated with space character.
compat (bool) – Whether to convert tokens into unicode (Python 2) or str (Python 3).

Returns:

Strings of the same structure/shape as str_.

strip_special_tokens¶

texar.tf.utils.strip_special_tokens(str_, strip_pad='<PAD>', strip_bos='<BOS>', strip_eos='<EOS>', is_token_list=False, compat=True)[source]¶

Removes special tokens in strings, including:

Removes EOS and all subsequent tokens

Removes leading and and trailing PAD tokens

Removes leading BOS tokens

Note that besides the special tokens, all leading and trailing whitespace characters are also removed.

This is a joint function of strip_eos(), strip_pad(), and strip_bos()

Parameters:

str – A str, or an n-D numpy array or (possibly nested) list of str.
strip_pad (str) – The PAD token to strip from the strings (i.e., remove the leading and trailing PAD tokens of the strings). Default is ‘<PAD>’ as defined in SpecialTokens.PAD. Set to None or False to disable the stripping.
strip_bos (str) – The BOS token to strip from the strings (i.e., remove the leading BOS tokens of the strings). Default is ‘<BOS>’ as defined in SpecialTokens.BOS. Set to None or False to disable the stripping.
strip_eos (str) – The EOS token to strip from the strings (i.e., remove the EOS tokens and all subsequent tokens of the strings). Default is ‘<EOS>’ as defined in SpecialTokens.EOS. Set to None or False to disable the stripping.
is_token_list (bool) – Whether each sentence in str_ is a list of tokens. If False, each sentence in str_ is assumed to contain tokens separated with space character.
compat (bool) – Whether to convert tokens into unicode (Python 2) or str (Python 3).

Returns:

Strings of the same shape of str_ with special tokens stripped.

str_join¶

texar.tf.utils.str_join(tokens, sep=' ', compat=True)[source]¶

Concats tokens along the last dimension with intervening occurrences of sep.

Parameters:	tokens – An n-D numpy array or (possibly nested) list of str. sep (str) – The string intervening between the tokens. compat (bool) – Whether to convert tokens into unicode (Python 2) or str (Python 3).
Returns:	An (n-1)-D numpy array (or list) of str.

default_str¶

texar.tf.utils.default_str(str_, default_str)[source]¶

Returns str_ if it is not None or empty, otherwise returns default_str.

Parameters:	str – A string. default_str – A string.
Returns:	Either `str_` or `default_str`.

uniquify_str¶

texar.tf.utils.uniquify_str(str_, str_set)[source]¶

Uniquifies str_ if str_ is included in str_set.

This is done by appending a number to str_. Returns str_ directly if it is not included in str_set.

Parameters:	str (string) – A string to uniquify. str_set (set, dict, or list) – A collection of strings. The returned string is guaranteed to be different from the elements in the collection.
Returns:	The uniquified string. Returns `str_` directly if it is already unique.

Example

print(uniquify_str('name', ['name', 'name_1']))
# 'name_2'

Meta¶

check_or_get_class¶

texar.tf.utils.check_or_get_class(class_or_name, module_path=None, superclass=None)[source]¶

Returns the class and checks if the class inherits superclass.

Parameters:	class_or_name – Name or full path to the class, or the class itself. module_paths (list, optional) – Paths to candidate modules to search for the class. This is used if `class_or_name` is a string and the class cannot be located solely based on `class_or_name`. The first module in the list that contains the class is used. superclass (optional) – A (list of) classes that the target class must inherit.
Returns:	The target class.
Raises:	`ValueError` – If class is not found based on `class_or_name` and `module_paths`. `TypeError` – If class does not inherits `superclass`.

get_class¶

texar.tf.utils.get_class(class_name, module_paths=None)[source]¶

Returns the class based on class name.

Parameters:	class_name (str) – Name or full path to the class. module_paths (list) – Paths to candidate modules to search for the class. This is used if the class cannot be located solely based on class_name. The first module in the list that contains the class is used.
Returns:	The target class.
Raises:	`ValueError` – If class is not found based on `class_name` and `module_paths`.

check_or_get_instance¶

texar.tf.utils.check_or_get_instance(ins_or_class_or_name, kwargs, module_paths=None, classtype=None)[source]¶

Returns a class instance and checks types.

Parameters:

ins_or_class_or_name –
Can be of 3 types:
- A class to instantiate.
- A string of the name or full path to a class to instantiate.
- The class instance to check types.
kwargs (dict) – Keyword arguments for the class constructor. Ignored if ins_or_class_or_name is a class instance.
module_paths (list, optional) – Paths to candidate modules to search for the class. This is used if the class cannot be located solely based on class_name. The first module in the list that contains the class is used.
classtype (optional) – A (list of) class of which the instance must be an instantiation.

Raises:

ValueError – If class is not found based on class_name and module_paths.
ValueError – If kwargs contains arguments that are invalid for the class construction.
TypeError – If the instance is not an instantiation of classtype.

get_instance¶

texar.tf.utils.get_instance(class_or_name, kwargs, module_paths=None)[source]¶

Creates a class instance.

Parameters:	class_or_name – A class, or its name or full path to a class to instantiate. kwargs (dict) – Keyword arguments for the class constructor. module_paths (list, optional) – Paths to candidate modules to search for the class. This is used if the class cannot be located solely based on `class_name`. The first module in the list that contains the class is used.
Returns:	A class instance.
Raises:	`ValueError` – If class is not found based on `class_or_name` and `module_paths`. `ValueError` – If `kwargs` contains arguments that are invalid for the class construction.

check_or_get_instance_with_redundant_kwargs¶

texar.tf.utils.check_or_get_instance_with_redundant_kwargs(ins_or_class_or_name, kwargs, module_paths=None, classtype=None)[source]¶

Returns a class instance and checks types.

Only those keyword arguments in kwargs that are included in the class construction method are used.

Parameters:

ins_or_class_or_name –
Can be of 3 types:
- A class to instantiate.
- A string of the name or module path to a class to instantiate.
- The class instance to check types.
kwargs (dict) – Keyword arguments for the class constructor.
module_paths (list, optional) – Paths to candidate modules to search for the class. This is used if the class cannot be located solely based on class_name. The first module in the list that contains the class is used.
classtype (optional) – A (list of) classes of which the instance must be an instantiation.

Raises:

ValueError – If class is not found based on class_name and module_paths.
ValueError – If kwargs contains arguments that are invalid for the class construction.
TypeError – If the instance is not an instantiation of classtype.

get_instance_with_redundant_kwargs¶

texar.tf.utils.get_instance_with_redundant_kwargs(class_name, kwargs, module_paths=None)[source]¶

Creates a class instance.

Only those keyword arguments in kwargs that are included in the class construction method are used.

Parameters:	class_name (str) – A class or its name or module path. kwargs (dict) – A dictionary of arguments for the class constructor. It may include invalid arguments which will be ignored. module_paths (list of str) – A list of paths to candidate modules to search for the class. This is used if the class cannot be located solely based on `class_name`. The first module in the list that contains the class is used.
Returns:	A class instance.
Raises:	`ValueError` – If class is not found based on `class_name` and `module_paths`.

get_function¶

texar.tf.utils.get_function(fn_or_name, module_paths=None)[source]¶

Returns the function of specified name and module.

Parameters:	fn_or_name (str or callable) – Name or full path to a function, or the function itself. module_paths (list, optional) – A list of paths to candidate modules to search for the function. This is used only when the function cannot be located solely based on `fn_or_name`. The first module in the list that contains the function is used.
Returns:	A function.

call_function_with_redundant_kwargs¶

texar.tf.utils.call_function_with_redundant_kwargs(fn, kwargs)[source]¶

Calls a function and returns the results.

Only those keyword arguments in kwargs that are included in the function’s argument list are used to call the function.

Parameters:	fn (function) – A callable. If `fn` is not a python function, `fn.__call__` is called. kwargs (dict) – A dict of arguments for the callable. It may include invalid arguments which will be ignored.
Returns:	The returned results by calling `fn`.

get_args¶

texar.tf.utils.get_args(fn)[source]¶

Gets the arguments of a function.

Parameters:	fn (callable) – The function to inspect.
Returns:	A list of argument names (str) of the function.
Return type:	list

get_default_arg_values¶

texar.tf.utils.get_default_arg_values(fn)[source]¶

Gets the arguments and respective default values of a function.

Only arguments with default values are included in the output dictionary.

Parameters:	fn (callable) – The function to inspect.
Returns:	A dictionary that maps argument names (str) to their default values. The dictionary is empty if no arguments have default values.
Return type:	dict

get_instance_kwargs¶

texar.tf.utils.get_instance_kwargs(kwargs, hparams)[source]¶

Makes a dict of keyword arguments with the following structure:

kwargs_ = {‘hparams’: dict(hparams), **kwargs}.

This is typically used for constructing a module which takes a set of arguments as well as a argument named hparams.

Parameters:	kwargs (dict) – A dict of keyword arguments. Can be None. hparams – A dict or an instance of `HParams` Can be None.
Returns:	A dict that contains the keyword arguments in `kwargs`, and an additional keyword argument named hparams.

Mode¶

switch_dropout¶

texar.tf.utils.switch_dropout(dropout_keep_prob, mode=None)[source]¶

Turns off dropout when not in training mode.

Parameters:	dropout_keep_prob – Dropout keep probability in training mode mode (optional) – A Tensor taking values of tf.estimator.ModeKeys. Dropout is activated if `mode` is TRAIN. If None, the mode is inferred from `texar.tf.global_mode()`.
Returns:	A unit Tensor that equals the dropout keep probability in TRAIN mode, and 1.0 in other modes.

maybe_global_mode¶

texar.tf.utils.maybe_global_mode(mode)[source]¶: Returns texar.tf.global_mode() if mode is None, otherwise returns mode as-is.

is_train_mode¶

texar.tf.utils.is_train_mode(mode)[source]¶: Returns a bool Tensor indicating whether the global mode is TRAIN. If mode is None, the mode is determined by texar.tf.global_mode().

is_eval_mode¶

texar.tf.utils.is_eval_mode(mode)[source]¶: Returns a bool Tensor indicating whether the global mode is EVAL. If mode is None, the mode is determined by texar.tf.global_mode().

is_predict_mode¶

texar.tf.utils.is_predict_mode(mode)[source]¶: Returns a bool Tensor indicating whether the global mode is PREDICT. If mode is None, the mode is determined by texar.tf.global_mode().

is_train_mode_py¶

texar.tf.utils.is_train_mode_py(mode, default=True)[source]¶

Returns a python boolean indicating whether the mode is TRAIN.

Parameters:	mode – A string taking value in tf.estimator.ModeKeys. Can be None. default (bool) – The return value when `mode` is None. Default is True.
Returns:	A python boolean.

is_eval_mode_py¶

texar.tf.utils.is_eval_mode_py(mode, default=False)[source]¶

Returns a python boolean indicating whether the mode is EVAL.

Parameters:	mode – A string taking value in tf.estimator.ModeKeys. Can be None. default (bool) – The return value when `mode` is None. Default is False.
Returns:	A python boolean.

is_predict_mode_py¶

texar.tf.utils.is_predict_mode_py(mode, default=False)[source]¶

Returns a python boolean indicating whether the mode is PREDICT.

Parameters:	mode – A string taking value in tf.estimator.ModeKeys. Can be None. default (bool) – The return value when `mode` is None. Default is False.
Returns:	A python boolean.

Misc¶

ceildiv¶

texar.tf.utils.ceildiv(a, b)[source]¶

Divides with ceil.

E.g., 5 / 2 = 2.5, ceildiv(5, 2) = 3.

Parameters:	a (int) – Dividend integer. b (int) – Divisor integer.
Returns:	Ceil quotient.
Return type:	int

straight_through¶

texar.tf.utils.straight_through(fw_tensor, bw_tensor)[source]¶

Use a tensor in forward pass while backpropagating gradient to another.

Parameters:	fw_tensor – A tensor to be used in the forward pass. bw_tensor – A tensor to which gradient is backpropagated. Must have the same shape and type with `fw_tensor`.
Returns:	A tensor of the same shape and value with `fw_tensor` but will direct gradient to bw_tensor.

truncate_seq_pair¶

texar.tf.utils.truncate_seq_pair(tokens_a: Union[List[int], List[str]], tokens_b: Union[List[int], List[str]], max_length: int)[source]¶

Truncates a sequence pair in place to the maximum length.

This is a simple heuristic which will always truncate the longer sequence one token at a time. This makes more sense than truncating an equal percent of tokens from each, since if one sequence is very short then each token that’s truncated likely contains more information than a longer sequence.

Example:

tokens_a = [1, 2, 3, 4, 5]
tokens_b = [6, 7]
truncate_seq_pair(tokens_a, tokens_b, 5)
tokens_a  # [1, 2, 3]
tokens_b  # [6, 7]

Parameters:	tokens_a – A list of tokens or token ids. tokens_b – A list of tokens or token ids. max_length – maximum sequence length.

AverageRecorder¶

class texar.tf.utils.AverageRecorder(size=None)[source]¶

Maintains the moving averages (i.e., the average of the latest N records) of (possibly multiple) fields.

Fields are determined by the first call of add().

Parameters:	size (int, optional) – The window size of moving average. If None, the average of all added records is maintained.

Example

## Use to maintain moving average of training loss
avg_rec = AverageRecorder(size=10) # average over latest 10 records
while training:
    loss_0, loss_1  = ...
    avg_rec.add([loss_0, loss_1])
    # avg_rec.avg() == [0.12343452, 0.567800323]
    # avg_rec.avg(0) == 0.12343452
    # avg_rec.to_str(precision=2, ) == '0.12 0.57'

## Use to maintain average of test metrics on the whole test set
avg_rec = AverageRecorder() # average over ALL records
while test:
    metric_0, metric_1  = ...
    avg_rec.add({'m0': metric_0, 'm1': metric_1}) # dict is allowed
print(avg_rec.to_str(precision=4, delimiter=' , '))
# 'm0: 0.1234 , m1: 0.5678'
#
# avg_rec.avg() == {'m0': 0.12343452, 'm1': 0.567800323}
# avg_rec.avg(0) == 0.12343452

add(record, weight=None)[source]¶

Appends a new record.

record can be a list, dict, or a single scalar. The record type is determined at the first time add() is called. All subsequent calls to add() must have the same type of record.

record in subsequent calls to add() can contain only a subset of fields than the first call to add().

Example

recorder.add({'1': 0.2, '2': 0.2}) # 1st call to `add`
x = recorder.add({'1': 0.4}) # 2nd call to `add`
# x == {'1': 0.3, '2': 0.2}

Parameters:	record – A single scalar, a list of scalars, or a dict of scalars. weight (optional) – A scalar, weight of the new record for calculating a weighted average. If None, weight is set to 1. For example, `weight` can be set to batch size and `record` the average value of certain metrics on the batch in order to calculate the average metric values on a whole dataset.
Returns:	The (moving) average after appending the record, with the same type as `record`.

avg(id_or_name=None)[source]¶

Returns the (moving) average.

Parameters:	id_or_name (optional) – A list of or a single element. Each element is the index (if the record type is list) or name (if the record type is dict) of the field for which the average is calculated. If not given, the average of all fields are returned.
Returns:	The average value(s). If `id_or_name` is a single element (not a list), then returns the average value of the corresponding field. Otherwise, if `id_or_name` is a list of element(s), then returns average value(s) in the same type as `record` of `add()`.

reset(id_or_name=None)[source]¶

Resets the record.

Parameters:	id_or_name (optional) – A list or a single element. Each element is the index (if the record type is list) or name (if the record type is dict) of the field to reset. If None, all fields are reset.

to_str(precision=None, delimiter=' ')[source]¶

Returns a string of the average values of the records.

Parameters:

precision (int, optional) – The number of decimal places to keep in the returned string. E.g., for an average value of 0.1234, precision = 2 leads to ‘0.12’.
delimiter (str) – The delimiter string that separates between fields.

Returns:

A string of the average values.

If record is of type dict, the string is a concatenation of ‘field_name: average_value’, delimited with delimiter. E.g., ‘field_name_1: 0.1234 field_name_2: 0.5678 …’.

Otherwise, the string is of a concatenation of ‘average_value’. E.g., ‘0.1234 0.5678 …’