hidet

Hidet is an open-source DNN inference framework based on compilation.

Functions:

empty(shape[, dtype, device, layout])	Create an uninitialized tensor.
randn(shape[, dtype, mean, stddev, device])	Create a tensor with uniformly distributed values.
zeros(shape[, dtype, device])	Create a tensor initialized with zero.
ones(shape[, dtype, device])	Create a tensor initialized with one.
full(shape, fill_value[, dtype, device])	Create a tensor initialized with given constant.
symbol(shape[, dtype, device, layout])	Create a symbolic tensor.
asarray(obj, /, *[, dtype, device])	Convert a list, tuple, or numpy ndarray to a hidet tensor.
from_torch(torch_tensor)	Create a hidet tensor from pytorch tensor.
empty_like(data[, shape, dtype, device, layout])	Create an uninitialized tensor with the same shape, dtype, and device as the given tensor.
randn_like(data[, mean, stddev, shape, ...])	Create a randomly initialized tensor with the same shape, dtype, and device as the given tensor.
zeros_like(data[, shape, dtype, device])	Create a tensor initialized with zero with the same shape, dtype, and device as the given tensor.
ones_like(data[, shape, dtype, device])	Create a tensor initialized with one with the same shape, dtype, and device as the given tensor.
symbol_like(data[, shape, dtype, device, layout])	Create a symbol tensor like an existing tensor.
full_like(data, fill_value[, shape, dtype, ...])	Create a tensor initialized with fill_value with the same shape, dtype, and device as the given tensor.
trace_from(tensor[, inputs])	Trace the flow graph given the output tensor(s).
from_dlpack(dltensor)	Create a hidet tensor from an object that implements the __dlpack__ protocol.
script(func)	Decorator to convert a Python function to a Hidet function.

hidet.empty(shape, dtype='float32', device='cpu', layout=None)

Create an uninitialized tensor.

Parameters:

• shape (Sequence[int]) – The shape of new tensor.

• dtype (str or DataType) – The data type of element of the tensor.

• device (Device or str, default 'cpu') – The device of the new tensor is created on.

• layout (DataLayout, optional) – The layout of the new tensor. None indicates the default layout (row-major layout).

Returns:

ret – The created tensor.

Return type:

Tensor

hidet.randn(shape, dtype='float32', mean=0.0, stddev=1.0, device='cpu')

Create a tensor with uniformly distributed values.

Parameters:

• shape (Sequence[int]) – The shape of new tensor.

• dtype (DataType or str, default 'float32') – The data type of element of the tensor.

• mean (float, default 0.0) – The mean of the uniform distribution.

• stddev (float, default 1.0) – The standard deviation of the uniform distribution.

• device (Device or str, default 'cpu') – The device of the new tensor is created on.

Returns:

ret – The created tensor.

Return type:

Tensor

Examples

>>> randn([2, 3])
Tensor(shape=[2, 3], dtype='float32', device='cuda')
[[ 0.10720467 -1.6906018   0.06347568]
 [-0.37061226  0.562728    1.857547  ]]

hidet.zeros(shape, dtype='float32', device='cpu')

Create a tensor initialized with zero.

Parameters:

• shape (Sequence[int]) – The shape of new tensor.

• dtype (str or DataType) – The data type of element of the tensor.

• device (Device or str, default 'cpu') – The device of the new tensor is created on.

Returns:

ret – The created tensor.

Return type:

Tensor

hidet.ones(shape, dtype='float32', device='cpu')

Create a tensor initialized with one.

Parameters:

• shape (Sequence[int]) – The shape of new tensor.

• dtype (DataType or str, default 'float32') – The data type of element of the tensor.

• device (Device or str, default 'cpu') – The device of the new tensor is created on.

Returns:

ret – The created tensor.

Return type:

Tensor

hidet.full(shape, fill_value, dtype='float32', device='cpu')

Create a tensor initialized with given constant.

Parameters:

• shape (Sequence[int]) – The shape of new tensor.

• fill_value (float or int or hidet.ir.Constant) – The constant to initialize the new tensor.

• dtype (DataType or str, default 'float32') – The data type of element of the tensor.

• device (Device or str, default 'cpu') – The device of the new tensor is created on.

Returns:

ret – The created tensor.

Return type:

Tensor

hidet.symbol(shape, dtype='float32', device='cpu', layout=None)

Create a symbolic tensor.

Parameters:

• shape (Sequence[Union[int, str, Expr]]) – The shape of new tensor. The shape can contain symbolic variables. str indicates the corresponding dimension is a symbolic variable with the given name.

• dtype (str or DataType) – The data type of element of the tensor.

• device (Device or str, default 'cpu') – The device of the new tensor is created on.

• layout (DataLayout, optional) – The layout of the new tensor. None indicates the default layout (row-major layout).

Returns:

ret – The created tensor.

Return type:

Tensor

hidet.asarray(obj, /, *, dtype=None, device=None)

Convert a list, tuple, or numpy ndarray to a hidet tensor.

Parameters:

• obj (bool, int, float, List, Tuple, Tensor, np.ndarray) – The object to be converted.

• dtype (DataType or str, optional) – The data type of the output tensor.

• device (Device or str) – The device of the output tensor.

Returns:

ret – The hidet tensor converted from given object.

Return type:

Tensor

hidet.from_torch(torch_tensor)

Create a hidet tensor from pytorch tensor.

The created tensor shared the same memory as given pytorch tensor. Thus, any content modification on one tensor would be reflected on the other one.

Parameters:

torch_tensor (torch.Tensor) – The pytorch tensor.

Returns:

ret – The created hidet tensor.

Return type:

Tensor

hidet.empty_like(data, shape=None, dtype=None, device=None, layout=None)

Create an uninitialized tensor with the same shape, dtype, and device as the given tensor.

Parameters:

• data (Tensor) – The tensor to copy shape, dtype, and device from.

• shape (Sequence[int], optional) – The shape of new tensor. If None, the shape of data is used.

• dtype (DataType or str, optional) – The data type of element of the tensor. If None, the dtype of data is used.

• device (Device or str, optional) – The device of the new tensor is created on. If None, the device of data is used.

• layout (DataLayout, optional) – The layout of the new tensor. If None, the layout of data is used.

Returns:

ret – The created tensor.

Return type:

Tensor

hidet.randn_like(data, mean=0.0, stddev=1.0, shape=None, dtype=None, device=None)

Create a randomly initialized tensor with the same shape, dtype, and device as the given tensor.

Parameters:

• data (Tensor) – The tensor to copy shape, dtype, and device from.

• mean (float, optional) – The mean of the normal distribution.

• stddev (float, optional) – The standard deviation of the normal distribution.

• shape (Sequence[int], optional) – The shape of new tensor. If None, the shape of data is used.

• dtype (DataType or str, optional) – The data type of element of the tensor. If None, the dtype of data is used.

• device (Device or str, optional) – The device of the new tensor is created on. If None, the device of data is used.

Returns:

ret – The created tensor with random values sampled from a normal distribution.

Return type:

Tensor

hidet.zeros_like(data, shape=None, dtype=None, device=None)

Create a tensor initialized with zero with the same shape, dtype, and device as the given tensor.

Parameters:

• data (Tensor) – The tensor to copy shape, dtype, and device from.

• shape (Sequence[int], optional) – The shape of new tensor. If None, the shape of data is used.

• dtype (DataType or str, optional) – The data type of element of the tensor. If None, the dtype of data is used.

• device (Device or str, optional) – The device of the new tensor is created on. If None, the device of data is used.

Returns:

ret – The created tensor with all elements as zero.

Return type:

Tensor

hidet.ones_like(data, shape=None, dtype=None, device=None)

Create a tensor initialized with one with the same shape, dtype, and device as the given tensor.

Parameters:

• data (Tensor) – The tensor to copy shape, dtype, and device from.

• shape (Sequence[int], optional) – The shape of new tensor. If None, the shape of data is used.

• dtype (DataType or str, optional) – The data type of element of the tensor. If None, the dtype of data is used.

• device (Device or str, optional) – The device of the new tensor is created on. If None, the device of data is used.

Returns:

ret – The created tensor with all elements as one.

Return type:

Tensor

hidet.symbol_like(data, shape=None, dtype=None, device=None, layout=None)

Create a symbol tensor like an existing tensor.

Parameters:

• data (Tensor) – The tensor to copy shape, dtype, and device from.

• shape (Sequence[int], optional) – The shape of new tensor. If None, the shape of data is used.

• dtype (DataType or str, optional) – The data type of element of the tensor. If None, the dtype of data is used.

• device (Device or str, optional) – The device of the new tensor is created on. If None, the device of data is used.

• layout (DataLayout, optional) – The layout of the new tensor. If None, the layout of data is used.

Returns:

ret – The created symbol tensor.

Return type:

Tensor

hidet.full_like(data, fill_value, shape=None, dtype=None, device=None)

Create a tensor initialized with fill_value with the same shape, dtype, and device as the given tensor.

Parameters:

• data (Tensor) – The tensor to copy shape, dtype, and device from.

• fill_value (int, float, bool, complex) – The value to fill the tensor with.

• shape (Sequence[int], optional) – The shape of new tensor. If None, the shape of data is used.

• dtype (DataType or str, optional) – The data type of element of the tensor. If None, the dtype of data is used.

• device (Device or str, optional) – The device of the new tensor is created on. If None, the device of data is used.

Returns:

ret – The created tensor with all elements as fill_value.

Return type:

Tensor

hidet.trace_from(tensor, inputs=None)

Trace the flow graph given the output tensor(s).

Each hidet.graph.Tensor has an attribute hidet.graph.Tensor.trace which indicates how the tensor is generated. If the tensor is generated by an operator with symbolic input(s), the tensor itself is also symbolic. And the tensor will have a reference to the operator that generates it. The reference is stored in this attribute.

What this function does is to walk through the trace of the given tensor(s) and construct a flow graph.

When there are multiple symbol inputs, it is mandatory to specify the “inputs” argument explicitly to avoid ambiguity.

Parameters:

• tensor (Tensor or List[Tensor]) – The output tensor(s) that we trace from.

• inputs (Optional, Tensor or List[Tensor]) – The inputs of the flow graph. When there is only a single symbol tensor in the flow graph, it is optional. When there are multiple inputs, this is required to specify the input order.

Returns:

ret – The flow graph that outputs the given input tensor(s).

Return type:

FlowGraph

hidet.from_dlpack(dltensor)

Create a hidet tensor from an object that implements the __dlpack__ protocol.

Parameters:

dltensor (an object that implements the DLPack protocol.) – The object must have the method __dlpack__ that returns a PyCapsule object with name dltensor.

Returns:

ret – The hidet tensor that shares the same storage with the DLPack tensor.

Return type:

Tensor

hidet.script(func)

Decorator to convert a Python function to a Hidet function.

Parameters:

func (FunctionType) – The python function to be converted to a Hidet function.

Returns:

ret – The hidet.ir.Function that is converted from the given Python function.

Return type:

Function