hidet.option
hidet.option¶
Classes:
The option context. |
Functions:
Dump the options in option context stack. |
|
|
Restore the options from dumped options. |
Get the current option context. |
|
|
Create a new option context. |
|
Set the value of an option in current option context. |
|
Get the value of an option in current option context. |
|
Set the benchmark config of operator tuning. |
Get the benchmark config of operator tuning. |
|
|
Set the schedule search space of tunable operator. |
Get the schedule search space of tunable operator. |
|
|
Whether to cache compiled operator on disk. |
Get the option value of whether to cache compiled operator on disk. |
|
|
Set the directory to store the cache. |
Get the directory to store the cache. |
|
|
Whether to build operators in parallel. |
Get the option value of whether to build operators in parallel. |
|
|
Whether to save the lower IR. |
Get the option value of whether to save the lower IR. |
|
|
Whether to cache the generated kernels during tuning. |
- hidet.option.dump_options()[source]¶
Dump the options in option context stack.
- Returns
ret – The dumped options.
- Return type
Dict[str, Any]
- hidet.option.restore_options(dumped_options)[source]¶
Restore the options from dumped options.
- Parameters
dumped_options (Dict[str, Any]) – The dumped options.
- hidet.option.current_context()[source]¶
Get the current option context.
To get the value of an option in the current context:
ctx = hidet.option.current_context() cache_dir: str = ctx.get_option('cache_dir') cache_operator: bool = ctx.get_option('cache_operator') ...
- Returns
ctx – The current option context.
- Return type
- hidet.option.context()[source]¶
Create a new option context.
To set options in the new context, use the
with
statement:with hidet.option.context() as ctx: hidet.option.cache_dir('./new_cache_dir') # set predefined option hidet.option.set_option('other_option', 'other_value') # set a custom option ...
- Returns
ctx – The new option context.
- Return type
- hidet.option.set_option(name, value)[source]¶
Set the value of an option in current option context.
The option must be registered before setting via
hidet.option.register_option()
.- Parameters
name (str) – The name of the option.
value (Any) – The value of the option.
- hidet.option.get_option(name)[source]¶
Get the value of an option in current option context.
- Parameters
name (str) – The name of the option.
- Returns
ret – The value of the option.
- Return type
Any
- hidet.option.bench_config(warmup=1, number=5, repeat=5)[source]¶
Set the benchmark config of operator tuning.
To profile a schedule, hidet will run the following code:
for i in range(warmup): run() latency = [] for i in range(repeat): synchronize device t1 = time() for j in range(number): run() synchronize device t2 = time() latency.append((t2 - t1) / number) return median of latency
Thus, there will be total
warmup + number * repeat
times of execution.- Parameters
warmup (int) – The number of warmup runs.
number (int) – The number of runs in a repeat.
repeat (int) – The number of repeats.
- hidet.option.get_bench_config()[source]¶
Get the benchmark config of operator tuning.
- Returns
ret – The benchmark config.
- Return type
Tuple[int, int, int]
- hidet.option.search_space(space)[source]¶
Set the schedule search space of tunable operator.
Some operators can be tuned in hidet to achieve the best performance, such as matrix multiplication.
During tuning, different operator schedules will be tried and profiled to get the best one.
We call the space of the tried operator schedule schedule space. There is a trade-off between the tuning time and the operator execution time. If we try more schedules, the tuning process would take longer time, and we are likely to find better schedule.
This function allows user to set the space level that controls the search space we tried.
By convention, we have space level
0 for schedule space contains only a single schedule.
1 for schedule space contains tens of schedules so that the tuning time will be less than 1 minute.
2 for arbitrary large space.
Usage
hidet.search_space(2)
After calling above function, all subsequent compilation would use space level 2, until we call this function again with another space level.
- Parameters
space (int) – The space level to use. Candidates: 0, 1, and 2.
- hidet.option.get_search_space()[source]¶
Get the schedule search space of tunable operator.
- Returns
ret – The schedule space level.
- Return type
int
- hidet.option.cache_operator(enabled=True)[source]¶
Whether to cache compiled operator on disk.
By default, hidet would cache all compiled operator and reuse whenever possible.
If user wants to disable the cache, run
hidet.option.cache_operator(False)
- Parameters
enabled (bool) – Whether to cache the compiled operator.
- hidet.option.get_cache_operator()[source]¶
Get the option value of whether to cache compiled operator on disk.
- Returns
ret – Whether to cache the compiled operator.
- Return type
bool
- hidet.option.cache_dir(new_dir)[source]¶
Set the directory to store the cache.
The default cache directory:
If the hidet code is in a git repo, the cache will be stored in the repo root:
hidet-repo/.hidet_cache
.Otherwise, the cache will be stored in the user home directory:
~/.hidet/cache
.
- Parameters
new_dir (str) – The new directory to store the cache.
- hidet.option.get_cache_dir()[source]¶
Get the directory to store the cache.
- Returns
ret – The directory to store the cache.
- Return type
str
- hidet.option.parallel_build(enabled=True)[source]¶
Whether to build operators in parallel.
- Parameters
enabled (bool) – Whether to build operators in parallel.
- hidet.option.get_parallel_build()[source]¶
Get the option value of whether to build operators in parallel.
- Returns
ret – Whether to build operators in parallel.
- Return type
bool
- hidet.option.save_lower_ir(enabled=True)[source]¶
Whether to save the lower IR.
- Parameters
enabled (bool) – Whether to save the lower IR.
- hidet.option.get_save_lower_ir()[source]¶
Get the option value of whether to save the lower IR.
- Returns
ret – Whether to save the lower IR.
- Return type
bool
- hidet.option.debug_cache_tuning(enabled=True)[source]¶
Whether to cache the generated kernels during tuning.
Note
This option is only used for debugging purpose. It will generate a lot of files in the cache directory and take a lot of disk space.
- Parameters
enabled (bool) – Whether to debug cache tuning.