10d919052b
The new proposed feature is to have trainers as generators.
The usage pattern is:
```python
trainer = OnPolicyTrainer(...)
for epoch, epoch_stat, info in trainer:
print(f"Epoch: {epoch}")
print(epoch_stat)
print(info)
do_something_with_policy()
query_something_about_policy()
make_a_plot_with(epoch_stat)
display(info)
```
- epoch int: the epoch number
- epoch_stat dict: a large collection of metrics of the current epoch, including stat
- info dict: the usual dict out of the non-generator version of the trainer
You can even iterate on several different trainers at the same time:
```python
trainer1 = OnPolicyTrainer(...)
trainer2 = OnPolicyTrainer(...)
for result1, result2, ... in zip(trainer1, trainer2, ...):
compare_results(result1, result2, ...)
```
Co-authored-by: Jiayi Weng <trinkle23897@gmail.com>
149 lines
1.1 KiB
Plaintext
149 lines
1.1 KiB
Plaintext
tianshou
|
|
arXiv
|
|
tanh
|
|
lr
|
|
logits
|
|
env
|
|
envs
|
|
optim
|
|
eps
|
|
timelimit
|
|
TimeLimit
|
|
envpool
|
|
EnvPool
|
|
maxsize
|
|
timestep
|
|
timesteps
|
|
numpy
|
|
ndarray
|
|
stackoverflow
|
|
tensorboard
|
|
len
|
|
tac
|
|
fqf
|
|
iqn
|
|
qrdqn
|
|
rl
|
|
offpolicy
|
|
onpolicy
|
|
quantile
|
|
quantiles
|
|
dqn
|
|
param
|
|
async
|
|
subprocess
|
|
deque
|
|
nn
|
|
equ
|
|
cql
|
|
fn
|
|
boolean
|
|
pre
|
|
np
|
|
cuda
|
|
rnn
|
|
rew
|
|
pre
|
|
perceptron
|
|
bsz
|
|
dataset
|
|
mujoco
|
|
jit
|
|
nstep
|
|
preprocess
|
|
repo
|
|
ReLU
|
|
namespace
|
|
recv
|
|
th
|
|
utils
|
|
NaN
|
|
linesearch
|
|
hyperparameters
|
|
pseudocode
|
|
entropies
|
|
nn
|
|
config
|
|
cpu
|
|
rms
|
|
debias
|
|
indice
|
|
regularizer
|
|
miniblock
|
|
modularize
|
|
serializable
|
|
softmax
|
|
vectorized
|
|
optimizers
|
|
undiscounted
|
|
submodule
|
|
subclasses
|
|
submodules
|
|
tfevent
|
|
dirichlet
|
|
docstring
|
|
webpage
|
|
formatter
|
|
num
|
|
py
|
|
pythonic
|
|
中文文档位于
|
|
conda
|
|
miniconda
|
|
Amir
|
|
Andreas
|
|
Antonoglou
|
|
Beattie
|
|
Bellemare
|
|
Charles
|
|
Daan
|
|
Demis
|
|
Dharshan
|
|
Fidjeland
|
|
Georg
|
|
Hassabis
|
|
Helen
|
|
Ioannis
|
|
Kavukcuoglu
|
|
King
|
|
Koray
|
|
Kumaran
|
|
Legg
|
|
Mnih
|
|
Ostrovski
|
|
Petersen
|
|
Riedmiller
|
|
Rusu
|
|
Sadik
|
|
Shane
|
|
Stig
|
|
Veness
|
|
Volodymyr
|
|
Wierstra
|
|
Lillicrap
|
|
Pritzel
|
|
Heess
|
|
Erez
|
|
Yuval
|
|
Tassa
|
|
Schulman
|
|
Filip
|
|
Wolski
|
|
Prafulla
|
|
Dhariwal
|
|
Radford
|
|
Oleg
|
|
Klimov
|
|
Kaichao
|
|
Jiayi
|
|
Weng
|
|
Duburcq
|
|
Huayu
|
|
Strens
|
|
Ornstein
|
|
Uhlenbeck
|
|
mse
|
|
gail
|
|
airl
|
|
ppo
|