Tianshou/CHANGELOG.md

# Changelog

## Release 1.1.0

### Api Extensions
- `data`:
  - `Batch`:
    - Add methods `to_dict` and `to_list_of_dicts`. #1063 #1098
    - Add methods `to_numpy_` and `to_torch_`. #1098, #1117
    - Add `__eq__` (semantic equality check). #1098
    - `keys()` deprecated in favor of `get_keys()` (needed to make iteration consistent with naming) #1105.
  - `data.collector`:
    - `Collector`:
      - Introduced `BaseCollector` as a base class for all collectors. #1123
      - Add method `close` #1063
      - Method `reset` is now more granular (new flags controlling behavior). #1063
    - `CollectStats`: Add convenience constructor `with_autogenerated_stats`. #1063
- `trainer`:
  - Trainers can now control whether collectors should be reset prior to training. #1063
- policy:
  - introduced attribute `in_training_step` that is controlled by the trainer. #1123
  - policy automatically set to `eval` mode when collecting and to `train` mode when updating. #1123
- `highlevel`:
  - `SamplingConfig`:
    - Add support for `batch_size=None`. #1077 
    - Add `training_seed` for explicit seeding of training and test environments, the `test_seed` is inferred from `training_seed`. #1074
  - `highlevel.experiment`: 
     - `Experiment` now has a `name` attribute, which can be set using `ExperimentBuilder.with_name` and 
       which determines the default run name and therefore the persistence subdirectory.
       It can still be overridden in `Experiment.run()`, the new parameter name being `run_name` rather than
       `experiment_name` (although the latter will still be interpreted correctly). #1074 #1131
     - Add class `ExperimentCollection` for the convenient execution of multiple experiment runs #1131
     - `ExperimentBuilder`: 
         - Add method `build_seeded_collection` for the sound creation of multiple
           experiments with varying random seeds #1131
         - Add method `copy` to facilitate the creation of multiple experiments from a single builder #1131
- `evaluation`: New package for repeating the same experiment with multiple seeds and aggregating the results. #1074
  - The module `evaluation.launchers` for parallelization is currently in alpha state. 
- Loggers can now restore the logged data into python by using the new `restore_logged_data` method. #1074
- `utils`:
  - `net.continuous.Critic`:
    - Add flag `apply_preprocess_net_to_obs_only` to allow the
      preprocessing network to be applied to the observations only (without
      the actions concatenated), which is essential for the case where we want
      to reuse the actor's preprocessing network #1128
  - `torch_utils` (new module)
    - Added contextmanagers `in`

### Fixes
- `CriticFactoryReuseActor`: Enable the Critic flag `apply_preprocess_net_to_obs_only` for continuous critics, 
  fixing the case where we want to reuse an actor's preprocessing network for the critic (affects usages
  of the experiment builder method `with_critic_factory_use_actor` with continuous environments) #1128
- `atari_network.DQN`:
  - Fix constructor input validation #1128
  - Fix `output_dim` not being set if `features_only`=True and `output_dim_added_layer` is not None #1128

### Internal Improvements
- `Collector`s rely less on state, the few stateful things are stored explicitly instead of through a `.data` attribute. #1063
- Introduced a first iteration of a naming convention for vars in `Collector`s. #1063
- Generally improved readability of Collector code and associated tests (still quite some way to go). #1063
- Improved typing for `exploration_noise` and within Collector. #1063
- Better variable names related to model outputs (logits, dist input etc.). #1032
- Improved typing for actors and critics, using Tianshou classes like `Actor`, `ActorProb`, etc., 
instead of just `nn.Module`. #1032
- Added interfaces for most `Actor` and `Critic` classes to enforce the presence of `forward` methods. #1032
- Simplified `PGPolicy` forward by unifying the `dist_fn` interface (see associated breaking change). #1032
- Use `.mode` of distribution instead of relying on knowledge of the distribution type. #1032
- Exception no longer raised on `len` of empty `Batch`. #1084
- tests and examples are covered by `mypy`. #1077
- `NetBase` is more used, stricter typing by making it generic. #1077
- Use explicit multiprocessing context for creating `Pipe` in `subproc.py`. #1102

### Breaking Changes
- `data`:
  - `Collector`:
    - Removed `.data` attribute. #1063
    - Collectors no longer reset the environment on initialization. 
      Instead, the user might have to call `reset` expicitly or pass `reset_before_collect=True` . #1063
    - Removed `no_grad` argument from `collect` method (was unused in tianshou). #1123
  - `Batch`:
    - Fixed `iter(Batch(...)` which now behaves the same way as `Batch(...).__iter__()`. 
      Can be considered a bugfix. #1063
    - The methods `to_numpy` and `to_torch` in are not in-place anymore 
      (use `to_numpy_` or `to_torch_` instead). #1098, #1117
- Logging:
  - `BaseLogger.prepare_dict_for_logging` is now abstract. #1074
  - Removed deprecated and unused `BasicLogger` (only affects users who subclassed it). #1074
- VectorEnvs now return an array of info-dicts on reset instead of a list. #1063
- Changed interface of `dist_fn` in `PGPolicy` and all subclasses to take a single argument in both
continuous and discrete cases. #1032
- `utils.net.common.Recurrent` now receives and returns a `RecurrentStateBatch` instead of a dict. #1077
- `AtariEnvFactory` constructor (in examples, so not really breaking) now requires explicit train and test seeds. #1074
- `EnvFactoryRegistered` now requires an explicit `test_seed` in the constructor. #1074


### Tests
- Fixed env seeding it `test_sac_with_il.py` so that the test doesn't fail randomly. #1081

### Dependencies
- [DeepDiff](https://github.com/seperman/deepdiff) added to help with diffs of batches in tests. #1098
- Bumped black, idna, pillow
- New extra "eval"

Started after v1.0.0