* minor polish
* improve and implement Batch.cat_
* bugfix for buffer.sample with field impt_weight
* restore the usage of a.cat_(b)
* fix 2 bugs in batch and add corresponding unittest
* code fix for update
* update is_empty to recognize empty over empty; bugfix for len
* bugfix for update and add testcase
* add testcase of update
* fix docs
* fix docs
* fix docs [ci skip]
* fix docs [ci skip]
Co-authored-by: Trinkle23897 <463003665@qq.com>
* make sure the key type of Batch is string, and add unit tests
* add is_empty() function and unit tests
* enable cat of mixing dict and Batch, just like stack
* bugfix for update with empty buffer; remove duplicate variable _weight_sum in PrioritizedReplayBuffer
* point out that ListReplayBuffer cannot be sampled
* remove useless _amortization_counter variable
* remove dummy net; delete two files
* split code to have backbone and head
* rename class
* change torch.float to torch.float32
* use flatten(1) instead of view(batch, -1)
* remove dummy net in docs
* bugfix for rnn
* fix cuda error
* minor fix of docs
* do not change the example code in dqn tutorial, since it is for demonstration
Co-authored-by: Trinkle23897 <463003665@qq.com>
This PR does the following:
- improvement: dramatic reduce of the call to _is_batch_set
- bugfix: list(Batch()) fail; Batch(a=[torch.ones(3), torch.ones(3)]) fail;
- misc: add type check for each element rather than the first element; add test case; _create_value with torch.Tensor does not have np.object type;
* in-place empty_ for Batch
* change Batch.empty to in-place fill; add copy option for Batch construction
* type signiture & remove shadow names for copy
* add doc for data type (only support numbers and object data type)
* add unit test for Batch copy
* fix pep8
* add test case for Batch.empty
* doc fix
* fix pep8
* use object to test Batch
* test commit
* refact
* change Batch(copy) testcase
* minor fix
Co-authored-by: Trinkle23897 <463003665@qq.com>
* Add auto alpha tuning and exploration noise for sac.
Add class BaseNoise and GaussianNoise for the concept of exploration noise.
Add new test for sac tested in MountainCarContinuous-v0,
which should benefits from the two above new feature.
* add exploration noise to collector, fix example to adapt modification
* fix#98
* enable off-policy to update multiple times in one step. (#99)
* Use lower-level API to reduce overhead.
* Further improvements.
* Buffer _add_to_buffer improvement.
* Do not use _data field to store Batch data to avoid overhead. Add back _meta field in Buffer.
* Restore metadata attribute to store batch in Buffer.
* Move out nested methods.
* Update try/catch instead of actual check to efficiency.
* Remove unsed branches for efficiency.
* Use np.array over list when possible for efficiency.
* Final performance improvement.
* Add unit tests for Batch size method.
* Add missing stack unit tests.
* Enforce Buffer initialization to zero.
Co-authored-by: Alexis Duburcq <alexis.duburcq@wandercraft.eu>
* Fix support of batch over batch for Buffer.
* Do not use internal __dict__ attribute to store batch data since it breaks inheritance.
* Various fixes.
* Improve robustness of Batch/Buffer by avoiding direct attribute assignment. Buffer refactoring.
* Add axis optional argument to Batch stack method.
* Add item assignment to Batch class.
* Fix list support for Buffer.
* Convert list to np.array by default for efficiency.
* Add missing unit test for Batch. Fix unit tests.
* Batch item assignment is now robust to key order.
* Do not use getattr/setattr explicity for simplicity.
* More flexible __setitem__.
* Fixes
* Remove broacasting at Batch level since it is unreliable.
* Forbid item assignement for inconsistent batches.
* Implement broadcasting at Buffer level.
* Add more unit test for Batch item assignment.
Co-authored-by: Alexis Duburcq <alexis.duburcq@wandercraft.eu>
* Fix support of 0-dim numpy array.
* Do not raise exception if Batch index does not make sense since it breaks existing code.
Co-authored-by: Alexis Duburcq <alexis.duburcq@wandercraft.eu>
* Enable to stack Batch instances. Add Batch cat static method. Rename cat in cat_ since inplace.
* Properly handle Batch init using np.array of dict.
* WIP
* Get rid of metadata.
* Update UT. Replace cat by cat_ everywhere.
* Do not sort Batch keys anymore for efficiency. Add items method.
* Fix cat copy issue.
* Add unit test to chack cat and stack methods.
* Remove used import.
* Fix linter issues.
* Fix unit tests.
Co-authored-by: Alexis Duburcq <alexis.duburcq@wandercraft.eu>
* Add auto alpha tuning and exploration noise for sac.
Add class BaseNoise and GaussianNoise for the concept of exploration noise.
Add new test for sac tested in MountainCarContinuous-v0,
which should benefits from the two above new feature.
* add exploration noise to collector, fix example to adapt modification
Add class BaseNoise and GaussianNoise for the concept of exploration noise.
Add new test for sac tested in MountainCarContinuous-v0,
which should benefits from the two above new feature.
* Enable getattr for SubprovVecEnv.
* Consistent API between VectorEnv and SubprocVecEnv.
* Avoid code duplication. Add unit tests.
* Add docstring.
* Test more branches.
* Fix UT.
Co-authored-by: Alexis Duburcq <alexis.duburcq@wandercraft.eu>
