* add doc for len exceptions
* doc move; unify is_scalar_value function
* remove some issubclass check
* bugfix for shape of Batch(a=1)
* keep moving doc
* keep writing batch tutorial
* draft version of Batch tutorial done
* improving doc
* keep improving doc
* batch tutorial done
* rename _is_number
* rename _is_scalar
* shape property do not raise exception
* restore some doc string
* grammarly [ci skip]
* grammarly + fix warning of building docs
* polish docs
* trim and re-arrange batch tutorial
* go straight to the point
* minor fix for batch doc
* add shape / len in basic usage
* keep improving tutorial
* unify _to_array_with_correct_type to remove duplicate code
* delegate type convertion to Batch.__init__
* further delegate type convertion to Batch.__init__
* bugfix for setattr
* add a _parse_value function
* remove dummy function call
* polish docs
Co-authored-by: Trinkle23897 <463003665@qq.com>
* code refactor; remove unused kwargs; add reward_normalization for dqn
* bugfix for __setitem__ with torch.Tensor; add Batch.condense
* minor fix
* support cat with empty Batch
* remove the dependency of is_empty on len; specify the semantic of empty Batch by test cases
* support stack with empty Batch
* remove condense
* refactor code to reflect the shared / partial / reserved categories of keys
* add is_empty(recursive=False)
* doc fix
* docfix and bugfix for _is_batch_set
* add doc for key reservation
* bugfix for algebra operators
* fix cat with lens hint
* code refactor
* bugfix for storing None
* use ValueError instead of exception
* hide lens away from users
* add comment for __cat
* move the computation of the initial value of lens in cat_ itself.
* change the place of doc string
* doc fix for Batch doc string
* change recursive to recurse
* doc string fix
* minor fix for batch doc
* remove multibuf
* reward_metric
* make fileds with empty Batch rather than None after reset
* many fixes and refactor
Co-authored-by: Trinkle23897 <463003665@qq.com>
* minor polish
* improve and implement Batch.cat_
* bugfix for buffer.sample with field impt_weight
* restore the usage of a.cat_(b)
* fix 2 bugs in batch and add corresponding unittest
* code fix for update
* update is_empty to recognize empty over empty; bugfix for len
* bugfix for update and add testcase
* add testcase of update
* fix docs
* fix docs
* fix docs [ci skip]
* fix docs [ci skip]
Co-authored-by: Trinkle23897 <463003665@qq.com>
* make sure the key type of Batch is string, and add unit tests
* add is_empty() function and unit tests
* enable cat of mixing dict and Batch, just like stack
* bugfix for update with empty buffer; remove duplicate variable _weight_sum in PrioritizedReplayBuffer
* point out that ListReplayBuffer cannot be sampled
* remove useless _amortization_counter variable
* remove dummy net; delete two files
* split code to have backbone and head
* rename class
* change torch.float to torch.float32
* use flatten(1) instead of view(batch, -1)
* remove dummy net in docs
* bugfix for rnn
* fix cuda error
* minor fix of docs
* do not change the example code in dqn tutorial, since it is for demonstration
Co-authored-by: Trinkle23897 <463003665@qq.com>
This PR does the following:
- improvement: dramatic reduce of the call to _is_batch_set
- bugfix: list(Batch()) fail; Batch(a=[torch.ones(3), torch.ones(3)]) fail;
- misc: add type check for each element rather than the first element; add test case; _create_value with torch.Tensor does not have np.object type;
* in-place empty_ for Batch
* change Batch.empty to in-place fill; add copy option for Batch construction
* type signiture & remove shadow names for copy
* add doc for data type (only support numbers and object data type)
* add unit test for Batch copy
* fix pep8
* add test case for Batch.empty
* doc fix
* fix pep8
* use object to test Batch
* test commit
* refact
* change Batch(copy) testcase
* minor fix
Co-authored-by: Trinkle23897 <463003665@qq.com>
* Add auto alpha tuning and exploration noise for sac.
Add class BaseNoise and GaussianNoise for the concept of exploration noise.
Add new test for sac tested in MountainCarContinuous-v0,
which should benefits from the two above new feature.
* add exploration noise to collector, fix example to adapt modification
* fix#98
* enable off-policy to update multiple times in one step. (#99)
* Use lower-level API to reduce overhead.
* Further improvements.
* Buffer _add_to_buffer improvement.
* Do not use _data field to store Batch data to avoid overhead. Add back _meta field in Buffer.
* Restore metadata attribute to store batch in Buffer.
* Move out nested methods.
* Update try/catch instead of actual check to efficiency.
* Remove unsed branches for efficiency.
* Use np.array over list when possible for efficiency.
* Final performance improvement.
* Add unit tests for Batch size method.
* Add missing stack unit tests.
* Enforce Buffer initialization to zero.
Co-authored-by: Alexis Duburcq <alexis.duburcq@wandercraft.eu>
* Fix support of batch over batch for Buffer.
* Do not use internal __dict__ attribute to store batch data since it breaks inheritance.
* Various fixes.
* Improve robustness of Batch/Buffer by avoiding direct attribute assignment. Buffer refactoring.
* Add axis optional argument to Batch stack method.
* Add item assignment to Batch class.
* Fix list support for Buffer.
* Convert list to np.array by default for efficiency.
* Add missing unit test for Batch. Fix unit tests.
* Batch item assignment is now robust to key order.
* Do not use getattr/setattr explicity for simplicity.
* More flexible __setitem__.
* Fixes
* Remove broacasting at Batch level since it is unreliable.
* Forbid item assignement for inconsistent batches.
* Implement broadcasting at Buffer level.
* Add more unit test for Batch item assignment.
Co-authored-by: Alexis Duburcq <alexis.duburcq@wandercraft.eu>
* Fix support of 0-dim numpy array.
* Do not raise exception if Batch index does not make sense since it breaks existing code.
Co-authored-by: Alexis Duburcq <alexis.duburcq@wandercraft.eu>
* Enable to stack Batch instances. Add Batch cat static method. Rename cat in cat_ since inplace.
* Properly handle Batch init using np.array of dict.
* WIP
* Get rid of metadata.
* Update UT. Replace cat by cat_ everywhere.
* Do not sort Batch keys anymore for efficiency. Add items method.
* Fix cat copy issue.
* Add unit test to chack cat and stack methods.
* Remove used import.
* Fix linter issues.
* Fix unit tests.
Co-authored-by: Alexis Duburcq <alexis.duburcq@wandercraft.eu>
* Add auto alpha tuning and exploration noise for sac.
Add class BaseNoise and GaussianNoise for the concept of exploration noise.
Add new test for sac tested in MountainCarContinuous-v0,
which should benefits from the two above new feature.
* add exploration noise to collector, fix example to adapt modification
Add class BaseNoise and GaussianNoise for the concept of exploration noise.
Add new test for sac tested in MountainCarContinuous-v0,
which should benefits from the two above new feature.
