tianshou

<div align="center">
  <a href="http://tianshou.readthedocs.io"><img width="300px" height="auto" src="https://github.com/thu-ml/tianshou/raw/master/docs/_static/images/tianshou-logo.png"></a>
</div>

---

[![PyPI](https://img.shields.io/pypi/v/tianshou)](https://pypi.org/project/tianshou/) [![Read the Docs](https://readthedocs.org/projects/tianshou/badge/?version=master)](https://tianshou.org/en/master/) [![Pytest](https://github.com/thu-ml/tianshou/actions/workflows/pytest.yml/badge.svg)](https://github.com/thu-ml/tianshou/actions) [![codecov](https://img.shields.io/codecov/c/gh/thu-ml/tianshou)](https://codecov.io/gh/thu-ml/tianshou) [![GitHub issues](https://img.shields.io/github/issues/thu-ml/tianshou)](https://github.com/thu-ml/tianshou/issues) [![GitHub stars](https://img.shields.io/github/stars/thu-ml/tianshou)](https://github.com/thu-ml/tianshou/stargazers) [![GitHub forks](https://img.shields.io/github/forks/thu-ml/tianshou)](https://github.com/thu-ml/tianshou/network) [![GitHub license](https://img.shields.io/github/license/thu-ml/tianshou)](https://github.com/thu-ml/tianshou/blob/master/LICENSE)

> [!NOTE]
> **Tianshou version 2 is here!**  
> 
> We have released the new major version of Tianshou on PyPI.  
> Version 2 is a complete overhaul of the software design of the procedural API, in which
>   * we establish a clear separation between learning algorithms and policies (via the separate abstractions `Algorithm` and `Policy`).
>   * we provide more well-defined, more usable interfaces with extensive documentation of all algorithm and trainer parameters,
>     renaming some parameters to make their names more consistent and intuitive.
>   * the class hierarchy is fully revised, establishing a clear separation between on-policy, off-policy and offline algorithms
>     at the type level and ensuring that all inheritance relationships are meaningful.
> 
> Because of the extent of the changes, this version is not backwards compatible with previous versions of Tianshou.
> For migration information, please see the [change log](CHANGELOG.md). 

**Tianshou** ([天授](https://baike.baidu.com/item/%E5%A4%A9%E6%8E%88)) is a reinforcement learning (RL) library based on pure PyTorch and [Gymnasium](http://github.com/Farama-Foundation/Gymnasium). Tianshou's main features at a glance are:

1. Modular low-level interfaces for algorithm developers (RL researchers) that are both flexible, hackable and type-safe.
1. Convenient high-level interfaces for applications of RL (training an implemented algorithm on a custom environment).
1. Large scope: online (on- and off-policy) and offline RL, experimental support for multi-agent RL (MARL), experimental support for model-based RL, and more

Unlike other reinforcement learning libraries, which may have complex codebases,
unfriendly high-level APIs, or are not optimized for speed, Tianshou provides a high-performance, modularized framework
and user-friendly interfaces for building deep reinforcement learning ag