malib.rl.a2c package

Submodules

malib.rl.a2c.config module

malib.rl.a2c.policy module

class malib.rl.a2c.policy.A2CPolicy(observation_space: Space, action_space: Space, model_config: Dict[str, Any], custom_config: Dict[str, Any], **kwargs)[source]

Bases: PGPolicy

Build a REINFORCE policy whose input and output dims are determined by observation_space and action_space, respectively.

Parameters:
  • observation_space (spaces.Space) – The observation space.

  • action_space (spaces.Space) – The action space.

  • model_config (Dict[str, Any]) – The model configuration dict.

  • custom_config (Dict[str, Any]) – The custom configuration dict.

  • is_fixed (bool, optional) – Indicates fixed policy or trainable policy. Defaults to False.

Raises:
  • NotImplementedError – Does not support other action space type settings except Box and Discrete.

  • TypeError – Unexpected action space.

value_function(observation: Tensor, evaluate: bool, **kwargs)[source]

Compute values of critic.

malib.rl.a2c.trainer module

class malib.rl.a2c.trainer.A2CTrainer(training_config: Dict[str, Any], policy_instance: Optional[Policy] = None)[source]

Bases: Trainer

Initialize a trainer for a type of policies.

Parameters:
  • learning_mode (str) – Learning mode inidication, could be off_policy or on_policy.

  • training_config (Dict[str, Any], optional) – The training configuration. Defaults to None.

  • policy_instance (Policy, optional) – A policy instance, if None, we must reset it. Defaults to None.

post_process(batch: Batch, agent_filter: Sequence[str]) Batch[source]

Batch post processing here.

Parameters:

batch (Batch) – Sampled batch.

Raises:

NotImplementedError – Not implemented.

Returns:

A batch instance.

Return type:

Batch

setup()[source]

Set up optimizers here.

train(batch: Batch) Dict[str, float][source]

Run training, and return info dict.

Parameters:

batch (Union[Dict[AgentID, Batch], Batch]) – A dict of batch or batch

Returns:

A training batch of data.

Return type:

Batch