Skip to content

Action bias is added twice in TD3 algorithm implementation #259

@implausibleDeniability

Description

@implausibleDeniability

Problem Description

In the implementation of TD3 algorithm, the action bias is added twice to the action. First, in the line 180 it is added during actor forward pass. Second, in the line 181 the random noise comes from a distribution with the center in actor.action_bias.

Possible Solution

I assume rewriting the 181th line to

actions += torch.normal(0, actor.action_scale * args.exploration_noise)

will solve the problem.

Metadata

Metadata

Assignees

Labels

No labels
No labels

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions