Don't automatically retrigger, and don't reduce parellism on retrigger

[jfrost-mo]

What problem does your feature request solve?

When actually using the workflow with the new graphing, the automatic retrigger has proven not to be helpful, as it is very rare for the issue to actually be an out of memory. Furthermore reducing the parallelism makes the task very slow to finish, as often the issue will cause all tasks to fail.

Finally, the implementation of the reduced parallelism is complex, requiring mucking about in the rose_bunch database. If we can remove that it would be good.

Describe the solution you'd like

Don't automatically retrigger, and don't reduce the parallelism on a retrigger. The user can set the directives appropriately when retriggering if needed.

Describe alternatives you've considered

[SGallagherMet]

I'm not certain about "as it is very rare for the issue to actually be an out of memory". I've definitely had tasks succeed in the retry. Unfortunately this has been in a branch where I'm still testing some changes to the variable loading/filtering so I can't say it was definitely memory issues.

In any case, in a long trial it would be unhelpful if users have to go through a long trial retriggering failed tasks if they're guaranteed to succeed on a retry.

[jfrost-mo]

Good to see it is working as intended! I've opened this issue as it makes things quite painful when the problem wasn't an out of memory, due to no longer parallelising within a cycle.

I'll probably park #1741 for a while, until I have a bit more time to look into this. I've got some ideas around dynamically varying the parallelism based on measuring the memory usage, but it will take some investigation first.