-
Notifications
You must be signed in to change notification settings - Fork 35
Open
Description
In the current implementation of metrics (#18), metrics are collected and aggregated from all tasks involved at the completion of a session.Run
. This has a few drawbacks. First, all tasks may not be available at the end of session.Run
(e.g., due to failed machines). Second, it ties stats collection to evaluation: we may in the future, persist task outputs to, e.g., s3, and clear task outputs from individual machines.
The solution to this is clear: we can flow Scopes along with task outputs; the executor can then aggregate them as the data flows.
Metadata
Metadata
Assignees
Labels
No labels