Skip to content

Conversation

gregtatum
Copy link
Contributor

In translations we have a really complicated taskgraph, and it's nice to output better debug information when things go wrong. With the debug information it's easy to see I should have used "compile" rather than "compile-marian-fork".

Error before:

Traceback (most recent call last):
  File "/builds/worker/checkouts/utils/preflight_check.py", line 554, in <module>
    main()
  File "/builds/worker/checkouts/utils/preflight_check.py", line 522, in main
    run_taskgraph(parsed_args.config, get_taskgraph_parameters())
  File "/builds/worker/checkouts/utils/preflight_check.py", line 137, in run_taskgraph
    taskgraph.actions.trigger_action_callback(
  File "/usr/local/lib/python3.10/dist-packages/taskgraph/actions/registry.py", line 344, in trigger_action_callback
    cb(Parameters(**parameters), graph_config, input, task_group_id, task_id)
  File "/builds/worker/checkouts/taskcluster/translations_taskgraph/actions/train.py", line 601, in train_action
    taskgraph_decision(
  File "/usr/local/lib/python3.10/dist-packages/taskgraph/decision.py", line 124, in taskgraph_decision
    full_task_json = tgg.full_task_graph.to_json()
  File "/usr/local/lib/python3.10/dist-packages/taskgraph/generator.py", line 170, in full_task_graph
    return self._run_until("full_task_graph")
  File "/usr/local/lib/python3.10/dist-packages/taskgraph/generator.py", line 427, in _run_until
    k, v = next(self._run)  # type: ignore
  File "/usr/local/lib/python3.10/dist-packages/taskgraph/generator.py", line 309, in _run
    kind = kinds[kind_name]
KeyError: 'compile-marian-fork'

Error after:

Traceback (most recent call last):
  File "/builds/worker/checkouts/utils/preflight_check.py", line 554, in <module>
    main()
  File "/builds/worker/checkouts/utils/preflight_check.py", line 522, in main
    run_taskgraph(parsed_args.config, get_taskgraph_parameters())
  File "/builds/worker/checkouts/utils/preflight_check.py", line 137, in run_taskgraph
    taskgraph.actions.trigger_action_callback(
  File "/usr/local/lib/python3.10/dist-packages/taskgraph/actions/registry.py", line 344, in trigger_action_callback
    cb(Parameters(**parameters), graph_config, input, task_group_id, task_id)
  File "/builds/worker/checkouts/taskcluster/translations_taskgraph/actions/train.py", line 601, in train_action
    taskgraph_decision(
  File "/usr/local/lib/python3.10/dist-packages/taskgraph/decision.py", line 124, in taskgraph_decision
    full_task_json = tgg.full_task_graph.to_json()
  File "/usr/local/lib/python3.10/dist-packages/taskgraph/generator.py", line 170, in full_task_graph
    return self._run_until("full_task_graph")
  File "/usr/local/lib/python3.10/dist-packages/taskgraph/generator.py", line 435, in _run_until
    k, v = next(self._run)  # type: ignore
  File "/usr/local/lib/python3.10/dist-packages/taskgraph/generator.py", line 317, in _run
    raise Exception(message)
Exception: Could not find the kind "compile-marian-fork"
Available kinds:
 - "all-pipeline"
 - "all-pr-pipeline"
 - "backtranslations-mono-trg-chunk"
 - "backtranslations-mono-trg-dechunk-translations"
 - "backtranslations-mono-trg-translate"
 - "backtranslations-train-backwards-model"
 - "build-vocab"
 - "compile"
 - "continuation-corpus"
 - "continuation-model"
 - "continuation-vocab"
 - "corpus-align-backtranslations"
 - "corpus-align-distillation"
 - "corpus-align-parallel"
 - "corpus-analyze-mono"
 - "corpus-analyze-parallel"
 - "corpus-clean-mono"
 - "corpus-clean-parallel"
 - "corpus-clean-parallel-bicleaner-ai"
 - "corpus-clean-parallel-fetch-bicleaner-model"
 - "corpus-merge-devset"
 - "corpus-merge-distillation"
 - "corpus-merge-mono"
 - "corpus-merge-parallel"
 - "dataset"
 - "distillation-corpus-build-shortlist"
 - "distillation-corpus-final-filtering"
 - "distillation-mono-src-chunk"
 - "distillation-mono-src-dechunk-translations"
 - "distillation-mono-src-translate"
 - "distillation-parallel-src-chunk"
 - "distillation-parallel-src-dechunk-translations"
 - "distillation-parallel-src-extract-best"
 - "distillation-parallel-src-translate"
 - "distillation-parallel-src-translations-score"
 - "distillation-student-model-finetune"
 - "distillation-student-model-quantize"
 - "distillation-student-model-train"
 - "docker-image"
 - "evaluate"
 - "evaluate-quantized"
 - "evaluate-teacher-ensemble"
 - "export"
 - "fetch"
 - "inference"
 - "tests"
 - "toolchain"
 - "train-teacher-model"

@gregtatum gregtatum requested a review from a team as a code owner July 17, 2025 15:59
@gregtatum gregtatum requested a review from bhearsum July 17, 2025 15:59
Copy link
Contributor

@bhearsum bhearsum left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you!

@bhearsum bhearsum merged commit b640f14 into taskcluster:main Jul 21, 2025
14 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants