Running a pipeline as a background job in target markdown

[jl5000]

Help

• I understand and agree to https://books.ropensci.org/targets/help.html.

Description

I'd really like to call tar_make(as_job = TRUE) in a target markdown so that I can see progress in real-time.

However, I'm finding that the document is rendering even though the pipeline isn't finished. I decided to put while(tar_active()){Sys.sleep(1)} after the tar_make() call, but it didn't seem to make any difference.

There are probably two issues here:

1. I can't make the render pause until the background job is finished;
2. tar_active() doesn't seem to do what the name suggests.

Any advice you can provide would be appreciated.

[wlandau]

By design, tar_make(as_job = TRUE) returns control to you immediately. That way, your R console is free to run e.g. tar_progress_summary() and tar_visnetwork() while the pipeline runs in the background. If you want to block the R Markdown document until the pipeline finishes, you could get the PID of the pipeline from tar_process(), then use the ps package to check if the process is still alive (from inside your while() loop).

tar_active() is used to check if the current context is inside a call to tar_make() (either in a target or somewhere in _targets.R). The main use case is if you want custom code in _targets.R to know if you are running the pipeline with tar_make() or just inspecting it with e.g. tar_visnetwork().