Error getting Stan progress output from parallelized brms model, positron opens port in browser with error message

[milesalanmoore]

System details:

Positron and OS details:

M1 2020 Macbook Pro
Sonoma 14.1.1

Positron Version: 2024.12.0 (Universal) build 96
Code - OSS Version: 1.93.0
Commit: c5ce275dc502f6b15433b271802cb33e1ba5ef68
Date: 2024-11-28T02:47:29.898Z (1 mo ago)
Electron: 30.4.0
Chromium: 124.0.6367.243
Node.js: 20.15.1
V8: 12.4.254.20-electron.0
OS: Darwin arm64 23.1.0

Interpreter details:

> sessionInfo()
R version 4.4.0 (2024-04-24)
Platform: aarch64-apple-darwin20
Running under: macOS Sonoma 14.1.1

Matrix products: default
BLAS:   /System/Library/Frameworks/Accelerate.framework/Versions/A/Frameworks/vecLib.framework/Versions/A/libBLAS.dylib 
LAPACK: /Library/Frameworks/R.framework/Versions/4.4-arm64/Resources/lib/libRlapack.dylib;  LAPACK version 3.12.0

locale:
[1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8

time zone: America/Denver
tzcode source: internal

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
[1] lme4_1.1-35.5 Matrix_1.7-0  brms_2.22.0   Rcpp_1.0.13  

loaded via a namespace (and not attached):
 [1] bridgesampling_1.1-2 tensorA_0.36.2.1     utf8_1.2.4          
 [4] generics_0.1.3       stringi_1.8.4        lattice_0.22-6      
 [7] magrittr_2.0.3       grid_4.4.0           mvtnorm_1.3-1       
[10] jsonlite_1.8.8       processx_3.8.4       pkgbuild_1.4.4      
[13] backports_1.5.0      ps_1.7.7             gridExtra_2.3       
[16] Brobdingnag_1.2-9    fansi_1.0.6          QuickJSR_1.1.3      
[19] scales_1.3.0         codetools_0.2-20     abind_1.4-5         
[22] cli_3.6.3            rlang_1.1.4          splines_4.4.0       
[25] munsell_0.5.1        StanHeaders_2.32.8   tools_4.4.0         
[28] rstan_2.32.6         inline_0.3.19        parallel_4.4.0      
[31] nloptr_2.1.1         rstantools_2.4.0     checkmate_2.3.2     
[34] coda_0.19-4.1        minqa_1.2.8          dplyr_1.1.4         
[37] colorspace_2.1-1     ggplot2_3.5.1        boot_1.3-30         
[40] curl_5.2.1           vctrs_0.6.5          posterior_1.6.0     
[43] R6_2.5.1             matrixStats_1.3.0    stats4_4.4.0        
[46] lifecycle_1.0.4      stringr_1.5.1        V8_5.0.0            
[49] MASS_7.3-60.2        callr_3.7.6          pkgconfig_2.0.3     
[52] RcppParallel_5.1.7   pillar_1.9.0         gtable_0.3.5        
[55] loo_2.8.0            glue_1.7.0           data.table_1.15.4   
[58] tibble_3.2.1         tidyselect_1.2.1     bayesplot_1.11.1    
[61] nlme_3.1-164         compiler_4.4.0       distributional_0.4.0

Describe the issue:

When running a brms:brm() model, if the user parallelizes the sampling via the cores argument as in brms::brm(y~x, chains = 4, cores=4) something unexpected happens. Although the model runs without errors in the R console and fits correctly, a port opens in a browser http://127.0.0.1:55309/f2f43fe2/file5568b9aee29_StanProgress.html with the text:

Cannot GET /var/folders/lm/4jkt0_jd37j4c_68xmrxp7f00000gn/T//Rtmpp3I1Pz/file5568b9aee29_StanProgress.txt

If the models is not run in parallel, the progress output prints and port is opened.

Steps to reproduce the issue:

Directly in the console:

library(brms)

# This runs without issue
brms::brm(
  mpg ~ hp * cyl, 
  data = mtcars)

# This opens the browser tab
m_mpg = brm(
  mpg ~ hp * cyl, 
  data = mtcars,
chains=4, cores=4)

Expected or desired behavior:

The Stan progress should print to console. From my own system terminal (iTerm):

> brm(
+ mpg ~ hp * cyl,
+ data=mtcars,
+ chains=4, cores=4)
Compiling Stan program...
Start sampling

SAMPLING FOR MODEL 'anon_model' NOW (CHAIN 1).
Chain 1:
Chain 1: Gradient evaluation took 2.9e-05 seconds
Chain 1: 1000 transitions using 10 leapfrog steps per transition would take 0.29 seconds.
Chain 1: Adjust your expectations accordingly!
Chain 1:
Chain 1:

SAMPLING FOR MODEL 'anon_model' NOW (CHAIN 2).
Chain 2:
Chain 2: Gradient evaluation took 1.3e-05 seconds
Chain 2: 1000 transitions using 10 leapfrog steps per transition would take 0.13 seconds.
Chain 2: Adjust your expectations accordingly!
Chain 2:
Chain 2:
Chain 1: Iteration:    1 / 2000 [  0%]  (Warmup)

SAMPLING FOR MODEL 'anon_model' NOW (CHAIN 3).
Chain 3:
Chain 3: Gradient evaluation took 1e-05 seconds
Chain 3: 1000 transitions using 10 leapfrog steps per transition would take 0.1 seconds.
Chain 3: Adjust your expectations accordingly!
Chain 3:
Chain 3:

SAMPLING FOR MODEL 'anon_model' NOW (CHAIN 4).
Chain 4:
Chain 4: Gradient evaluation took 1.3e-05 seconds
Chain 4: 1000 transitions using 10 leapfrog steps per transition would take 0.13 seconds.
Chain 4: Adjust your expectations accordingly!
Chain 4:
Chain 4:
Chain 2: Iteration:    1 / 2000 [  0%]  (Warmup)
Chain 4: Iteration:    1 / 2000 [  0%]  (Warmup)
Chain 3: Iteration:    1 / 2000 [  0%]  (Warmup)
Chain 2: Iteration:  200 / 2000 [ 10%]  (Warmup)
Chain 2: Iteration:  400 / 2000 [ 20%]  (Warmup)
Chain 1: Iteration:  200 / 2000 [ 10%]  (Warmup)
Chain 2: Iteration:  600 / 2000 [ 30%]  (Warmup)
Chain 1: Iteration:  400 / 2000 [ 20%]  (Warmup)
Chain 4: Iteration:  200 / 2000 [ 10%]  (Warmup)
Chain 3: Iteration:  200 / 2000 [ 10%]  (Warmup)
Chain 2: Iteration:  800 / 2000 [ 40%]  (Warmup)
Chain 1: Iteration:  600 / 2000 [ 30%]  (Warmup)
Chain 4: Iteration:  400 / 2000 [ 20%]  (Warmup)
Chain 2: Iteration: 1000 / 2000 [ 50%]  (Warmup)
Chain 2: Iteration: 1001 / 2000 [ 50%]  (Sampling)
Chain 3: Iteration:  400 / 2000 [ 20%]  (Warmup)
Chain 1: Iteration:  800 / 2000 [ 40%]  (Warmup)
Chain 4: Iteration:  600 / 2000 [ 30%]  (Warmup)
Chain 2: Iteration: 1200 / 2000 [ 60%]  (Sampling)
Chain 3: Iteration:  600 / 2000 [ 30%]  (Warmup)
Chain 1: Iteration: 1000 / 2000 [ 50%]  (Warmup)
Chain 1: Iteration: 1001 / 2000 [ 50%]  (Sampling)
Chain 4: Iteration:  800 / 2000 [ 40%]  (Warmup)
Chain 3: Iteration:  800 / 2000 [ 40%]  (Warmup)
Chain 2: Iteration: 1400 / 2000 [ 70%]  (Sampling)
Chain 4: Iteration: 1000 / 2000 [ 50%]  (Warmup)
Chain 1: Iteration: 1200 / 2000 [ 60%]  (Sampling)
Chain 4: Iteration: 1001 / 2000 [ 50%]  (Sampling)
Chain 3: Iteration: 1000 / 2000 [ 50%]  (Warmup)
Chain 3: Iteration: 1001 / 2000 [ 50%]  (Sampling)
Chain 2: Iteration: 1600 / 2000 [ 80%]  (Sampling)
Chain 1: Iteration: 1400 / 2000 [ 70%]  (Sampling)
Chain 4: Iteration: 1200 / 2000 [ 60%]  (Sampling)
Chain 3: Iteration: 1200 / 2000 [ 60%]  (Sampling)
Chain 1: Iteration: 1600 / 2000 [ 80%]  (Sampling)
Chain 2: Iteration: 1800 / 2000 [ 90%]  (Sampling)
Chain 3: Iteration: 1400 / 2000 [ 70%]  (Sampling)
Chain 4: Iteration: 1400 / 2000 [ 70%]  (Sampling)
Chain 1: Iteration: 1800 / 2000 [ 90%]  (Sampling)
Chain 2: Iteration: 2000 / 2000 [100%]  (Sampling)
Chain 2:
Chain 2:  Elapsed Time: 0.112 seconds (Warm-up)
Chain 2:                0.103 seconds (Sampling)
Chain 2:                0.215 seconds (Total)
Chain 2:
Chain 3: Iteration: 1600 / 2000 [ 80%]  (Sampling)
Chain 4: Iteration: 1600 / 2000 [ 80%]  (Sampling)
Chain 1: Iteration: 2000 / 2000 [100%]  (Sampling)
Chain 1:
Chain 1:  Elapsed Time: 0.155 seconds (Warm-up)
Chain 1:                0.089 seconds (Sampling)
Chain 1:                0.244 seconds (Total)
Chain 1:
Chain 3: Iteration: 1800 / 2000 [ 90%]  (Sampling)
Chain 4: Iteration: 1800 / 2000 [ 90%]  (Sampling)
Chain 3: Iteration: 2000 / 2000 [100%]  (Sampling)
Chain 3:
Chain 3:  Elapsed Time: 0.157 seconds (Warm-up)
Chain 3:                0.087 seconds (Sampling)
Chain 3:                0.244 seconds (Total)
Chain 3:
Chain 4: Iteration: 2000 / 2000 [100%]  (Sampling)
Chain 4:
Chain 4:  Elapsed Time: 0.155 seconds (Warm-up)
Chain 4:                0.107 seconds (Sampling)
Chain 4:                0.262 seconds (Total)
Chain 4:
 Family: gaussian
  Links: mu = identity; sigma = identity
Formula: mpg ~ hp * cyl
   Data: mtcars (Number of observations: 32)
  Draws: 4 chains, each with iter = 2000; warmup = 1000; thin = 1;
         total post-warmup draws = 4000

Regression Coefficients:
          Estimate Est.Error l-95% CI u-95% CI Rhat Bulk_ESS Tail_ESS
Intercept    50.97      7.07    36.76    64.83 1.00     1278     1625
hp           -0.17      0.07    -0.32    -0.03 1.00     1381     1586
cyl          -4.15      1.07    -6.25    -1.96 1.00     1414     1681
hp:cyl        0.02      0.01     0.00     0.04 1.00     1293     1372

Further Distributional Parameters:
      Estimate Est.Error l-95% CI u-95% CI Rhat Bulk_ESS Tail_ESS
sigma     3.09      0.45     2.38     4.12 1.00     2002     2074

Draws were sampled using sampling(NUTS). For each parameter, Bulk_ESS
and Tail_ESS are effective sample size measures, and Rhat is the potential
scale reduction factor on split chains (at convergence, Rhat = 1).

This (expected) behavior also occurs in Rstudio, and an R-interactive session from Visual Studio Code, leading me to believe it may be an issue with Positron or Ark?

Were there any error messages in the UI, Output panel, or Developer Tools console?

Yes, opens in a browser http://127.0.0.1:55309/f2f43fe2/file5568b9aee29_StanProgress.html with the text:

Cannot GET /var/folders/lm/4jkt0_jd37j4c_68xmrxp7f00000gn/T//Rtmpp3I1Pz/file5568b9aee29_StanProgress.txt

Edit: Just to say, fantastic IDE, I am a huge fan so far!

[milesalanmoore]

Following up as I also tried this on my Window's PC and get the same behavior/error.

System specifications:

x64
AMD Ryzen 7 7800X3D 8-Core Processor
Windows 11 Home 26100.2605

Positron specifications:

Positron Version: 2024.12.0 (system setup) build 96
Code - OSS Version: 1.93.0
Commit: c5ce275dc502f6b15433b271802cb33e1ba5ef68
Date: 2024-11-28T02:50:45.229Z
Electron: 30.4.0
Chromium: 124.0.6367.243
Node.js: 20.15.1
V8: 12.4.254.20-electron.0
OS: Windows_NT x64 10.0.26100

[juliasilge]

Looks like this comes from the rstan package:
https://github.com/stan-dev/rstan/blob/9b8d8fe715f1d51063621e6ca01d6265b02356fe/rstan/rstan/R/stanmodel-class.R#L647

Looks like the problem is the use in that file of Sys.getenv("RSTUDIO"), which is not set to 1 in Positron.

Could you open an issue on the rstan repo asking if they would be willing to generalize their IDE checks to include Positron? They can do a similar check like identical(Sys.getenv("POSITRON"), "1".

As a workaround for the time being, you can trick rstan into thinking it is running in RStudio by running Sys.setenv(RSTUDIO = 1) before running your modeling code; from my testing, it looks like everything else is working well since we provide the needed rstudioapi functionality.

[milesalanmoore]

Thank you for the response! Yes, of course. I've opened an issue at rstan, here.

[juliasilge]

Wonderful! Thank you so much for opening that issue; we're totally happy to help out if appropriate/needed.