cyto_transform(gs) Error: 'trans' must be a list of transformer objects

[EmelineFavreau]

Describe the bug
cyto_transform(gs) is giving a new error: Error: 'trans' must be a list of transformer objects (generated by scales::trans_new method)

To Reproduce
Follow the vignette: https://dillonhammill.github.io/CytoExploreR/articles/CytoExploreR.html

# Bioconductor
install.packages("BiocManager")
# Install cytoinstaller
remotes::install_github("RGLab/cytoinstaller")
# Install cytoverse packages
cytoinstaller::install_cyto(bioc_ver = "devel")
# CytoExploreRData 
devtools::install_github("DillonHammill/CytoExploreRData")
# CytoExploreR 
devtools::install_github("DillonHammill/CytoExploreR")
# Load required packages
library(CytoExploreR)
library(CytoExploreRData)

# Compensation FCS Files
cyto_save(Compensation, 
          save_as = "Compensation-Samples")

# Activation FCS Files
cyto_save(Activation,
          save_as = "Activation-Samples")

# Setup compensation controls
gs <- cyto_setup("Compensation-Samples",
                 gatingTemplate = "Compensation-gatingTemplate.csv")

# Transform fluorescent channels - default logicle transformations
gs <- cyto_transform(gs)

Expected behavior
I expected the object to be created without error (which was the case two weeks ago when I last ran it)

sessionInfo():

R version 4.3.0 (2023-04-21)
Platform: aarch64-apple-darwin20 (64-bit)
Running under: macOS Ventura 13.5.2

Matrix products: default
BLAS:   /System/Library/Frameworks/Accelerate.framework/Versions/A/Frameworks/vecLib.framework/Versions/A/libBLAS.dylib 
LAPACK: /Library/Frameworks/R.framework/Versions/4.3-arm64/Resources/lib/libRlapack.dylib;  LAPACK version 3.11.0

locale:
[1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8

time zone: Europe/London
tzcode source: internal

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
 [1] CytoExploreRData_1.0.3 shiny_1.8.0            cytoqc_0.99.2          strex_1.6.1            stringr_1.5.1          flowAI_1.30.0          dplyr_1.1.4           
 [8] magrittr_2.0.3         CytoExploreR_1.1.0     openCyto_2.12.0        flowWorkspace_4.12.1   flowCore_2.12.2       

loaded via a namespace (and not attached):
  [1] RBGL_1.76.0         remotes_2.4.2.1     rlang_1.1.2         matrixStats_1.1.0   compiler_4.3.0      callr_3.7.3         png_0.1-8           systemfonts_1.0.5   vctrs_0.6.4        
 [10] reshape2_1.4.4      profvis_0.3.8       rvest_1.0.3         pkgconfig_2.0.3     crayon_1.5.2        fastmap_1.1.1       ellipsis_0.3.2      utf8_1.2.4          promises_1.2.1     
 [19] ncdfFlow_2.46.0     rmarkdown_2.25      sessioninfo_1.2.2   ps_1.7.5            graph_1.78.0        purrr_1.0.2         xfun_0.41           zlibbioc_1.46.0     cachem_1.0.8       
 [28] jsonlite_1.8.7      later_1.3.1         prettyunits_1.2.0   parallel_4.3.0      R6_2.5.1            bslib_0.6.0         stringi_1.8.2       RColorBrewer_1.1-3  reticulate_1.34.0  
 [37] pkgload_1.3.3       jquerylib_0.1.4     Rcpp_1.0.11         knitr_1.45          usethis_2.2.2       zoo_1.8-12          httpuv_1.6.12       Matrix_1.6-3        tidyselect_1.2.0   
 [46] yaml_2.3.7          rstudioapi_0.15.0   miniUI_0.1.1.1      curl_5.1.0          processx_3.8.2      pkgbuild_1.4.2      lattice_0.22-5      tibble_3.2.1        plyr_1.8.9         
 [55] Biobase_2.60.0      askpass_1.2.0       evaluate_0.23       Rtsne_0.16          EmbedSOM_2.1.2      desc_1.4.2          urlchecker_1.0.1    xml2_1.3.5          pillar_1.9.0       
 [64] KernSmooth_2.23-22  DT_0.30             stats4_4.3.0        generics_0.1.3      rprojroot_2.0.4     S4Vectors_0.38.2    ggplot2_3.4.4       munsell_0.5.0       scales_1.3.0       
 [73] xtable_1.8-4        glue_1.6.2          changepoint_2.2.4   tools_4.3.0         colortable_0.3.0    robustbase_0.99-0   data.table_1.14.8   RSpectra_0.16-1     webshot_0.5.5      
 [82] fs_1.6.3            visNetwork_2.1.2    XML_3.99-0.15       grid_4.3.0          flowClust_3.38.0    tidyr_1.3.0         RProtoBufLib_2.12.1 umap_0.2.10.0       devtools_2.4.5     
 [91] colorspace_2.1-0    cli_3.6.1           rsvd_1.0.5          kableExtra_1.3.4    rhandsontable_0.3.8 fansi_1.0.5         cytolib_2.12.1      viridisLite_0.4.2   svglite_2.1.2      
[100] Rgraphviz_2.44.0    gtable_0.3.4        DEoptimR_1.1-3      sass_0.4.7          digest_0.6.33       BiocGenerics_0.46.0 htmlwidgets_1.6.3   memoise_2.0.1       htmltools_0.5.7    
[109] lifecycle_1.0.4     httr_1.4.7          mime_0.12           openssl_2.1.1       MASS_7.3-60        

Additional context
I have been running this code several times without an issue, and now I see this issue. I do not understand why. Is it something to do with the package scales?

[SamGG]

Look at RGLab/flowWorkspace#394

[mikejiang]

The fix is already in bioconductor latest release

[EmelineFavreau]

Thanks for commenting and providing a fix. Can you explain how to get the latest release of bioconductor?

[SamGG]

Did you try to update your current installation?
https://www.bioconductor.org/install/#update-bioconductor-packages
The current release is 3.18.

[EmelineFavreau]

Thanks, I followed your instructions but still getting the error:

BiocManager::install(version = "3.18")

##### load libraries
library(CytoExploreR)
library(CytoExploreRData)

##### follow tutorial: https://dillonhammill.github.io/CytoExploreR/articles/CytoExploreR.html
cyto_save(Compensation, 
         save_as = "Compensation-Samples")

gs <- cyto_setup("Compensation-Samples",
               gatingTemplate = "Compensation-gatingTemplate.csv")

gs <- cyto_transform(gs)

# the error is still there
Error: 'trans' must be a list of transformer objects (generated by scales::trans_new method)

# list session
sessionInfo()
R version 4.3.2 (2023-10-31)
Platform: aarch64-apple-darwin20 (64-bit)
Running under: macOS Ventura 13.5.2

Matrix products: default
BLAS:   /System/Library/Frameworks/Accelerate.framework/Versions/A/Frameworks/vecLib.framework/Versions/A/libBLAS.dylib 
LAPACK: /Library/Frameworks/R.framework/Versions/4.3-arm64/Resources/lib/libRlapack.dylib;  LAPACK version 3.11.0

locale:
[1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8

time zone: Europe/London
tzcode source: internal

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
 [1] shiny_1.8.0          cytoqc_0.99.2        strex_1.6.1          stringr_1.5.1        flowAI_1.32.0        dplyr_1.1.4         
 [7] magrittr_2.0.3       CytoExploreR_1.1.0   openCyto_2.14.0      flowWorkspace_4.14.0 flowCore_2.14.0      BiocManager_1.30.22 

loaded via a namespace (and not attached):
 [1] RBGL_1.78.0         rlang_1.1.2         matrixStats_1.1.0   compiler_4.3.2      png_0.1-8           systemfonts_1.0.5  
 [7] vctrs_0.6.5         reshape2_1.4.4      rvest_1.0.3         pkgconfig_2.0.3     crayon_1.5.2        fastmap_1.1.1      
[13] ellipsis_0.3.2      utf8_1.2.4          promises_1.2.1      ncdfFlow_2.48.0     rmarkdown_2.25      graph_1.80.0       
[19] purrr_1.0.2         xfun_0.41           zlibbioc_1.48.0     cachem_1.0.8        jsonlite_1.8.8      later_1.3.2        
[25] parallel_4.3.2      R6_2.5.1            bslib_0.6.1         stringi_1.8.2       RColorBrewer_1.1-3  reticulate_1.34.0  
[31] jquerylib_0.1.4     Rcpp_1.0.11         knitr_1.45          zoo_1.8-12          httpuv_1.6.13       Matrix_1.6-4       
[37] tidyselect_1.2.0    yaml_2.3.7          rstudioapi_0.15.0   lattice_0.22-5      tibble_3.2.1        plyr_1.8.9         
[43] Biobase_2.62.0      askpass_1.2.0       evaluate_0.23       Rtsne_0.17          EmbedSOM_2.1.2      xml2_1.3.6         
[49] pillar_1.9.0        KernSmooth_2.23-22  DT_0.31             stats4_4.3.2        generics_0.1.3      S4Vectors_0.40.2   
[55] ggplot2_3.4.4       munsell_0.5.0       scales_1.3.0        xtable_1.8-4        glue_1.6.2          changepoint_2.2.4  
[61] tools_4.3.2         colortable_0.3.0    robustbase_0.99-1   data.table_1.14.10  RSpectra_0.16-1     webshot_0.5.5      
[67] fs_1.6.3            visNetwork_2.1.2    XML_3.99-0.16       grid_4.3.2          flowClust_3.40.0    tidyr_1.3.0        
[73] RProtoBufLib_2.14.0 umap_0.2.10.0       colorspace_2.1-0    cli_3.6.2           rsvd_1.0.5          kableExtra_1.3.4   
[79] rhandsontable_0.3.8 fansi_1.0.6         cytolib_2.14.0      viridisLite_0.4.2   svglite_2.1.3       Rgraphviz_2.46.0   
[85] gtable_0.3.4        DEoptimR_1.1-3      sass_0.4.8          digest_0.6.33       BiocGenerics_0.48.1 htmlwidgets_1.6.4  
[91] memoise_2.0.1       htmltools_0.5.7     lifecycle_1.0.4     httr_1.4.7          mime_0.12           openssl_2.1.1      
[97] MASS_7.3-60 ```

[SamGG]

Your flowWorkspace version is still 4.14.0. The update by Mike is available in 4.14.1.
You can check it at https://code.bioconductor.org/browse/flowWorkspace/blob/RELEASE_3_18/R/GatingSet_Methods.R#L1102
The current version for all operating systems is 4.14.1 except for MacOS (arm64). I guess this is the one you got. Check the bottom of the page at https://bioconductor.org/packages/flowWorkspace/.
As I don't use Mac, I can't advise to install from source, because I can deal with problems that could occur.
@mikejiang an advice?
@DillonHammill any idea?

[EmelineFavreau]

Thanks for explaining the issue @SamGG. Yes, my mac is arm64 (M2 chip). Is there an timeline as to when the version 4.14.1 of flowWorkspace will be available for MacOsS arm64?

[mikejiang]

different architectures from bioconductor are inconsistent about scales package version

  namespace ‘scales’ 1.2.1 is being loaded, but >= 1.3.0 is required

arm is lagging behind the rest of others, which is assume they will make patch pretty soon (feel free to send a message to their core team to report this hopefully get them act properly)

[DillonHammill]

Thanks for fixing this, @mikejiang. ARM builds typically lag behind other architectures by a week or so, so it should be available for download soon. @EmelineFavreau if you require a stable version of CytoExploreR to use in the meantime, I'd recommend using the docker image: https://github.com/djhammill/CytoExploreR-Docker. This docker image contains version 2.0.0 of CytoExploreR, so there might be some differences, please refer to the documentation and report any issues on the CytoExploreR GitHub page: https://github.com/DillonHammill/CytoExploreR.

[EmelineFavreau]

Hi all, can you please help me understand where the issue is?

@DillonHammill, I tried to use your Docker container, but the Rstudio session aborted a couple of times when data were being produced (using the downstream analysis package flowAI). Additionally, the usually super handy gating 'cyto_gate_draw' failed to delivered. Indeed, the result of the mouse pointer was not precise (ie landed roughly at coordinates x-2 and y+3).

@mikejiang , you refers to the scales package lagging to be updated for arm64 flavour. But on the website it looks like the version 1.3.0 is now available (https://cran.r-project.org/package=scales).

Would you be able to update flowWorkspace now that scales is available, if this is possible at all?

Many thanks!

[mikejiang]

flowWorkspace source is already up to date, if you are asking for binary for mac from bioc, that has to wait until bioc team sort out the build problem in their mac server

[rebeccaongmtu]

Frustratingly, arm64 is now two versions behind (still at 4.14.0) almost 3 months later. I cannot move forward on my project. Who exactly do I pester about this?

[gfinak]

From the build reports it looks like RProtoBufLib (a dependency of the flow tools) isn't getting installed due to a lock file error and that is cascading down to all the flow packages. This looks like a Bioc build system issue to me. Perhaps an email to the biconductor maintainers could help illuminate.

[djhammill]

That makes sense @gfinak, we have merged a fix to flowWorkspace that will work with new and old versions of scales (RGLab/flowWorkspace#397) but perhaps with the build errors that hasn't made its way to BioC yet.

For users struggling with ARM64 at the moment, I'd recommend using one of the cytoexplorer-devel docker images from docker hub: https://hub.docker.com/r/dhammill/cytoexplorer-devel/tags. These images are well maintained and automatically built whenever a change is made to CytoExploreR for both ARM64 and AMD64 architectures.

I haven't been able to test the image locally (I'm on AMD64) but the image has built successfully so it should provide a stable cytoverse working environment that includes CytoExploreR. @EmelineFavreau when using CytoExploreR through docker you need to make sure that your browser zoom is set to 100% (to avoid weird gating issues), you can also run graphics.off() and refresh the browser page if you run into issues. I would also recommend using cyto_clean() instead of FlowAI directly as it is a stripped back version that is less likely to cause your session to crash.

Example docker command below, just need to modify the mount source path to the desired location on your machine:

docker run --rm -dit --mount type=bind,source=/path/to/mount,destination=/home/rstudio/project -p 8787:8787 -e PASSWORD=cytoexplorer dhammill/cytoexplorer-devel:latest

[rebeccaongmtu]

Thanks @djhammill! I've never used Docker, so I decided to first try running with previous versions of scales (v. 1.2.0) and ggplot2 (v. 3.4.4) and that worked. So I can at least get the code to run again while waiting for the updates.