Can't run spinetoolbox after upgrade to newest version

[JanaTerem]

Describe your problem

I upgraded SpineToolbox, SpineOpt and SpineInterface some days ago, and fall into troubles:

Execution started at 2024-08-28T12:38:38.468
WARNING: using JuMP.Parameter in module SpineInterface conflicts with an existing identifier.
Preparing SpineOpt for sqlite:///C:\Users\Teremranova\Downloads\SignAture_2020_MAIN.sqlite?spinedbfilter=cfg%3Ascenario%3AMAIN...
ERROR: LoadError: version mismatch: DB server requires client version 8, whereas current version is 6; please update SpineInterface Stacktrace

Attach your model as a JSON file (encouraged)

No response

What versions are you using?

SpineOpt:
SpineInterface:
SpineToolbox: 0.8.3
Julia: 1.10.0

[tarskul]

Originally I sent you to Spine Toolbox developers as I thought that they were better equiped to deal with this (spine-tools/Spine-Toolbox#2919). However, it seems they bounced you back to us. I don't know the solution but I'm commenting here already such that other SpineOpt developers do not accidentally send you back.

In the meanwhile, do you have some more information on the problem? What was the original version that worked for you? Do you still have that old version? When you opened the project in the new version, did you get a prompt to upgrade the database?

[JanaTerem]

I don't know the previous version no., but yes, it was offering for db upgrade I agreed.

I try install Spine again, it was ok with SpineToolbox, but installing SpineOpt (through Julia), the following error appeared:
Installed SpineInterface ─ v0.14.7
Progress [========================================>]
ERROR: IOError: mkdir("C:\Windows\System32\jenv"; mode=0o777): permission denied (EACCES)
Stacktrace:
[1] uv_error
@ .\libuv.jl:100 [inlined]
[2] mkdir(path::String; mode::UInt16)
@ Base.Filesystem .\file.jl:185
[3] mkdir
@ .\file.jl:177 [inlined]
[4] mkpath(path::String; mode::UInt16)
@ Base.Filesystem .\file.jl:241
[5] mkpath
@ .\file.jl:235 [inlined]
[6] write_project(env::Pkg.Types.EnvCache)

Some type information was truncated. Use show(err) to see complete types.

julia> show(err)
1-element ExceptionStack:
IOError: mkdir("C:\Windows\System32\jenv"; mode=0o777): permission denied (EACCES)
Stacktrace:
[1] uv_error
@ .\libuv.jl:100 [inlined]
[2] mkdir(path::String; mode::UInt16)
@ Base.Filesystem .\file.jl:185
[3] mkdir
@ .\file.jl:177 [inlined]
[4] mkpath(path::String; mode::UInt16)
@ Base.Filesystem .\file.jl:241
[5] mkpath
@ .\file.jl:235 [inlined]
[6] write_project(env::Pkg.Types.EnvCache)
@ Pkg.Types C:\Users\janad.julia\juliaup\julia-1.10.4+0.x64.w64.mingw32\share\julia\stdlib\v1.10\Pkg\src\project.jl:195
[7] write_env(env::Pkg.Types.EnvCache; update_undo::Bool, skip_writing_project::Bool)
@ Pkg.Types C:\Users\janad.julia\juliaup\julia-1.10.4+0.x64.w64.mingw32\share\julia\stdlib\v1.10\Pkg\src\Types.jl:1113
[8] write_env
@ C:\Users\janad.julia\juliaup\julia-1.10.4+0.x64.w64.mingw32\share\julia\stdlib\v1.10\Pkg\src\Types.jl:1110 [inlined]
[9] add(ctx::Pkg.Types.Context, pkgs::Vector{Pkg.Types.PackageSpec}, new_git::Set{Base.UUID}; preserve::Pkg.Types.PreserveLevel, platform::Base.BinaryPlatforms.Platform)
@ Pkg.Operations C:\Users\janad.julia\juliaup\julia-1.10.4+0.x64.w64.mingw32\share\julia\stdlib\v1.10\Pkg\src\Operations.jl:1397
[10] add
@ C:\Users\janad.julia\juliaup\julia-1.10.4+0.x64.w64.mingw32\share\julia\stdlib\v1.10\Pkg\src\Operations.jl:1377 [inlined]
[11] add(ctx::Pkg.Types.Context, pkgs::Vector{Pkg.Types.PackageSpec}; preserve::Pkg.Types.PreserveLevel, platform::Base.BinaryPlatforms.Platform, kwargs::@kwargs{io::Base.TTY})
@ Pkg.API C:\Users\janad.julia\juliaup\julia-1.10.4+0.x64.w64.mingw32\share\julia\stdlib\v1.10\Pkg\src\API.jl:278
[12] add(pkgs::Vector{Pkg.Types.PackageSpec}; io::Base.TTY, kwargs::@kwargs{})
@ Pkg.API C:\Users\janad.julia\juliaup\julia-1.10.4+0.x64.w64.mingw32\share\julia\stdlib\v1.10\Pkg\src\API.jl:159
[13] add(pkgs::Vector{Pkg.Types.PackageSpec})
@ Pkg.API C:\Users\janad.julia\juliaup\julia-1.10.4+0.x64.w64.mingw32\share\julia\stdlib\v1.10\Pkg\src\API.jl:148
[14] add
@ C:\Users\janad.julia\juliaup\julia-1.10.4+0.x64.w64.mingw32\share\julia\stdlib\v1.10\Pkg\src\API.jl:147 [inlined]
[15] add(pkg::String)
@ Pkg.API C:\Users\janad.julia\juliaup\julia-1.10.4+0.x64.w64.mingw32\share\julia\stdlib\v1.10\Pkg\src\API.jl:146
[16] top-level scope
@ REPL[6]:1

May be you can help me with that?

[tarskul]

Hm, I have not seen that error before. I would not expect a permission error when you install SpineOpt through the Julia REPL. Which installation instructions were you following exactly?

[JanaTerem]

this one:

import Pkg # the package manager
Pkg.activate("jenv") # only needed if you want to use a virtual environment
Pkg.Registry.add("General")
Pkg.Registry.add(Pkg.RegistrySpec(url = "https://github.com/spine-tools/SpineJuliaRegistry")) # Add SpineJuliaRegistry as an available registry for your Julia
Pkg.add("SpineOpt") # Install SpineOpt from the SpineJuliaRegistry

[tarskul]

I ran that code myself just now and I did not run into issues so it seems to be specific to your system. How did you install julia?

[JanaTerem]

I'm not sure.. possible, through pyton

[tarskul]

Maybe that is not ideal for windows systems (not sure) as it seems that it has installed a 32 bit version instead of 64 (at least it seems to try to write into a folder that is more typical for 32bit, again not sure). I'll update the installation instructions to reflect this better even though this may not be the real problem.

[JanaTerem]

Really, the installation happen in
"Activating new project at C:\WINDOWS\system32\jenv"

May be it'll be better to not to use jenv?

Can it be correct Julia version for me? -
" The latest version of Julia in the release channel is 1.10.5+0.x64.w64.mingw32. You currently have 1.10.4+0.x64.w64.mingw32 installed"

[tarskul]

jenv is simply the name of the virtual environment we chose. That environment can have any name and can be located in any folder on your system. Without a virtual environment you are installing SpineOpt on your main system. That may interfere with other julia projects (perhaps in the future).

Whenever you run Pkg.activate("jenv") it will check the location you are in your terminal. If there is a virtual environment in this location it will use that one, otherwise it will create a virtual environment there. Maybe you need to open the terminal in a different location such that the virtual environment is created there. Do not forget to add this virtual environment to the settings of Spine Toolbox if you want Spine Toolbox to find the correct installation of SpineOpt (in your attempts you may have accidentally installed one on your main system).

I'll separate the code in the documentation to make that more clear at least.

[tarskul]

It may be good to rule out the possibility that two processes were trying to access that folder at the same time (e.g. an antivirus scan or whatever) by trying it again at least once more.

[JanaTerem]

ok, I'll try bit later

[tarskul]

Have you tried it once more? Did you manage with the virtual environments?

[JanaTerem]

Hi,
I still receive this kind of messages:

[tarskul]

Does this only happen for your old projects or also when you create a new one?

[JanaTerem]

I can't create a new project, only overwrite existing

[tarskul]

That is rather peculiar. It sounds like we should better get in touch to see more clearly what is going on. The spine tools user meeting would be the most appropriate place to deal with this. However the next instance (in two weeks) is going to be rather special so I'm not sure whether we'll get to it there. But perhaps we can figure something out there.

[JanaTerem]

Hi I think the problem can be in those parts of the old version which are not deleted/overwriting by installing new version. Maybe you can guide me how to delete completely SpineToolbox / SpineOpt / SpineInterface before installation of new version? Regards Dr. Jana Teremranova [cid:717d69d4-0216-4e0e-8bac-e2a7fda59c6c] Industriālās elektronikas, elektrotehnikas un enerģētikas institūts Datorzinātnes, informācijas tehnoloģijas un enerģētikas fakultāte Āzenes 12/1, Rīga www.rtu.lv<http://www.rtu.lv/>

…

________________________________ From: tarskul ***@***.***> Sent: Wednesday, September 18, 2024 2:14 PM To: spine-tools/SpineOpt.jl ***@***.***> Cc: Jana Teremranova ***@***.***>; Author ***@***.***> Subject: Re: [spine-tools/SpineOpt.jl] Can't run spinetoolbox after upgrade to newest version (Discussion #1075) INFORMĀCIJAI: E-pasta vēstules sūtītājs ir ārējs adresāts. Lūdzu pievērsiet uzmanību vēstules saturam. FYI: The e-mail is from an external sender. Please pay attention to the content of the e-mail. That is rather peculiar. It sounds like we should better get in touch to see more clearly what is going on. The spine tools user meeting would be the most appropriate place to deal with this. However the next instance (in two weeks) is going to be rather special so I'm not sure whether we'll get to it there. But perhaps we can figure something out there. — Reply to this email directly, view it on GitHub<#1075 (reply in thread)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/AZUOQZZSZO7UDAT2PCD35Z3ZXFODHAVCNFSM6AAAAABNIBKGPWVHI2DSMVQWIX3LMV43URDJONRXK43TNFXW4Q3PNVWWK3TUHMYTANRYGA2DMOA>. You are receiving this because you authored the thread.Message ID: ***@***.***>

[tarskul]

It is true that there are often some remaining parts once you have installed Spine tools. However, that typically means that the toolbox settings remains, nothing more. We mostly have to check the configuration to see whether each part points to the new files instead of the old files.

I suggest we meet on gitter and schedule a meeting to get this sorted out for you.

But if you do insist on removing the old installation, I can think of at least these steps:

• Open julia in the terminal and remove the spineopt and spineinterface packages (check each julia environment that you have made)
• If you have stored the source files of spineopt/spineinterface/spine toolbox, delete these files
• Check the environment variables in the registry of your computer and delete everything spine tools related. Be careful not to remove anything else as this may cause issues on your computer.

[JanaTerem]

I'm moved forward a lot, clearing and removing all programs including spinetoolbox, spineopt, spineinterface, python and julia and re-installing them.

Now the error appeared, seems, because of parameter not included in Spine template. I tried to create a new project using SpineToolbox template:

ERROR: LoadError: UndefVarError: unit_incremental_heat_rate not defined.

I did not found it in parameters, but seems that program needs it for correct work.

[tarskul]

Try first loading an empty SpineOpt template before loading another template. We are seeing issues that the tutorial files pass the unit tests despite containing older data structures. The incremental heat rate has recently been replaced by another parameter but seems to be remaining in the tutorial files. We are unsure yet how to make sure that the tutorial files do not pass the unit test if the data structure is wrong. Regardless, an intermediate solution seems to be to first load an empty SpineOpt template.

[JanaTerem]

I used empty template for the new project, and it was ok, then I created simple system model as described in tutorial, and fall in problem:

Warning: Some items are missing from the input database.
│ We'll assume sensitive defaults for any missing parameter definitions,
│ and empty collections for any missing classes.
│ SpineOpt might still be able to run, but otherwise you'd need to check your input database.
│
│ Missing item list follows:
│
│ parameters unit_idle_heat_rate
│ unit_incremental_heat_rate
│
└ @ SpineOpt C:\Users\janad.julia\packages\SpineOpt\ZV0Yy\src\run_spineopt.jl:215
ERROR: LoadError: MethodError: no method matching zero(::Symbol)

Closest candidates are:
zero(::Type{Union{}}, Any...)
@ Base number.jl:310
zero(::Type{Pkg.Resolve.VersionWeight})
@ Pkg C:\Users\janad\AppData\Local\julias\julia-1.10\share\julia\stdlib\v1.10\Pkg\src\Resolve\versionweights.jl:15
zero(::Type{LibGit2.GitHash})
@ LibGit2 C:\Users\janad\AppData\Local\julias\julia-1.10\share\julia\stdlib\v1.10\LibGit2\src\oid.jl:221

[tarskul]

How did you add the empty template? Through 'import' or with the 'load template' tool? Because at first glance it seems that your template version does correspond to your SpineOpt version. The version error can typically be avoided with the 'load template' tool (I think).

[JanaTerem]

I used 'load template' tool.

…

________________________________ From: tarskul ***@***.***> Sent: Friday, October 4, 2024 3:30 PM To: spine-tools/SpineOpt.jl ***@***.***> Cc: Jana Teremranova ***@***.***>; Author ***@***.***> Subject: Re: [spine-tools/SpineOpt.jl] Can't run spinetoolbox after upgrade to newest version (Discussion #1075) INFORMĀCIJAI: E-pasta vēstules sūtītājs ir ārējs adresāts. Lūdzu pievērsiet uzmanību vēstules saturam. FYI: The e-mail is from an external sender. Please pay attention to the content of the e-mail. How did you add the empty template? Through 'import' or with the 'load template' tool? Because at first glance it seems that your template version does correspond to your SpineOpt version. — Reply to this email directly, view it on GitHub<#1075 (reply in thread)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/AZUOQZYG4D6PAHG6DQXSDFDZZ2C5DAVCNFSM6AAAAABNIBKGPWVHI2DSMVQWIX3LMV43URDJONRXK43TNFXW4Q3PNVWWK3TUHMYTAOBUGM4DIMI>. You are receiving this because you authored the thread.Message ID: ***@***.***>

[tarskul]

I really think we need to examine your system in real time because this goes very slow. Regardless, have you tried loading in the tutorial simple system file (after loading the empty template) to see whether your installation works as intended before moving on to making tutorials yourself?

[JanaTerem]

How can we meet in real time?
Yes, I tried to load tutorial simple system file after loading the empty template. The results: ERROR: LoadError: UndefVarError: unit_incremental_heat_rate not defined

[tarskul]

In the readme there is a link to our the community gitter room. In gitter it will be easier to get in touch.

[JanaTerem]

Today, trying to run the simple system project - all is ok, but
running my model (going perfectly yesterday) - execution stuck in reading run SpineOpt:

julia> # Running julia run_spineopt.jl http://127.0.0.1:63117 http://127.0.0.1:63120
2-element Vector{String}:
"http://127.0.0.1:63117"
"http://127.0.0.1:63120"
WARNING: using JuMP.parameter_value in module SpineInterface conflicts with an existing identifier.
WARNING: using JuMP.Parameter in module SpineInterface conflicts with an existing identifier.

Running SpineOpt...
[SpineOpt version 0.8.3 (git hash: 1b515b5)]
[SpineInterface version 0.14.8 (git hash: 4502ba4c804c72452a59a3fb3d35364d9a01cec6)]
Execution started at 2024-10-08T10:40:44.098
Reading input data from sqlite:///C:\Users\janad\Downloads\SignAture_2020_MAIN (3).sqlite?spinedbfilter=cfg%3Ascenario%3AMAIN...

julia>

[tarskul]

Sorry, I can't deduce the underlying issue from this output. I don't have another idea than the classic turning your computer off and on again and trying again with a new project where you check whether everything is set correctly (including upgrade=true).

[m-bah]

Hi @JanaTerem. Were you able to resolve your issue? I am having a similar problem when trying to run simple models Let me know

[tarskul]

@m-bah Can you create a separate discussion where you try to be more specific what your issue is? There were quite a few changes to SpineOpt since this discussion so it may be a separate problem and it may be better to close this discussion.