Tethering #3459
Tethering #3459
Conversation
samsrabin
added
enhancement
|
Great to see this, Dan, thanks! I'm going to unsubscribe for now to reduce my notification load, but please ping me with any questions or if you want me to have a look. |
|
Cool, thanks for making this a PR @djk2120. It's helpful to start to look through the code. I can tell that I'll need you to do a run-through of the code with us at some point. We should probably rebase this to b4b-dev, to come in sooner. But, we'll wait for you to feel it's closer to ready. |
|
OK, I have a first draft in place. The tether utility and a spinup stability script are in place. And then two main examples ("example1" and "sasu_spinup") for how they could be deployed. What are the next steps @samsrabin @ekluzek ? Should I present the code at an upcoming software engineering meeting (or a smaller meeting)? Does it make sense for somebody else to beta-test one or both of the examples? |
samsrabin
added
the
next
|
Excellent, thanks! Before one of us starts testing, could you merge in the ctsm5.3.073 tag and make sure it still works for you? git remote add escomp [email protected]:ESCOMP/CTSM.git
git fetch escomp
git merge --no-ff ctsm5.3.073 |
|
(Or alternatively, rebase onto that tag if that's something you're comfortable doing.) |
Update .gitmodules to cesm3_0_alpha07c Update ccs_config_cesm1.0.48 to ccs_config_cesm1.0.56. Update cime6.1.112 to cime6.1.113. Answers change in gnu and nvhpc tests on derecho (details in the PR). New bugs found and reported in issues ESCOMP#3453 FAIL MKSURFDATAESMF_...intel NLCOMP ESCOMP#3454 FAIL SUBSETDATA* tests NLCOMP PR ESCOMP#3422 Preparing beta version of tether utility
|
OK, @samsrabin. I was able to merge in the new tag uneventfully. |
|
Nice, thanks! I've marked this as "next" so we should discuss it in our next SE meeting. |
|
I'm also realizing that maybe this is actually two tools: a spinup stability python tool and a tethering utility, which I should probably split into two folders in tools/ (and potentially split into two PRs?) |
ekluzek
removed
the
next
|
@wwieder will test this in some upcoming spinup runs. @adrifoster will also test. @djk2120 let's keep this as one PR, but it'd be good if you could go ahead and split out the code for the spinup stability script. |
There was a problem hiding this comment.
Hey, Daniel I really appreciate all this work. I made a ton of comments / suggestions (and didn't even get through the PR). Some of these you can address, and others may be larger SE questions to think about. A few high level thoughts.
- most immediately I wasn't able to get the tethering example1 provided to work. Maybe we need to start there and figure out what's going on?
- (selfishly) an example for an out of the box historical case would be idea here, as that's maybe the most useful of Keith's old scripts that no longer work. Happy to itterate on this once I have a better handle on what's going on?
| ## Usage | ||
|
|
||
| 1. load a suitable python environment. | ||
| 2. python spinup_stability.py my_config.yml |
There was a problem hiding this comment.
can we just use/modify ctsm_pylib to have the conda environment needed to support this?
| #PBS -l walltime=1:00:00 | ||
| #PBS -A P93300041 | ||
| #PBS -k eod | ||
| #PBS -l select=1:ncpus=5 |
There was a problem hiding this comment.
Should the account bet set in the yaml file, or at least not hardcoded here? Maybe the queue should also not be hard coded either (or are we just supporting this on derecho)? The later is a bigger SE issue.
| - TOTSOMC | ||
| - TWS | ||
|
|
||
|
|
There was a problem hiding this comment.
Comments could be helpful in this file, e.g. what are cfs?
Maybe also clearly define what new users may have to modify, and what parts of this are likely OK to leave alone (similar to the old spinup_stability.ncl scripts)
| @@ -0,0 +1,45 @@ | |||
| # detailing the spinup case and stability thresholds | |||
| case: I1850Clm50Bgc.CPLHIST.default.pAD | |||
| hist_dir: /glade/derecho/scratch/djk2120/archive/I1850Clm50Bgc.CPLHIST.default.pAD/lnd/hist | |||
There was a problem hiding this comment.
- maybe hist_dir can pull in the case name defined above
- less important, but should the user name also be more clearly defined
| TOTVEGC: 0.02 | ||
| GPP: 0.02 | ||
| TLAI: 0.02 | ||
| TEC_gridded: 1 |
| ``` | ||
|
|
||
| ### commands.txt tips | ||
| - commands.txt should exist within ```$WDIR``` |
There was a problem hiding this comment.
OK, so I have to copy commands.txt, case.txt, and tether.sh into my $WDIR. This is OK, but again I wonder if we could set this up with single config file that user set up?
| 1) ./create_config.sh | ||
| 2) ./setup_config.sh | ||
| 3) cd to tethered_sims/c2025xxxx directory | ||
| 4) qsub segment001.job |
There was a problem hiding this comment.
do I have to modify the example1/derecho.template file to get this to work?
It would be nice to have these examples work more easily of the box, if possible.
There was a problem hiding this comment.
There's a spinup.config file here too. Does spinup-stability get called for this case, or does it need to?
| # --------------------------------------------------------- | ||
|
|
||
| tdir="/glade/u/home/djk2120/vp/scripts/" | ||
| $tdir"tether.sh" wdir commands template |
There was a problem hiding this comment.
is this tdir the same and the $WDIR noted above, or it the directory for the tethering script? Is this something that users will also have to modify?
| - only implemented for regular grids | ||
| - no sparsegrid | ||
| - no spectral element | ||
|
|
There was a problem hiding this comment.
Seems like the defaults are for a bgc case. Would it be hard to make an SP version?
There was a problem hiding this comment.
This doesn't work for me from /glade/u/home/wwieder/ctsm5.4.CMIP7_ciso/tools/tether/example1
$./create_config.sh
sed: -e expression #1, char 22: unknown option to `s'
5 participants
Description of changes
Adding a new tool to tether CTSM simulations automatically.
Specific notes
Contributors other than yourself, if any:
Advice and guinea pig feedback from @linniahawkins @samsrabin @adrifoster
CTSM Issues Fixed:
Are answers expected to change (and if so in what way)? No.
Any User Interface Changes (namelist or namelist defaults changes)? This would create a new tool for automating submission of tethered CTSM simulations, but otherwise doesn't actually change the basic CTSM user interface.
Does this create a need to change or add documentation? Did you do so? Yes, this will need documentation. I've started drafting documentation in tools/tether/README.md. It's likely that we'll want to have generic "tether" documentation, as well as specific documentation for any supported sequences, e.g. SASU spinup, TRENDY S3, etc.
Testing performed, if any: None