Cmdstan 2.23 release candidate

[rok-cesnovar]

Summary:

We are going to have a release candidate for this next release. Opening an issue to track everything about it.

@serban-nicusor-toptal can you make release candidate tarballs at the end of your work day? On the very unlikely chance something gets merged after that we will update tomorrow. But I doubt there will me much action. Once we have the release candidate tarballs ready and we can proceed I will make a post on Discourse for users to try it out.

Draft of the text that will go with the announcement:

New features:
- introduces `reduce_sum` and `reduce_sum_static`
- added OpenCL (GPU) support for GLMs when using the "~" syntax (previously only supported  with target +=)
- upgraded to Sundials 5.2.0
- including files snytax supports < > brackets

Notable user-facing bugfixes:
 - fix to enable compiling of user-defined rng functions 
 - fixed derivatives of log_sum_exp for large arguments
 - fix for a bug with normal_id_glm when sigma is constant
 - better checks for positive-definitnes in mdivide_left_spd
 - more consistent throwing behaviour on QR functions
 - clearer messages when using a variable name as a keyword
 - arbitrary spaces allowed betweeen words in "transformed data" and "transformed parameters"

There are also some ongoing refactors that don't have a direct user-facing impact at the moment (generalizing functions in the math library for example), but will in the future. We want to also make sure those didn't break any of your models.

Current Version:

v2.22

[rok-cesnovar]

@wds15 @bbbales2 would you mind helping me expand the first line on reduce_sum? Also, do we have a simple example we can show to people that isnt too complex but has some speedup?
No need to be too polished right now, so users can test it with the release candidate. Should I just use the base or nbme model?

[wds15]

Sure, I am happy to give a hand. I have some time tomorrow for this.

What a release!!!

[wds15]

Actually...we can point people to the intro Ben wrote which is a rewrite of an intro to Map rect. In addition we have Good Material in the Stan user guide already. That should both do it, I think.

[rok-cesnovar]

Oh yeah, forgot about that. That is more than enough.

[rok-cesnovar]

@serban-nicusor-toptal let me know when you make the release candidate tarballs so I can post the news. Thanks!

[serban-nicusor-toptal]

Good morning @rok-cesnovar
You can find the candidate release here with /bin from stanc3 nightly. Sorry for being a bit late!
Is there anything else I can do to prepare us ahead for the next release ?

[rok-cesnovar]

Thanks! Looks good. I updated the name to -rc1 in the case we need to do another one. Will make a post soon.

Is there anything else I can do to prepare us ahead for the next release ?

The PDF of the cmdstan guide is missing but that is it. We have some stanc3 fixes lined up already that will need to be swaped for the final release. But I guess that is the easy thing to change. If there are no severe bugs in Math/Stan/Cmdstan that will hopefully be everything.

[mitzimorris]

The PDF of the cmdstan guide is missing

cmdstan makefile task manual will generate pdf in dir doc

[mitzimorris]

checking the CmdStan guide now - found broken links in the doc - fix in time for release?

[mitzimorris]

fwiw, RC builds on both High Sierra and Catalina - but my environments are relatively cruft-free - using XCode's toolchain, i.e., clang++

[rok-cesnovar]

checking the CmdStan guide now - found broken links in the doc - fix in time for release?

Yes, please. Feel free to tag for review.

[mitzimorris]

@rok-cesnovar - on ubuntu I got this error message while running makefile task build:

/bin/sh: 1: [[: not found

found this on SO - seems relevant - https://stackoverflow.com/questions/12230690/string-comparison-in-bash-not-found/12230723

this is used in two of the makefile files:

• cmdstan/make/program

  Line 58 in 0ee3153

  @if ! [[ -f "$(@D)/tbb.dll" ]]; then \

• https://github.com/stan-dev/math/blob/ee3a6604d3f2b1acb9c609f698af3731c5086964/make/libraries#L126-L127

I think the logic that's triggering this message is only relevant for Windows - what do you think?

[serban-nicusor-toptal]

@mitzimorris That may be because make is by default using bin/sh which is not the actual /bin/bash it should be an easy fix following this post

[rok-cesnovar]

@serban-nicusor-toptal would you mind doing a release candidate 2?

We had to fix a few things and I think that would be best. So cmdstan develop + stanc3 nightly.

I would say we move the actual release to Wednesday or Thursday to not prolong the feature freeze too much?

[serban-nicusor-toptal]

Hey @rok-cesnovar you can find rc-2 here please let me know if everything is fine, thanks!
I'm fine with any release day as long as everything is done for the current release and everyone is okay with it.

[rok-cesnovar]

Cmdstan release notes:

• Updated docs for stanc compiler
• when random seed = -1, the actually used seed is written to the output and CSV file
• fixed make clean-all to cleanup all build files
• an easier way of building models with the previous compiler (stanc2) for debugging
• info on threading, MPI and OpenCL is now shown in the output and the CSV file
• downloading stanc3 binaries now retries on failure
• fixes for allowing execution of the downloaded stanc3 binary on Catalina
• updated Stan submodule to 2.23
• updated Stan Math submodule to 3.2

[mitzimorris]

comment in above release notes, each entry should be capitalized and have a period at the end (i.e., a sentence) - or all entries should be lowercase with no period (i.e., a phrase)

[rok-cesnovar]

I will fix that in a sec. If anyone volunteers for making a post once released, please let me know. Else I will make one in the evening CET time.

[mitzimorris]

happy to start work on a post - we should also put an announcement up on Andrew's blog

[serban-nicusor-toptal]

Hey, math 3.2.0 is done.
There were a few merge conflicts that I had to solve and it resulted in this change being done: stan-dev/math@5f28b8f

Posting here to be sure that won't break anything and is intended.

[rok-cesnovar]

happy to start work on a post - we should also put an announcement up on Andrew's blog

Thanks! Here are the relevant changes for stanc3 (there were quite a few bugfixes from RC1.

stan-dev/stanc3#498 (comment)

Hey, math 3.2.0 is done.
There was a few merge conflicts

Great! Yes. this is due to the hotfixes we had to do the last time. This looks fine.

[mitzimorris]

@serban-nicusor-toptal - are the docs getting updated automatically as part of the release now? the reduce_sum case study links to the docs at the end.

also, I think the 2_23 docs for the User's Guide will have a bunch of really great new content - checking now.

[serban-nicusor-toptal]

I didn't tie it to the release but it's a button press away, I will do it after the stan and cmstan release. Will let you know when done so you can review.

[serban-nicusor-toptal]

Update: The release for math, stan and cmdstan is done.

I am currently waiting on:

• stanc3 2.23.0 release, done through Jenkins
• docs 2.23 done through Jenkins

I am sorry that it takes more than usual but there is a VERY high load on AWS and our machines get terminated randomly because they're bought as on-demand ( we are using spot instances, which are instances not in use put for 80-90% less ).
I have updated out instances types pool and raised the quota from 3 machines to 6. This will let it choose from more machine types and hopefully speed up our release builds.

[serban-nicusor-toptal]

Hey @mitzimorris docs have been generated again here PR can you please take a look and confirm Jenkins did everything all right then merge? Thank you!

What Jenkins did can be found here

PS: Added manual to cmdstan release!

[mitzimorris]

looks great - merged it!

[serban-nicusor-toptal]

Everything works fine on the stan release thank you!

[mitzimorris]

checking 2_22 docs - don't see redirects to 2_23 - maybe it'll take a few minutes to refresh?

[serban-nicusor-toptal]

Checking the logs I don't see any error popping so everything should be working fine. Maybe it's cached somewhere. How can I debug this, on what page should I see a redirect ? I can check in the repository to see if it's there.

[mitzimorris]

https://mc-stan.org/docs/2_22/stan-users-guide/index.html

[mitzimorris]

https://mc-stan.org/docs/2_22/stan-users-guide/regression-models.html

[mitzimorris]

https://github.com/stan-dev/docs/blob/master/link_to_latest.py - this needs to be run on 2_22

but it's a destructive hack-and-a-half, so it shouldn't be run more than once.

[wds15]

On 2.22? Did you mean 2.23?

[serban-nicusor-toptal]

I understand you mean the This is an old version, view current version.
Which can be seen here: https://mc-stan.org/docs/2_20/stan-users-guide/index.html
I just found out it's not on 2.22 either https://mc-stan.org/docs/2_22/stan-users-guide/index.html

I now understand what is wrong, will fix it in a PR in a second.

[serban-nicusor-toptal]

Hey @mitzimorris please take a look at this PR it will fix the redirects.

What happened ? Totally my mistake sorry, forgot to make Jenkins read the Jenkinsfile from the docs repository and it was using a static outdated version. Also found a typo, everything should be fine now!

The job takes now an extra argument last_docs_version_dir which refers to the old version of docs, in this case 2_22 which will be passed to the add_links script. @wds15 that's why 2_22 and not 2_23.

Question for @mitzimorris we don't really need this folder generated here right ? 2_22_bak it's there just for safety and can be removed ? removed the directory

[mitzimorris]

looks great!

[serban-nicusor-toptal]

Checked above links plus some extra, everything looks fine now. I have updated scripts and Jenkinsfile so next time it will work flawlessly.

I think everything is done for the current release:

• math v3.2.0
• stan v2.23.0
• cmdstan v2.23.0
• stanc3 v2.23.0
• docs v2.23

If anyone has a few minutes it would help a lot to double-check, thank you!

[mitzimorris]

I updated 2 pages in the stan-dev.github.io repo to point to the 2.23 docs for Stan and CmdStan.
stan-dev/stan-dev.github.io@319f89d

don't know if this is worth automating.

[serban-nicusor-toptal]

I'll do it now, better to offload this kind of stuff to automation.

[rok-cesnovar]

I think everything looks great. Thanks @serban-nicusor-toptal!
Closing.