Some insider stuff on the Stan refactor

(This article was originally published at Statistical Modeling, Causal Inference, and Social Science, and syndicated at StatsBlogs.)

From the stan-dev list, Bob wrote [and has since added brms based on comments; the * packages are ones that aren’t developed or maintained by the stan-dev team, so we only know what we hear from their authors]:

The bigger picture is this, and you see the stan-dev/stan repo really spans three logical layers:

                      stan
          ----------------------------------
  math <- language <- algorithms <- services <- pystan
                                             <- rstan   <- rstanarm
                                                        <- rethinking (*)
                                                        <- brms (*)
                                             <- cmdstan <- statastan
                                                        <- matlabstan
                                                        <- stan.jl

What we are trying to do with the services refactor is make a clean services layer between the core interfaces (pystan, rstan, cmdstan) so that these don't have to know anything below the services layer. Ideally, there wouldn't be any calls from pystan, rstan, or cmdstan other than ones to the stan::services namespace. services, on the other hand, is almost certainly going to need to know about things below the algorithms level in language and math.

And Daniel followed up with:

This clarified a lot of things. I think this is what we should do:

1. Split algorithms and services into their own repos. (Language too, but that's a given.)
2. Each "route" to calling an algorithm should live in the "algorithms" repo. That is, algorithms should expose a simple function for calling it directly. It'll be a C++ API, but not ones that the interfaces use directly.
3. In "services," we'll have a config object with validation and only a handful of calls that pystan, rstan, cmdstan call. The config object needs to be simple and safe, but I think the pseudocode Bob and I created (which is looking really close to Michael's config object if it were safe) will suffice.

I don't really know what they're talking about but I thought it might be interesting to those of you who don’t usually see software development from the inside.

The post Some insider stuff on the Stan refactor appeared first on Statistical Modeling, Causal Inference, and Social Science.

Please comment on the article here: Statistical Modeling, Causal Inference, and Social Science

Tags: STAN, Statistical computing