Can you enforce linear constraints when sampling with liesel.goose?

[jeli2103]

Dear Liesels,
I am interested in setting up P-Splines in Liesel. There are different options of how to deal with the identifiability issue, e.g. you can directly impose a centering constraint to the design matrix (works fine with Liesel) or sampling the coefficients (c) under the linear constraint 1'c = 0. Is it possible in Liesel to sample under linear constraints?

[jobrachem]

Yes, it is possible. You can, for example, use the approach detailed in Wood (2017), Chapter 1.8.1 to reparameterize your coefficients and your penalty matrix. That way you can enforce general linear constraints.

Wood, S. N. (2017). Generalized Additive Models (2nd ed.). Chapman & Hall/CRC.

[jeli2103]

This is the general approach, which requires some more work as the user need to reparameterize the variables (in particular in my case for some blockdiagonal structures). Additionally, in some settings reparametization might not be wanted. In MCMC one can optimize the parameters under the linear constraint sum(coefficients) = 0, not requiring any additional reparameterization. However, in my case the approach you suggested works as well. I will mark it as answered, but it would be nice to see some linear constraint implementation in liesel.goose.

[Seb-Lorek]

Hey @jobrachem I think the question is asking specifically for the approach of including the constraint during the sampling process. As far as I know this is not possible in liesel. See Lang et al. 2004, p. 189 for the theory, it involves centering all smooth coefficient blocks (etc.) during each sampling iteration and adding the means to the intercept. At least from my perspective this introduces some unnecessary overhead into the inference algorithm, which can be avoided by using the approach @jobrachem is referencing to.

[jobrachem]

Thanks for the added context @Seb-Lorek. I agree that the Lang et al. approach introduces some overhead that can be avoided by using the reparameterization. I think what the Lang et al. approach sometimes allows for is to make use of sparseness to increase the efficiency of computations - but I am not familiar with any details on that.

In any case, my only objection is to the phrasing that "this is not possible in liesel". As far as I can see it, you just need to define appropriate lsl.Calc objects that implement the functions you need.

[Seb-Lorek]

I see so it should rather be: While liesel.goose does not implement a specific procedure to center any smooth effects automatically, one can always define a graph in liesel.model that allows for sampling under linear constraints. Thanks for the clarification!

[jeli2103]

I see that this would be another option as well. But as you say, it might not be the most efficient one.
Nothing is impossible - liesel.