[AutoDiff] [CC] Add ti.extension.adstack support to C backend

[archibate]

Concisely describe the proposed feature
I'd like to add the support of adstack to the C backend.
Some of the advanced usage of autodiff seems depends on this feature very much, e.g., mass_spring.py in difftaichi, as @samuela reported in #1644 (comment).

Describe the solution you'd like (if any)
I'm not super clear about the autodiff system, so I feel #1257 hard for me to understand. @k-ye Would you mind provide me with more information?

• StackPopStmt
• StackPushStmt
• StackLoadTopStmt
• StackLoadTopAdjStmt
• StackAccAdjointStmt

What does adjoint and primal mean and their difference? It seems we have two 32-bit float values per-stack push? Is Adj a shortcut for Adjoint, or simply adjust stack?

Additional comments
@samuela's idea is to call Taichi kernels directly from Julia, that's why we need C backend instead of the LLVM backend, which doesn't have an export option for now.

[k-ye]

What does adjoint and primal mean and their difference?

adjoint just stores the gradient of the corresponding variable. Maybe this is a good read: https://rufflewind.com/2016-12-30/reverse-mode-automatic-differentiation ?

It seems we have two 32-bit float values per-stack push?

Yeah. One to store the variable itself (primal), the other to store its gradient (adjoint)

Is Adj a shortcut for Adjoint, or simply adjust stack?

It stands for Adjoint. IIRC, stack adjustments are explicitly named push and pop in this case.

TBH I only have a rough idea of Taichi's autodiff system, which was enough to implement AdStack. As you've pointed out, AdStack itself is just a list of float pairs. So long as we implement those IRs correctly, I think the AD system will just work for the backend.