lang: Program composition

[armaniferrante]

One should be able to compose programs together similar to traits or mixins to combine the behavior of different programs.

For example, suppose you had the following two programs

#[program]
mod foo {
  fn initialize(Context<Initialize>) { ... }
}

#[program]
mod bar {
  fn update(Context<Update>) { ... }
}

Then one could combine these two programs into one with some syntax like

#[program(foo, bar)]
mod composed {
   fn delete(Context<Delete>) {...}
}

Where the program would have three instructions: initialize, update, and delete.

There are several important questions to answer:

1. How do we deal with conflict resolution for method identifiers when combining programs that may have duplicate instruction names?

• There are at least three potential options here. First, we could simply not allow it. Second, we could perform some type of implicit resolution, e.g., take the first instruction found from left to right in the mixin list. Third, we can change the sighash method identifier to take into consideration the mod name. So the sighash would be sha256(<mod-name>:<method-name>)[..8]. Then one could use instructions with duplicate identifiers as long as the mod names were distinct. I prefer the last option.

2. How do we combine idl type information?

• Since IDL information is parsed from the Anchor CLI, we can add an [idl] section to the Anchor.toml that tells the CLI the location of the IDLs of all the programs being composed together. The CLI will take that information and merge together all IDLs for each program being used into a single IDL. This "location" can be a program id, a filepath, or a git url. To start, we can simply use a filepath.

3. How do we generate the code to combine all the programs together?

• When executing method dispatch, if the identifier doesn't match any of the main program's methods, then we call the dispatch function for each of the mixin programs being composed until we get one that matches. If none match, then we can call the fallback function for the current program, if defined. Similarly, we'd have to answer how we execute fallbacks, say, if each mixin had a fallback defined themself. One solution would be to execute each fallback for each mixin if no matches are found.

4. What should the javascript api look like to interact with each of these composed programs

• We can add explicit namespaces for each of the programs. So to call update in the example above, we can do something like program.bar.rpc.update. We could also consider putting the methods on the main program object, e.g., program.rpc.update if there are no collisions and so the name is not ambiguous. (Tangent. Instead of putting the client namespaces, e.g., rpc before the instruction name, we should move them to the end, which feels more naturally, particularly in the context of composing programs. Though this is a separate issue.)

5. How do we generate CPI clients for the single program?

• We can re-export the cpi namespaces for each composed program under the program's generated cpi module. So to call update in the example above, we would have composed::cpi::bar::update (or maybe composed::cpi::update depending on if/how conflict resolution is done).

6. How do we generate rust clients for the single program?

• We can do the same as 5). I.e. just re-export the accounts and instructions generated modules.

Edit. A slight modification we might want to make here is to leverage the trait system. So we can wrap all the instructions here with a struct and impl block that implements a trait, similar to the current #[interface] mechanism.

[armaniferrante]

Another important consideration is how program composition should work in the context of statically defined program IDs as suggested in #645, which is useful so that all CPI accounts and clients can have program ownership validation checks--which would eliminate easy to miss footguns that are exposed to anyone invoking the program via CPI.

One way of accommodating this use case is by using meta macros when defining a library component, so that the statically defined program ID is available to all parts being composed. For example, we could define the initialize program above as

// Initialize crate.
contract! {
    #[program] 
    mod initialize {
        fn initialize(Context<Initialize>) { ... }
    }
    
    #[account]
    pub MyAccount {...}
}

// Update crate.
contract! {
    #[program]
    mod update {
        fn update(Context<Update>) { ... }
    }
    
    #[account]
    pub MyOtherAccount {...}
}

Which would generate a new import! macro that uses the output of declare_id in ownership validation checks. For example,

// These import lines can be hidden/called inside the program macro.
initialize::import!();
update::import!();

declare_id!("81ikgMpXEEjUH9pBWPvFM5uBKv4FQgYBa4bMpJryY3Sd");

#[program(initialize, update)]
mod composed {
  fn delete(Context<Delete>) { ... }
}

[juliotpaez]

If I can ask it, why is this even necessary?? If I'm not completely wrong, programs are not so long in terms of number of instructions, therefore doing something like the following does not imply too many more code or maintenance, even if you use many programs (like I do).

#[program]
mod foo {
  fn initialize(Context<Initialize>) { ... }
}

#[program]
mod bar {
  fn update(Context<Update>) { ... }
}

#[program]
mod composed {
   fn initialize(state: Context<foo::state::Initialize>) { foo::instructions::initialize(state) }
   fn update(state: Context<bar::state::Update>) { bar::instructions::update(state) }
   fn delete(Context<Delete>) {...}
}

With this simple approach you get rid of almost all your points, the only one that remains is (2) or how to make the Anchor CLI read imported modules to get the necessary info from them.