CLI anchor idl command to use dump base64 for set-buffer and close #2486

ochaloup · 2023-05-11T09:03:00Z

A proposal for discussion.

We need to use IDL upgrade from anchor being called from multisig wallet of SPL Governance.
There is a way to pass a custom base64 instruction that is placed into proposal.
A way is to ask anchor idl cli to generate such instruction format instead of execution. As there could be called write-buffer -> set-authority it's all fine to have such functionality only for the set-buffer command.

I asked about it at discord (https://discord.com/channels/889577356681945098/889577399308656662/1105766024650956851) but haven't got any response if there is some way to do so.
This is a solution to get base64 printed. I will be glad for review and happy to hear about better solutions.

vercel · 2023-05-11T09:03:04Z

@ochaloup is attempting to deploy a commit to the coral-xyz Team on Vercel.

A member of the Team first needs to authorize it.

acheroncrypto

Thank you, this seems like a plausible solution. My question is would IDL commands other than set-buffer and close be useful for SPL Governance?

cli/src/base64serialize.rs

cli/src/lib.rs

ochaloup · 2023-05-15T07:02:13Z

Thank your @acheroncrypto for the review and your points.
I updated the code to address the review part.

About the other commands. This change was done from perspective where the idl authority is in use. When the authority is a multisig - it could not be just spl gov but it could be some other multisig system - then the multisig requires to sign.
From that I think the set-authority (https://github.com/coral-xyz/anchor/blob/master/lang/syn/src/codegen/program/idl.rs#L289) is one more command to be added into the scope.
(erase authority and write buffer can be done in two steps)

ochaloup · 2023-05-15T10:29:02Z

@acheroncrypto I discussed this PR with my colleague and he has got a really nice point that the flag should be only like --print-only without forcing to provide the current authority manually.
If you be so kind and check the PR with this new adjustment it would be great. Thank you.

acheroncrypto · 2023-05-15T13:29:14Z

@acheroncrypto I discussed this PR with my colleague and he has got a really nice point that the flag should be only like --print-only without forcing to provide the current authority manually. If you be so kind and check the PR with this new adjustment it would be great. Thank you.

I think this looks much better.

One issue is that we are completely recreating Instruction and AccountMeta structs. What would you say to using bincode encoding instead of borsh to get rid of this duplication?

ochaloup · 2023-05-16T07:41:51Z

One issue is that we are completely recreating Instruction and AccountMeta structs. What would you say to using bincode encoding instead of borsh to get rid of this duplication?

Thanks @acheroncrypto for the feedback. Sorry I missed the point about the bincode in the first post.
I'm fine to use the bincode serialization but the deserialization at spl gov or serum multisig work with borsh and I'm not sure how those two (bincode vs borsh) may work together.

I did some experiments
ochaloup@3db0202#diff-c1f8f7498da827a634bddc8a7559198bc99b296e9d9e8b91a70b503662995b8cR1975
and the results are different. It seems the borsh align the output to 4 bytes size while bicode somehow strip additional 0. The bincode output fails to be decoded by e.g., spl gov UI processing.

It's like

borsh

 DPRl9bEiiOHiF8/bqgrjrSFA8fetonnejseq+Zra59sDAAAAZNYA/eqUTTit8mSlVRWZCfA/KzcYPpqXMVZTCg5j64cAAQEUEwbrCwK1nWBRfcsHmsSPv3Hq9xKaNnH3AoQH4ngFAAFY1yGdsjj432ypEHVaegNWnDLmTi9zdrRFcIm8Q9RyBwEBCQAAAED0vHin6WkKAw==

[12,244,101,245,177,34,136,225,226,23,207,219,170,10,227,173,33,64,241,247,173,162,121,222,142,199,170,249,154,218,231,219,3,0,0,0,100,...

bincode

DPRl9bEiiOHiF8/bqgrjrSFA8fetonnejseq+Zra59sDZNYA/eqUTTit8mSlVRWZCfA/KzcYPpqXMVZTCg5j64cAAQEUEwbrCwK1nWBRfcsHmsSPv3Hq9xKaNnH3AoQH4ngFAAFY1yGdsjj432ypEHVaegNWnDLmTi9zdrRFcIm8Q9RyBwEBCUD0vHin6WkKAw==

[12,244,101,245,177,34,136,225,226,23,207,219,170,10,227,173,33,64,241,247,173,162,121,222,142,199,170,249,154,218,231,219,3,100,...

Would you know how to make the bincode to generate the serialized output aligned to borsh expectations?

acheroncrypto · 2023-05-16T09:12:14Z

Would you know how to make the bincode to generate the serialized output aligned to borsh expectations?

Yes, we can transform the bincode serialized data if SPL Governance only accepts borsh encoding. I can add this functionality if you'd like.

ochaloup · 2023-05-16T10:48:52Z

Yes, we can transform the bincode serialized data if SPL Governance only accepts borsh encoding. I can add this functionality if you'd like.

If you link me to an example in a repo or some doc I would be glad to learn.
If it's easier for you to add it then I will be happy if you can.

acheroncrypto · 2023-05-16T11:17:41Z

If you link me to an example in a repo or some doc I would be glad to learn.

As you've figured, bincode uses 8 bytes(LE) to store the length of the elements meanwhile borsh is using 4 bytes(LE). We can remove extra 4 bytes to make the instructions compatible with borsh.

If it's easier for you to add it then I will be happy if you can.

I've added it but seems like I don't have the rights to push to this branch, could you allow edits by maintainers?

…ty and close

ochaloup · 2023-05-16T15:49:40Z

@acheroncrypto awesome, it works nicely. I fixed merge conflicts, squashed my commits and placed your one on top.
It's ready for a final review, I assume.

cli/src/lib.rs

acheroncrypto · 2023-05-17T15:44:58Z

@acheroncrypto awesome, it works nicely. I fixed merge conflicts, squashed my commits and placed your one on top. It's ready for a final review, I assume.

Thanks, it looks great. Could you add a feature entry to the CHANGELOG for this PR?

ochaloup · 2023-05-18T10:27:30Z

Thanks, it looks great. Could you add a feature entry to the CHANGELOG for this PR?

I updated the changelog, hopefully in format similar to others.

cli/src/lib.rs

Co-authored-by: acheron <98934430+acheroncrypto@users.noreply.github.com>

commit e1afcbf Author: acheron <98934430+acheroncrypto@users.noreply.github.com> Date: Fri Jun 9 18:00:35 2023 +0200 v0.28.0 (coral-xyz#2527) commit c7c7319 Author: acheron <98934430+acheroncrypto@users.noreply.github.com> Date: Thu Jun 8 18:59:44 2023 +0200 Allow wider range of dependency versions to reduce dependency issues (coral-xyz#2524) commit 6df34e7 Author: acheron <98934430+acheroncrypto@users.noreply.github.com> Date: Wed Jun 7 19:12:56 2023 +0200 Update crate authors and remove outdated registry (coral-xyz#2522) commit 1705d16 Author: Jean Marchand (Exotic Markets) <jeanno11@orange.fr> Date: Wed Jun 7 16:29:23 2023 +0200 docs: Add doc for InitSpace macro (coral-xyz#2521) commit 3d7c97b Author: acheron <98934430+acheroncrypto@users.noreply.github.com> Date: Tue Jun 6 19:28:24 2023 +0200 cli: Accept program lib name for `anchor deploy --program-name` (coral-xyz#2519) commit a88be42 Author: Sergo <rogaldh@radsh.red> Date: Tue Jun 6 14:07:33 2023 +0300 ts: Validate `error.data` exists on simulation response (coral-xyz#2508) commit 65c9d6e Author: Jean Marchand (Exotic Markets) <jeanno11@orange.fr> Date: Tue Jun 6 09:43:46 2023 +0200 client: Add async to anchor-client (coral-xyz#2488) Co-authored-by: acheron <acheroncrypto@gmail.com> commit b8eda69 Author: Deep Mehta <65382963+0xdeepmehta@users.noreply.github.com> Date: Mon Jun 5 22:35:24 2023 +0530 cli: Print not found message if the given program cannot be found during deployment (coral-xyz#2517) commit 1902b8e Author: CanardMandarin <thibault.marboud@gmail.com> Date: Mon Jun 5 14:16:10 2023 +0200 cli: Update programs in `Anchor.toml` when using `anchor new` (coral-xyz#2516) commit 383e440 Author: acheron <98934430+acheroncrypto@users.noreply.github.com> Date: Sun Jun 4 21:02:16 2023 +0200 cli: Initialize with the correct program id (coral-xyz#2509) commit 835dc5b Author: Sarfaraz Nawaz <sir_nawaz959@yahoo.com> Date: Sun Jun 4 23:20:03 2023 +0530 lang: Rename derive_anchor_deserialize -> derive_init_space (coral-xyz#2510) commit 1c6f86e Author: acheron <98934430+acheroncrypto@users.noreply.github.com> Date: Sun Jun 4 13:09:39 2023 +0200 Upgrade Solana to 1.16.0 (coral-xyz#2512) commit 2bf8afe Author: acheron <98934430+acheroncrypto@users.noreply.github.com> Date: Tue May 30 19:50:45 2023 +0200 cli: Use `confirmed` commitment level in commands (coral-xyz#2506) commit 70d9223 Author: acheron <98934430+acheroncrypto@users.noreply.github.com> Date: Sun May 28 22:34:53 2023 +0200 cli: Add `anchor keys sync` command (coral-xyz#2505) commit 0c8498d Author: cavemanloverboy <93507302+cavemanloverboy@users.noreply.github.com> Date: Sat May 27 06:53:02 2023 -0700 cli: Exit `anchor clean` without error when dirs don't exist (coral-xyz#2504) commit 23b90bf Author: Noah Gundotra <ngundotra@users.noreply.github.com> Date: Fri May 26 12:36:46 2023 -0400 Feature: CPI Events API (coral-xyz#2438) Co-authored-by: acheron <acheroncrypto@gmail.com> commit c3625c8 Author: Last Emperor <46998219+lastemp@users.noreply.github.com> Date: Wed May 24 15:05:47 2023 +0300 examples: Add an example with `instruction` method (coral-xyz#2501) Co-authored-by: acheron <acheroncrypto@gmail.com> commit 67eb752 Author: acheron <98934430+acheroncrypto@users.noreply.github.com> Date: Sat May 20 20:34:38 2023 +0200 tests: Fix zero-copy tests (coral-xyz#2498) commit f9d0eca Author: acheron <98934430+acheroncrypto@users.noreply.github.com> Date: Fri May 19 13:18:14 2023 +0200 spl: Update `spl-token-2022` to 0.6.1 (coral-xyz#2496) commit 4793b90 Author: acheron <98934430+acheroncrypto@users.noreply.github.com> Date: Fri May 19 10:58:16 2023 +0200 Fix `toml_datetime` 1.64.0 MSRV error (coral-xyz#2495) commit 41a4d82 Author: chalda <chalda@seznam.cz> Date: Thu May 18 19:12:25 2023 +0200 cli: Add print base64 instruction option for some of the IDL commands (coral-xyz#2486) Co-authored-by: acheron <acheroncrypto@gmail.com> commit b7bada1 Author: Pierre <Arrowana@users.noreply.github.com> Date: Tue May 16 23:46:40 2023 +1000 fix: remove skip preflight from cli (coral-xyz#2492) --------- Co-authored-by: acheron <acheroncrypto@gmail.com> commit 89e94d1 Author: Ryan De La O <rdelao@users.noreply.github.com> Date: Sat May 13 02:17:47 2023 -0700 cli: Fix incorrect metadata.address generation (coral-xyz#2485) Currently when running 'anchor deploy --program-name <name> --program-keypair <specified keypair>' the cli still uses the auto-generated keypair when fetching the program id to add to the IDL metadata at the end. It should instead use the address from the specified keypair. --------- Co-authored-by: acheron <acheroncrypto@gmail.com> commit 714d524 Author: CanardMandarin <thibault.marboud@gmail.com> Date: Tue May 9 16:17:11 2023 +0200 lang: Add error message when Mint and TokenAccount with `init` are not ordered correctly (coral-xyz#2484) commit 9a93a2e Author: James <juicy66173@gmail.com> Date: Mon May 8 10:17:51 2023 +0100 ts: Improve IDL typing (coral-xyz#2482) * Use XOR pattern for enum variants to prevent two variants being used at the same time. * Fix unknown for types like Option<[u8; 32]> commit d1ddf00 Author: CanardMandarin <thibault.marboud@gmail.com> Date: Sun May 7 11:03:37 2023 +0200 lang: Fix incorrectly checking the first init constraint (coral-xyz#2483)

…coral-xyz#2486) Co-authored-by: acheron <acheroncrypto@gmail.com>

acheroncrypto reviewed May 14, 2023

View reviewed changes

cli/src/base64serialize.rs Outdated Show resolved Hide resolved

cli/src/lib.rs Outdated Show resolved Hide resolved

cli/src/lib.rs Outdated Show resolved Hide resolved

cli/src/lib.rs Outdated Show resolved Hide resolved

acheroncrypto added cli idl related to the IDL, either program or client side labels May 14, 2023

ochaloup force-pushed the play-with-anchor-cli branch from 1860a96 to 375f21b Compare May 14, 2023 21:28

ochaloup and others added 2 commits May 16, 2023 17:41

CLI anchor idl command to use dump base64 for set-buffer, set-authori…

ff4d9f9

…ty and close

Remove Instruction duplication and add a common print function

b99bece

ochaloup force-pushed the play-with-anchor-cli branch from 43aca39 to b99bece Compare May 16, 2023 15:44

acheroncrypto reviewed May 17, 2023

View reviewed changes

cli/src/lib.rs Show resolved Hide resolved

Updating CHANGELOG.md with info on idl --print-only argument

2b2ec07

acheroncrypto reviewed May 18, 2023

View reviewed changes

cli/src/lib.rs Outdated Show resolved Hide resolved

Update cli/src/lib.rs: clippy fix

7f8d9f5

Co-authored-by: acheron <98934430+acheroncrypto@users.noreply.github.com>

acheroncrypto approved these changes May 18, 2023

View reviewed changes

acheroncrypto merged commit 41a4d82 into coral-xyz:master May 18, 2023

vadorovsky pushed a commit to Lightprotocol/anchor that referenced this pull request Jun 20, 2023

cli: Add print base64 instruction option for some of the IDL commands (…

e401c25

…coral-xyz#2486) Co-authored-by: acheron <acheroncrypto@gmail.com>

ananas-block pushed a commit to Lightprotocol/anchor that referenced this pull request Jun 20, 2023

cli: Add print base64 instruction option for some of the IDL commands (…

ba018c3

…coral-xyz#2486) Co-authored-by: acheron <acheroncrypto@gmail.com>

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

CLI anchor idl command to use dump base64 for set-buffer and close #2486

CLI anchor idl command to use dump base64 for set-buffer and close #2486

ochaloup commented May 11, 2023

vercel bot commented May 11, 2023

acheroncrypto left a comment

ochaloup commented May 15, 2023

ochaloup commented May 15, 2023

acheroncrypto commented May 15, 2023

ochaloup commented May 16, 2023 •

edited

Loading

acheroncrypto commented May 16, 2023

ochaloup commented May 16, 2023

acheroncrypto commented May 16, 2023 •

edited

Loading

ochaloup commented May 16, 2023

acheroncrypto commented May 17, 2023

ochaloup commented May 18, 2023

CLI anchor idl command to use dump base64 for set-buffer and close #2486

CLI anchor idl command to use dump base64 for set-buffer and close #2486

Conversation

ochaloup commented May 11, 2023

vercel bot commented May 11, 2023

acheroncrypto left a comment

Choose a reason for hiding this comment

ochaloup commented May 15, 2023

ochaloup commented May 15, 2023

acheroncrypto commented May 15, 2023

ochaloup commented May 16, 2023 • edited Loading

acheroncrypto commented May 16, 2023

ochaloup commented May 16, 2023

acheroncrypto commented May 16, 2023 • edited Loading

ochaloup commented May 16, 2023

acheroncrypto commented May 17, 2023

ochaloup commented May 18, 2023

ochaloup commented May 16, 2023 •

edited

Loading

acheroncrypto commented May 16, 2023 •

edited

Loading