Add support for procedure annotations to the assembler

[bobbinth]

There are several use cases for adding support for procedure annotations in MASM. For example:

• Directives to the assembler on how to assembler a program.
  • One example of this is inline annotation which could inform the assembler that the annotated procedure should be inlined.
  • Another example is an annotation indicating whether a given procedure is expected to be call-ed, syscall-ed, or exec-ed.
• Information about procedure signature which could be useful for the compiler (and maybe debugger).
• User defined annotation. One example of these is the storage_offset annotation which we need in miden base to define storage offsets for account procedures.

I think @bitwalker had some ideas about how annotations could work. Let's discuss these in this issue.

[bitwalker]

In #1436, I proposed some syntax changes in Miden Assembly that I think would solve for our issues with kernel procedure visibilities, and defining/maintaining important procedure metadata like the type signature.

However, there are a number of other things, such as inlining for which annotations would be preferable to dedicated syntax:

• Inlining
• Interacting with external tooling, such as linter/formatter
• Decorators, such as storage offsets, automatically emitting debug events when a given procedure is called, etc.

The following is my proposed syntax for annotations:

# Grammar Rules
#
# ANNO := "@" "[" <META> "]" => <>
#
# META := <name:ID> => Annotation::from(name)
#       | <name:ID> "(" <tokens:Token> ")" => Annotation::from((name, tokens))
#       | <name:ID> "=" <value:META_EXPR> => Annotation::from((name, value))
#
#  META_EXPR := <name:ID> => Meta::Path(name.into())
#             | <string:String> => Meta::String(string)
#             | <int:u64> => Meta::Int(int)

# Following is an example of how this would be used to solve various things from our list:

# Inline this procedure if we're looking for inline opportunities
@[inline]
proc.add_unchecked
  ...
end

# Force this procedure to be inlined
@[inline(always)]
proc.intrinsic
   ...
end

# Mark this procedure as deprecated, and tell the linter to ignore any warnings generated by its body
@[allow(warnings::all)]
@[deprecated(since = "0.10.0")]
proc.deprecated
   ...
end

# Specify a storage offset
@[storage_offset = 1024]
proc.get_item
   ...
end

I think this syntax provides us with enough flexibility to support essentially any type of attribute we might feasibly need, and supports arbitrary extension for user-defined attributes.

I think we should require registering callbacks with the parser for user-defined attributes, dispatched based on the attribute name. Then, if we parse a module which contains unhandled attributes, we can raise a diagnostic. The user-provided callback would be given the opportunity to parse the attribute value, and return that value or a diagnostic if the value is invalid. This makes the system very extensible, while still keeping the API quite simple.

It would be important to store attributes in the procedure metadata of compiled libraries as well, so that this information can be used downstream. For attributes which only apply during parsing/compilation, they can be stripped, but some of these cannot: inline, deprecated at least, maybe storage_offset?. In any case, user-defined attributes would certainly need to be preserved, as we cannot know how they are meant to be used (or at the least, we provide a mechanism to indicate whether to preserve them or not as part of "registering" an attribute).

I think that about sums it up for now. I'm using "attribute" and "annotation" interchangeably here, but they are the same thing IMO.

[Fumuran]

Probably we also need to add an annotation for a dead code, which should be compiled anyway.

We came across this use case in the 0xPolygonMiden/miden-base#803 PR: almost all procedures in the kernel will be called only by their hash, so it makes sense to change them from export to proc, but in that case we cannot get the hashes of these procedures, since they were not compiled and were not added to the MastForest.

As @bobbinth mentioned in his comment, we should add an annotation for such procedures, which should be compiled even if they are considered a dead code.

[bobbinth]

The above is essentially for procedures which could be invoked dynamically but we don't want to export them from a give module. So, maybe "dead code" is not the right term here. Maybe we could use something like dynamic attribute like so:

@[dynamic]
proc.foo
    ...
end

The assembler would treat such a procedure as if it was an export procedure but it would not add it to the list of exports for the module.

Another option is to use a visibility modifier as discussed in #1436 (comment) and the related thread. Maybe something like:

proc(dynamic).foo
    ...
end