Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implements flatten, make_list, meta #874

Merged
merged 6 commits into from
Dec 9, 2024
Merged

Implements flatten, make_list, meta #874

merged 6 commits into from
Dec 9, 2024

Conversation

zslayton
Copy link
Contributor

@zslayton zslayton commented Dec 9, 2024

Changes:

  • Modifies the system macro table to allow macros to easily depend on earlier macros
  • Adds an implementation of the flatten macro
  • Defines make_list as a template that leverages flatten
  • Re-defines make_sexp as a template that leverages flatten
  • Removes old bespoke impl of make_sexp
  • Defines meta as a template that leverages none

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

Copy link

codecov bot commented Dec 9, 2024

Codecov Report

Attention: Patch coverage is 88.69048% with 19 lines in your changes missing coverage. Please review.

Project coverage is 77.68%. Comparing base (9cf7f06) to head (fcd5368).
Report is 1 commits behind head on main.

Files with missing lines Patch % Lines
src/lazy/expanded/macro_evaluator.rs 87.64% 6 Missing and 5 partials ⚠️
src/lazy/expanded/macro_table.rs 95.23% 2 Missing and 1 partial ⚠️
src/lazy/binary/raw/v1_1/immutable_buffer.rs 0.00% 0 Missing and 1 partial ⚠️
src/lazy/encoder/text/v1_1/writer.rs 0.00% 0 Missing and 1 partial ⚠️
src/lazy/expanded/mod.rs 50.00% 1 Missing ⚠️
src/lazy/system_reader.rs 0.00% 0 Missing and 1 partial ⚠️
src/lazy/text/raw/v1_1/reader.rs 0.00% 0 Missing and 1 partial ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main     #874      +/-   ##
==========================================
+ Coverage   77.61%   77.68%   +0.06%     
==========================================
  Files         136      136              
  Lines       34141    34188      +47     
  Branches    34141    34188      +47     
==========================================
+ Hits        26498    26558      +60     
+ Misses       5666     5648      -18     
- Partials     1977     1982       +5     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@zslayton zslayton marked this pull request as ready for review December 9, 2024 18:29
Copy link
Contributor Author

@zslayton zslayton left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🗺️ PR Tour 🧭

@@ -438,7 +438,7 @@ mod benchmark {
let mut context = EncodingContext::for_ion_version(IonVersion::v1_1);
context
.macro_table_mut()
.add_macro(compiled_macro.clone())
.add_template_macro(compiled_macro.clone())
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🪧 This method (which is specific to template macros) was renamed to accommodate adding a more general purpose add_macro method alongside it.

Comment on lines -126 to +131
MacroKind::MakeSExp => MacroExpansionKind::MakeSExp(MakeSExpExpansion::new(arguments)),
MacroKind::Annotate => MacroExpansionKind::Annotate(AnnotateExpansion::new(arguments)),
MacroKind::Flatten => MacroExpansionKind::Flatten(FlattenExpansion::new(
self.context,
environment,
arguments,
)),
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🪧 Now that make_sexp is a trivial template that uses flatten, we no longer need a special MacroKind and custom impl for it. While this PR adds 3 new system macros, MacroKind's number of variants is unchanged.

Comment on lines -422 to +423
#[derive(Copy, Clone, Debug)]
#[derive(Debug)]
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🪧 flatten is the first system macro to be implemented which requires a macro evaluator to persist across calls to evaluator.next(). As a result, the MacroExpansion family of types can no longer implement Copy, as each one might own a bump-allocated evaluator of its own. The Copy impl was only being used directly in one speculatively defined function that had no callers. I'll point it out later in the diff.


// Calculate the next step in this macro expansion without advancing the expansion.
pub fn peek_next_step(&self) -> IonResult<MacroExpansionStep<'top, D>> {
let mut expansion_copy = *self;
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🪧 The aforementioned use of Copy, which would have enabled peek()ing at the next step of a macro. This is still possible without Copy, but will take a bit more code.

Comment on lines -721 to +715
match self.state {
match mem::take(&mut self.state) {
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🪧 An indirect use of Copy: because self.state isn't Copy any more, we need to set its value to the default (Empty) so we can update its state using the (owned) contents.

Comment on lines -191 to -197
pub const SYSTEM_MACRO_KINDS: &'static [MacroKind] = &[
MacroKind::None,
MacroKind::ExprGroup,
MacroKind::MakeString,
MacroKind::MakeSExp,
MacroKind::Annotate,
];
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🪧 This was a holdover from before we had a proper system macro table. It's no longer in use.

@@ -511,18 +528,21 @@ impl MacroTable {
Ok(id)
}

pub(crate) fn append_macro(&mut self, macro_ref: &Arc<Macro>) -> IonResult<()> {
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🪧 This method was extracted from the append_all_macros_from function below. It is now also be used in compile_system_macros().

Comment on lines -493 to +510
pub fn add_macro(&mut self, template: TemplateMacro) -> IonResult<usize> {
pub fn add_template_macro(&mut self, template: TemplateMacro) -> IonResult<usize> {
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🪧 As mentioned prior, this was renamed to accommodate the introduction of a more general append_macro method (see below).

Comment on lines -215 to -216
/// The SExp was produced by a call to `make_sexp`.
Constructed(Environment<'top, D>, MacroExprArgsIterator<'top, D>),
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Defining make_sexp in terms of flatten allowed me to delete lots of special-case code for make_sexp. 🎉

self.context
.macro_table
.add_template_macro(new_macro)
.unwrap();
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🪧 rustfmt change

@zslayton zslayton merged commit 8a0425f into main Dec 9, 2024
35 checks passed
@zslayton zslayton deleted the flatten-macro branch December 9, 2024 21:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants