[Feature] Merge priorities

doc/manual/merging.md

@@ -254,16 +254,64 @@ final record:
 ## Merging record with metadata
 
 Metadata can be attached to values thanks to the `|` operator. Metadata
-currently includes contract annotations, default value, and documentation. We
-describe in this section how metadata interacts with merging.
+currently includes contract annotations, default value, merge priority, and
+documentation. We describe in this section how metadata interacts with merging.
 
-### Default values
+### Merge priorities
+
+Priorities are specified using the `priority` annotation, followed by a number
+literal. There are also two other special priorities, the bottom priority, specified
+using the `default` annotation, and the top priority, specified using the
+`force` annotation.
+
+Priorities dictate which values take precedence over other values. By default,
+values are given the priority `0`. Values with the same priority are recursively
+merged as specified in this document, which can mean failure if the values can't
+be meaningfully merged:
+
+```text
+nickel> {foo = 1} & {foo = 2}
+error: non mergeable terms
+  ┌─ repl-input-1:1:8
+  │
+1 │ {foo = 1} & {foo = 2}
+  │        ^           ^ with this expression
+  │        │
+  │        cannot merge this expression
+  │
+  = Both values have the same merge priority but they can't be combined
+```
+
+On the other hand, if the priorities differ, the value with highest priority
+simply erases the other in the final result:
+
+```text
+nickel> {foo | priority 1 = 1} & {foo = 2}
+{ foo = 1 }
+
+nickel> {foo | priority -1 = 1} & {foo = 2}
+{ foo = 2 }
+```
+
+The priorities are ordered in the following way:
+
+- bottom is the lowest priority
+- numeral priorities are ordered as usual numbers (priorities can be any valid Nickel
+  number, including fractions and negative values)
+- top is the highest priority
+
+#### Default values
 
 A `default` annotation can be used to provide a base value, but let it be
 overridable through merging. For example, `{foo | default = 1} & {foo = 2}`
-evaluates to `{foo = 2}`. Without the default value, this merge would have
-failed with a `non mergeable fields` error, because merging being symmetric, it
-doesn't know how to combine `1` and `2` in a generic and meaningful way.
+evaluates to `{foo = 2}`. A default value is just a special case of a priority,
+being the lowest possible one.
+
+#### Forcing values
+
+Dually, values with the `force` annotation are given the highest priority. Such
+a value can never be overriden, and will either take precedence over another
+value or be tentatively merged if the other value is forcing as well.
 
 #### Specification
 
@@ -284,10 +332,6 @@ same on both side:
 }
 ```
 
-Currently, there are only two priorities, `normal` (by default, when nothing is
-specified) and the `default` one, with `default < normal`. We plan to add more
-in the future (see [RFC001](https://github.com/tweag/nickel/blob/c21cf280dc610821fceed4c2caafedb60ce7177c/rfcs/001-overriding.md#priorities)).
-
 #### Example
 
 Let us stick to our firewall example. Thanks to default values, we set the most

doc/manual/syntax.md

@@ -12,7 +12,7 @@ alphanumeric characters, `_` (underscores) or `'` (single quotes). For example,
 
 ## Simple values
 
-There are four basic kind of values in Nickel :
+There are four basic kinds of values in Nickel :
 
 1. numeric values
 2. boolean values
@@ -532,9 +532,14 @@ Adding documentation can be done with `| doc < string >`. Examples:
 true
 ```
 
-Record contracts can set default values using the `default` metadata: It is
-noted as `| default = < default value >`. This is especially useful when merging
-records (more about this in the dedicated document about merge).
+Metadata can also set merge priorities using the following annotations:
+
+- `default` gives the lowest priority (default values)
+- `priority NN`, where `NN` is a number literal, gives a numeral priority
+- `force` gives the highest priority
+
+If there is no priority specified, `priority 0` is given by default. See more
+about this in the dedicated section on merging.
 
 Examples:
 
@@ -552,4 +557,13 @@ Examples:
 
 > {foo | default = 1, bar = foo + 1} & {foo = 2}
 { foo = 2, bar = 3 }
+
+> {foo | force = 1, bar = foo + 1} & {foo = 2}
+{ bar = 2, foo = 1 }
+
+> {foo | priority 10 = 1} & {foo | priority 8 = 2} & {foo = 3}
+{ foo = 1 }
+
+> {foo | priority -1 = 1} & {foo = 2}
+{ foo = 2 }
 ```

examples/merge-priorities/README.md

@@ -0,0 +1,12 @@
+# Merge priorities
+
+This example is a variant of the merge example that makes use of merge
+priorities. The code to run lies in `main.ncl`. The default value
+`firewall.enabled` defined in `security.ncl` is overwritten in the final
+configuration.
+
+## Run
+
+```console
+nickel -f main.ncl export
+```

examples/merge-priorities/main.ncl

@@ -0,0 +1,20 @@
+# Merge several blocks into one final configuration. In a real world case, one
+# would also want contracts to validate the shape of the data.
+let server = import "server.ncl" in
+let security = import "security.ncl" in
+# Disabling firewall in the final result
+server & security & {
+  # As opposed to the simple merge example, this would now fail
+  # firewall.enabled = false
+
+  firewall.open_ports | priority 10 = [80],
+  firewall.type = "superiptables",
+  server.host.ip = "89.22.11.01",
+
+  # this will only be selected if no values with higher priority (or no priority
+  # annotation, which is the same as priority 0) is ever defined
+  # because there's a definite value below, this won't be selected.
+  server.host.name | priority -1 = "hello-world.backup.com",
+} & {
+  server.host.name = "hello-world.main.com",
+}

examples/merge-priorities/security.ncl

@@ -0,0 +1,8 @@
+{
+  server.host.options | priority 10 = "TLS",
+
+  # force make it impossible to override this value with false
+  firewall.enabled | force = true,
+  firewall.type | default = "iptables",
+  firewall.open_ports | priority 5 = [21, 80, 443],
+}

examples/merge-priorities/server.ncl

@@ -0,0 +1,5 @@
+{
+  server.host.ip | default = "182.168.1.1",
+  server.host.port | default = 80,
+  server.host.name | default = "hello-world.net",
+}

src/error.rs

@@ -1116,7 +1116,10 @@ impl ToDiagnostic<FileId> for EvalError {
 
                 vec![Diagnostic::error()
                     .with_message("non mergeable terms")
-                    .with_labels(labels)]
+                    .with_labels(labels)
+                    .with_notes(vec![String::from(
+                        "Both values have the same merge priority but they can't be combined",
+                    )])]
             }
             EvalError::UnboundIdentifier(ident, span_opt) => vec![Diagnostic::error()
                 .with_message("unbound identifier")

src/eval/tests.rs

@@ -102,7 +102,7 @@ fn mk_default(t: RichTerm) -> Term {
     use crate::term::MergePriority;
 
     let mut meta = MetaValue::from(t);
-    meta.priority = MergePriority::Default;
+    meta.priority = MergePriority::Bottom;
     Term::MetaValue(meta)
 }
 

src/parser/grammar.lalrpop

@@ -60,7 +60,7 @@ use crate::{
     term::{
         BinaryOp, RichTerm, Term, UnaryOp, StrChunk, MetaValue,
         MergePriority, Contract, NAryOp, RecordAttrs, SharedTerm,
-        make as mk_term},
+        NumeralPriority, make as mk_term},
     types::{Types, AbsType},
     position::TermPos,
     label::Label,
@@ -101,7 +101,16 @@ AnnotAtom<TypeRule>: MetaValue = {
         ..Default::default()
     },
     "|" "default" => MetaValue {
-        priority: MergePriority::Default,
+        priority: MergePriority::Bottom,
+        ..Default::default()
+    },
+    "|" "force" => MetaValue {
+        priority: MergePriority::Top,
+        ..Default::default()
+    },
+    "|" "priority" <SignedNumLiteral> => MetaValue {
+        // unwrap(): a literal can't be NaN
+        priority: MergePriority::Numeral(NumeralPriority::try_from(<>).unwrap()),
         ..Default::default()
     },
     "|" "doc" <s: StaticString> => MetaValue {
@@ -427,7 +436,7 @@ Match: Match = {
 
 // A default annotation in a pattern.
 DefaultAnnot: MetaValue = "?" <t: Term> => MetaValue {
-    priority: MergePriority::Default,
+    priority: MergePriority::Bottom,
     value: Some(t),
     ..Default::default()
 };
@@ -532,7 +541,7 @@ UOp: UnaryOp = {
     "record_map" => UnaryOp::RecordMap(),
     "seq" => UnaryOp::Seq(),
     "deep_seq" => UnaryOp::DeepSeq(None),
-    "force" => UnaryOp::Force(None),
+    "op force" => UnaryOp::Force(None),
     "head" => UnaryOp::ArrayHead(),
     "tail" => UnaryOp::ArrayTail(),
     "length" => UnaryOp::ArrayLength(),
@@ -778,6 +787,14 @@ TypeAtom: Types = {
     },
 }
 
+SignedNumLiteral: f64 = <sign: "-"?> <value: "num literal"> => {
+    if sign.is_some() {
+        -value
+    } else {
+        value
+    }
+};
+
 extern {
     type Location = usize;
     type Error = ParseError;
@@ -849,7 +866,7 @@ extern {
         "typeof" => Token::Normal(NormalToken::Typeof),
         "assume" => Token::Normal(NormalToken::Assume),
         "array_lazy_assume" => Token::Normal(NormalToken::ArrayLazyAssume),
-        "force" => Token::Normal(NormalToken::Force),
+        "op force" => Token::Normal(NormalToken::OpForce),
         "blame" => Token::Normal(NormalToken::Blame),
         "chng_pol" => Token::Normal(NormalToken::ChangePol),
         "polarity" => Token::Normal(NormalToken::Polarity),
@@ -878,8 +895,10 @@ extern {
         "elem_at" => Token::Normal(NormalToken::ElemAt),
         "merge" => Token::Normal(NormalToken::Merge),
         "default" => Token::Normal(NormalToken::Default),
+        "force" => Token::Normal(NormalToken::Force),
         "doc" => Token::Normal(NormalToken::Doc),
         "optional" => Token::Normal(NormalToken::Optional),
+        "priority" => Token::Normal(NormalToken::Priority),
 
         "hash" => Token::Normal(NormalToken::OpHash),
         "serialize" => Token::Normal(NormalToken::Serialize),

src/parser/lexer.rs

@@ -186,7 +186,7 @@ pub enum NormalToken<'input> {
     #[token("%deep_seq%")]
     DeepSeq,
     #[token("%force%")]
-    Force,
+    OpForce,
     #[token("%head%")]
     Head,
     #[token("%tail%")]
@@ -216,6 +216,10 @@ pub enum NormalToken<'input> {
     Doc,
     #[token("optional")]
     Optional,
+    #[token("priority")]
+    Priority,
+    #[token("force")]
+    Force,
 
     #[token("%hash%")]
     OpHash,

src/parser/uniterm.rs

@@ -211,7 +211,7 @@ impl UniRecord {
                                         types: Some(ctrt),
                                         contracts,
                                         opt: false,
-                                        priority: MergePriority::Normal,
+                                        priority: MergePriority::Neutral,
                                         value: None,
                                     }) if contracts.is_empty() => Ok(Types(AbsType::RowExtend(
                                         id,

src/pretty.rs

@@ -99,7 +99,7 @@ where
             }),
             self.line().clone(),
         ))
-        .append(if mv.priority == crate::term::MergePriority::Default {
+        .append(if mv.priority == crate::term::MergePriority::Bottom {
             self.line().append(self.text("| default"))
         } else {
             self.nil()
@@ -207,7 +207,7 @@ where
                                 MetaValue {
                                     types,
                                     contracts,
-                                    priority: crate::term::MergePriority::Default,
+                                    priority: crate::term::MergePriority::Bottom,
                                     value: Some(value),
                                     ..
                                 } => allocator

src/repl/query_print.rs

@@ -235,18 +235,19 @@ fn write_query_result_<R: QueryPrinter>(
 
             match &meta {
                 MetaValue {
-                    priority: MergePriority::Default,
+                    priority: MergePriority::Bottom,
                     value: Some(t),
                     ..
                 } if selected_attrs.default => {
                     renderer.write_metadata(out, "default", &t.as_ref().shallow_repr())?;
                     found = true;
                 }
                 MetaValue {
-                    priority: MergePriority::Normal,
+                    priority: MergePriority::Numeral(n),
                     value: Some(t),
                     ..
                 } if selected_attrs.value => {
+                    renderer.write_metadata(out, "priority", &format!("{}", n))?;
                     renderer.write_metadata(out, "value", &t.as_ref().shallow_repr())?;
                     found = true;
                 }