DDB Mapper filter expressions (runtime components) #1401

ianbotsf · 2024-09-09T22:15:45Z

Issue #

Related to #76

Description of changes

This change adds low-level filter expressions support to the runtime, although nothing uses them yet. The filter DSL is defined in the Filter interface and will eventually (in the next PR) be part of a Lambda signature in a codegenned member for operation requests.

Something like this:

// Generated code
interface QueryRequest<T> {
    // Property syntax
    val filter: Filter.() -> BooleanExpr
    
    // DSL method syntax
    fun filter(block: Filter.() -> BooleanExpr)

    // …all other Query members
}

Callers will then use this when building their query structures to specify their filter expression (if desired):

// User code
table.query {
    filter {
        and(
            attr("productName") eq "Nimbus 2000",
            attr("condition") isIn setOf("New", "Refurbished"),
            attr("price") lt 500.00,
        )
    }

    // …set other Query members
}

In addition to the normal things, I could especially use feedback on the DSL—names, conventions, types, etc. If it's incomprehensible to you then it'll probably be even worse for users!

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

github-actions · 2024-09-09T22:24:57Z

A new generated diff is ready to view.

No codegen difference in the AWS SDK

ianbotsf

Captured some notes from a live code review with the team today. More comments/reviews to follow.

ianbotsf · 2024-09-11T16:47:57Z

...db-mapper/dynamodb-mapper/common/src/aws/sdk/kotlin/hll/dynamodbmapper/expressions/Filter.kt

+    /**
+     * Creates an equality expression for verifying two expressions are equal to each other
+     * @param value The other value in the comparison
+     */
+    public infix fun Expression.eq(value: Boolean?): BooleanExpr = eq(LiteralExpr(value))


live review: Can we collapse KDocs for multiple methods into a single shared instance?

Turns out we can but it only affects Dokka-generated documentation. IntelliJ will only find KDocs directly attached to each overload. I'm going to leave the documentation on overloads for the time being.

ianbotsf · 2024-09-11T16:57:39Z

...db-mapper/dynamodb-mapper/common/src/aws/sdk/kotlin/hll/dynamodbmapper/expressions/Filter.kt

+    /**
+     * Creates a range expression for verifying this expression is in the given range
+     * @param range The range to check
+     */
+    @Suppress("INAPPLICABLE_JVM_NAME")
+    @JvmName("isInRangeNumber")
+    public infix fun <N> AttributePath.isIn(range: ClosedRange<N>): BooleanExpr where N : Number, N : Comparable<N> =
+        isBetween(LiteralExpr(range.start), LiteralExpr(range.endInclusive))


live review: Should we support open ranges now or in the future? Investigate and/or leave a TODO

ianbotsf · 2024-09-11T17:01:35Z

...db-mapper/dynamodb-mapper/common/src/aws/sdk/kotlin/hll/dynamodbmapper/expressions/Filter.kt

+ * Several additional filter expressions are possible via the following methods/properties:
+ *
+ * * [contains] — checks if a string/list attribute value contains the given value
+ * * [exists] — checks if _any value_ (including `null`) exists for an attribute
+ * * [notExists] — checks if no value is present for an attribute (i.e., the attribute is "undefined" for an item)
+ * * [isOfType] — checks if an attribute value is of the given type
+ * * [size] — gets the size of an attribute (e.g., number of elements in list/map/set, the length of a string, etc.)
+ * * [startsWith] — checks if a string attribute value starts with the given value


live review: Here and elsewhere when referring to Kotlin-ized names of DDB primitives, mention the underlying primitive to help users get up to speed, transition from another mapper, etc.

Also add a few examples here.

ianbotsf · 2024-09-11T17:18:01Z

settings.gradle.kts

-    include(":hll:dynamodb-mapper:tests:dynamodb-mapper-schema-generator-plugin-test")
+    // include(":hll:dynamodb-mapper:tests:dynamodb-mapper-schema-generator-plugin-test")


live review: accidental inclusion, remove

ianbotsf · 2024-09-11T17:22:32Z

...pper/dynamodb-mapper/common/src/aws/sdk/kotlin/hll/dynamodbmapper/expressions/Expressions.kt

+/**
+ * Represents any kind of expression. This is an abstract top-level interface and describes no details about an
+ * expression on its own. Expressions may be of various specific types (e.g., [AttributePath], [ComparisonExpr],
+ * [AndExpr], etc.) each of which have specific data detailing the expression.
+ *
+ * [Expression] and its derivatives support the [visitor design pattern](https://en.wikipedia.org/wiki/Visitor_pattern)
+ * by way of an [accept] method.
+ */
+public sealed interface Expression {
+    /**
+     * Accepts a visitor that is traversing an expression tree by dispatching to a subtype implementation. Subtype
+     * implementations MUST call the [ExpressionVisitor.visit] overload for their concrete type (effectively forming a
+     * [double dispatch](https://en.wikipedia.org/wiki/Double_dispatch)) and MUST return the resulting value.
+     * @param visitor The [ExpressionVisitor] that is traversing an expression
+     */
+    public fun <T> accept(visitor: ExpressionVisitor<T>): T
+}


live review: split out hierarchy into different smaller files

lauzadis

No strong comments / suggestions outside of what we covered in the live review!

lauzadis · 2024-09-10T13:07:44Z

...er/dynamodb-mapper/common/src/aws/sdk/kotlin/hll/dynamodbmapper/expressions/AttributeType.kt

+    String("S"),
+
+    /**
+     * String set data type, denoted in DynamoDB as `S`


lauzadis · 2024-09-10T13:17:35Z

...db-mapper/dynamodb-mapper/common/src/aws/sdk/kotlin/hll/dynamodbmapper/expressions/Filter.kt

+ * * The value of attribute `bar` is less than the value of `baz` **–and–** the value of `baz` is greater/equal to `42`
+ * * The value of attribute `qux` is not one of `"ready"`, `"steady"`, or `"go"`
+ *
+ * This is roughly equivalent to the Kotlin syntax:


roughly equivalent -> logically equivalent?

lauzadis · 2024-09-11T18:13:02Z

...apper/dynamodb-mapper/common/src/aws/sdk/kotlin/hll/dynamodbmapper/expressions/Comparator.kt

+    GREATER_THAN(">"),
+
+    /**
+     * A greater-than-or-equal-to comparison, equivalent to `>=` in Kotlin


nit: "in Kotlin and DynamoDB"?

lauzadis · 2024-09-11T18:26:40Z

...-mapper/common/src/aws/sdk/kotlin/hll/dynamodbmapper/expressions/internal/ExpressionImpls.kt

opinion: I feel like the implementations don't need to repeat the KDocs of what they are implementing

Can you clarify what you mean? This ScalarFunc enum doesn't implement a particular interface...it identifies a particular low-level function.

This comment is referring to the entire ExpressionImpls.kt file

Oh jeez, I can't read. 🤦‍♂️

Yeah, I can remove the KDocs for the internal implementations. Still calibrating when it's helpful to leave KDocs on non-public stuff vs when it's just noise.

Yeah, I was thinking in this case they are repeated on the public interfaces, so the internal implementations don't need them

…heir own files, revert accidental settings.gradle.kts change, clean up various KDocs

sonarqubecloud · 2024-09-12T22:25:35Z

Quality Gate passed

Issues
12 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
2.7% Duplication on New Code

See analysis details on SonarCloud

github-actions · 2024-09-12T22:33:47Z

A new generated diff is ready to view.

No codegen difference in the AWS SDK

…lin (#1451) * initial poc commit of DynamoDB Mapper (#1232) * add support for Mapper initialization (#1237) * implement mapper pipeline (#1266) * initial implementation of codegen for low-level operations/types (#1357) * initial implementation of secondary index support (#1375) * Create new codegen module and refactor annotation processor to use it (#1382) * feat: add Schema generator Gradle plugin (#1385) * Fix plugin test package * add attribute converters for "standard" values (#1381) * fix: schema generator plugin test module (#1394) * feat: annotation processor codegen configuration (#1392) * feat: add `@DynamoDbIgnore` annotation (#1402) * DDB Mapper filter expressions (runtime components) (#1401) * feat: basic annotation processing (#1399) * add DSL overloads, paginators, and better builder integration for DDB Mapper ops codegen (#1409) * chore: split dynamodb-mapper-codegen into two modules (#1414) * emit DDB_MAPPER business metric (#1426) * feat: setup DynamoDbMapper publication (#1419) * DDB Mapper filter expressions (codegen components) (#1424) * correct docs * mark every HLL/DDBM API experimental (#1428) * fix accidental inclusion of expression attribute members in high-level DynamoDB Mapper requests (#1432) * Upgrade to latest build plugin version * fix: various issues found during testing (#1450) * chore: update Athena changelog notes for 1.3.57 (2024-10-18) release (#1449) * feat: update AWS API models * feat: update AWS service endpoints metadata * chore: release 1.3.60 * chore: bump snapshot version to 1.3.61-SNAPSHOT * feat: initial release of Developer Preview of DynamoDB Mapper for Kotlin * Fix Kotlin gradle-plugin version * fix: ddb mapper tests (#1453) * Bump build plugin version --------- Co-authored-by: Matas <lauzmata@amazon.com> Co-authored-by: aws-sdk-kotlin-ci <aws-kotlin-sdk-automation@amazon.com>

ianbotsf added 3 commits September 9, 2024 21:58

runtime component of DDB Mapper filter expressions

1dd4c86

(remove accidentally-included file)

6e70b84

(remove debugging build plugin)

4668fd0

ianbotsf requested a review from a team as a code owner September 9, 2024 22:15

ianbotsf commented Sep 11, 2024

View reviewed changes

lauzadis approved these changes Sep 11, 2024

View reviewed changes

addressing PR feedback: refactor subinterfaces/implementations into t…

aa20947

…heir own files, revert accidental settings.gradle.kts change, clean up various KDocs

0marperez approved these changes Sep 13, 2024

View reviewed changes

lauzadis approved these changes Sep 13, 2024

View reviewed changes

ianbotsf merged commit 4d71a04 into feat-ddb-mapper Sep 17, 2024
11 checks passed

ianbotsf deleted the ddbmapper-filter-expressions-runtime branch September 17, 2024 19:03

ianbotsf mentioned this pull request Sep 29, 2024

DDB Mapper filter expressions (codegen components) #1424

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

DDB Mapper filter expressions (runtime components) #1401

DDB Mapper filter expressions (runtime components) #1401

ianbotsf commented Sep 9, 2024

github-actions bot commented Sep 9, 2024

ianbotsf left a comment

ianbotsf Sep 11, 2024

ianbotsf Sep 12, 2024

ianbotsf Sep 11, 2024

ianbotsf Sep 11, 2024

ianbotsf Sep 11, 2024

ianbotsf Sep 11, 2024

ianbotsf Sep 11, 2024

lauzadis left a comment

lauzadis Sep 10, 2024

lauzadis Sep 10, 2024

lauzadis Sep 11, 2024

lauzadis Sep 11, 2024

ianbotsf Sep 12, 2024

lauzadis Sep 12, 2024

ianbotsf Sep 12, 2024

lauzadis Sep 12, 2024

sonarqubecloud bot commented Sep 12, 2024

github-actions bot commented Sep 12, 2024

		include(":hll:dynamodb-mapper:tests:dynamodb-mapper-schema-generator-plugin-test")
		// include(":hll:dynamodb-mapper:tests:dynamodb-mapper-schema-generator-plugin-test")

DDB Mapper filter expressions (runtime components) #1401

DDB Mapper filter expressions (runtime components) #1401

Conversation

ianbotsf commented Sep 9, 2024

Issue #

Description of changes

github-actions bot commented Sep 9, 2024

ianbotsf left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

lauzadis left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

sonarqubecloud bot commented Sep 12, 2024

Quality Gate passed

github-actions bot commented Sep 12, 2024