Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Extend wrapDef to def and overdef #2060

Merged
merged 9 commits into from
Dec 20, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 3 additions & 3 deletions packages/malloy/src/dialect/duckdb/dialect_functions.ts
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ import {
DefinitionBlueprint,
DefinitionBlueprintMap,
OverloadedDefinitionBlueprint,
wrapDef,
def,
} from '../functions/util';

const list_extract: DefinitionBlueprint = {
Expand Down Expand Up @@ -102,6 +102,6 @@ export const DUCKDB_DIALECT_FUNCTIONS: DefinitionBlueprintMap = {
string_agg_distinct,
to_seconds,
date_part,
...wrapDef('repeat', {'str': 'string', 'n': 'number'}, 'string'),
...wrapDef('reverse', {'str': 'string'}, 'string'),
...def('repeat', {'str': 'string', 'n': 'number'}, 'string'),
...def('reverse', {'str': 'string'}, 'string'),
};
35 changes: 32 additions & 3 deletions packages/malloy/src/dialect/functions/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -31,14 +31,14 @@ const abs: DefinitionFor<Standard['my_function']> = {
* `${...param}` is expanded to a spread fragment, which is expanded to a comma-separated list of arguments for a variadic parameter
* `${order_by:}` and `${limit:}` expand to `aggregate_order_by` and `aggregate_limit` fragments respectively
* `{ expr: { node: "node_type", ... } }`, for cases where the `function` or `sql` options are insufficiently flexible; generates SQL for an arbitrary node, which may include parameter fragments, spread fragments (optionally with prefix and suffix), and order by or limit fragments.
* `generic` specifies that the overload is generic over some types; it should be an array `['T', types]`, where `'T'` is the name of the generic, and `types` is an array of Malloy types: e.g. `generic: ['T', ['string', 'number', 'timestamp', 'date', 'json']],`. Note that currently we only allow one generic per overload, since that's all we need right now. Under the hood, this creates multiple overloads, one for each type in the generic. Eventually we may make the function system smarter and understand generics more deeply.
* `generic` specifies that the overload is generic over some types; it should be an object `{'T': types}`, where `'T'` is the name of the generic, and `types` is an array of Malloy types: e.g. `generic: {'T', ['string', 'number', 'timestamp', 'date', 'json']},`. Note that currently we only allow one generic per overload, since that's all we need right now. Under the hood, this creates multiple overloads, one for each type in the generic. Eventually we may make the function system smarter and understand generics more deeply.
* `supportsOrderBy` is for aggregate functions (e.g. `string_agg`) which can accept an `{ order_by: value }`. `false` is the default value. `true` indicates that any column may be used in the `order_by`. A value of `'default_only'` means that only the more limited `{ order_by: asc }` syntax is allowed.
* `supportsLimit`: is for aggregate functions (e.g. `string_agg`) which can accept a `{ limit: 10 }`. Default `false`.
* `isSymmetric` is for aggregate functions to indicate whether they are symmetric. Default `false`.

A function with multiple overloads is defined like:

```
```TypeScript
const concat: DefinitionFor<Standard['concat']> = {
'empty': {
takes: {},
Expand All @@ -55,7 +55,7 @@ const concat: DefinitionFor<Standard['concat']> = {

Each dialect may override the standard implementations (that is, the `impl` part only). To do so:

```
```TypeScript
import {MalloyStandardFunctionImplementations as OverrideMap} from '../functions/malloy_standard_functions';

export const DIALECT_OVERRIDES: OverrideMap = {
Expand Down Expand Up @@ -106,6 +106,35 @@ export const DIALECT_FUNCTIONS: DefinitionBlueprintMap = {
};
```

### The `def()` experiment
There is also an experimental shortcut for simple wrapper definitions where the name of the function in SQL and in Malloy is the same, and the definition is not overloaded. Here's an example if using `def` to define the string length function ...

Instead of writing

```ts
const length: DefinitionBluePrint = {
takes: {str: 'string'},
returns: 'number',
impl: {function: 'LENGTH'},
}
export const DIALECT_FUNCTIONS: DefinitionBlueprintMap = {
my_function,
length
}
```

The shortcut looks like this ...

```ts
export const DIALECT_FUNCTIONS: DefinitionBlueprintMap = {
my_function,
...def('length', {str: 'string'}, 'number')
};
```

We are waiting on user feedback on this experiment. While these are simpler to write, they are not simpler to read for a human scanning the file for the definition of a function.

### Dialect Implementation
The `Dialect` implementation then implements `getDialectFunctions`. You should use `expandBlueprintMap` to expand the function blueprints into `DialectFunctionOverloadDef`s.

```ts
Expand Down
91 changes: 52 additions & 39 deletions packages/malloy/src/dialect/functions/util.ts
Original file line number Diff line number Diff line change
Expand Up @@ -724,45 +724,6 @@ export function expandOverrideMapFromBase(
return map;
}

/**
* Shortcut for functions wrapper definitions. Useful only if ....
* 0) Not an overloaded definition
* 1) The function has the same name in malloy and in the dialect
* 2) Every generic in args or return generics is type ['any']
* USAGE:
*
* ...wrapDef('func_name', {'arg0': 'type0', 'arg1': 'type1'}, 'return-type')
*
* @param name name of function
* @param takes Record<Argument blueprint>
* @param returns Return Blueprint
* @returns dot dot dot able blueprint definition
*/
export function wrapDef(
name: string,
takes: Record<string, TypeDescBlueprint>,
returns: TypeDescBlueprint
): DefinitionBlueprintMap {
let anyGenerics = false;
const generic: {[name: string]: TypeDescElementBlueprintOrNamedGeneric[]} =
{};
for (const argType of Object.values(takes)) {
for (const genericRef of findGenerics(argType)) {
generic[genericRef.generic] = ['any'];
anyGenerics = true;
}
}
const newDef: DefinitionBlueprint = {
takes,
returns,
impl: {function: name.toUpperCase()},
};
if (anyGenerics) {
newDef.generic = generic;
}
return {[name]: newDef};
}

/**
* Walks a type and returns all the generic references
* @param tdbp A type
Expand Down Expand Up @@ -795,3 +756,55 @@ function* findGenerics(
}
}
}

/**
* Shortcut for non overloaded functions definitions. Default implementation
* will be the function name turned to upper case. Default type for
* any generics encountered will be `['any']`. Both of these can be over-ridden
* in the `options` parameter.
*
* The two implict defaults (which can be over-ridden) are that the
* impl: will be the upper case version of the function name, and that
* any generic reference will be of type 'any'.
*
* USAGE:
*
* ...def('func_name', {'arg0': 'type0', 'arg1': 'type1'}, 'return-type')
*
* @param name name of function
* @param takes Record<Argument blueprint>
* @param returns Return Blueprint
* @param options Everything from a `DefinitionBlueprint` except `takes` and `returns`
* @returns dot dot dot able blueprint definition
*/
export function def(
name: string,
takes: Record<string, TypeDescBlueprint>,
returns: TypeDescBlueprint,
options: Partial<Omit<DefinitionBlueprint, 'takes' | 'returns'>> = {}
) {
let anyGenerics = false;
const generic: {[name: string]: TypeDescElementBlueprintOrNamedGeneric[]} =
{};
for (const argType of Object.values(takes)) {
for (const genericRef of findGenerics(argType)) {
generic[genericRef.generic] = ['any'];
anyGenerics = true;
}
}
// We have found all the generic references and given them all
// T: ['any']. Use this as a default if the options section
// doesn't provide types for the generics.
if (anyGenerics) {
if (options.generic === undefined) {
options.generic = generic;
}
}
const newDef: DefinitionBlueprint = {
takes,
returns,
impl: {function: name.toUpperCase()},
...options,
};
return {[name]: newDef};
}
6 changes: 3 additions & 3 deletions packages/malloy/src/dialect/mysql/dialect_functions.ts
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@
import {
DefinitionBlueprintMap,
OverloadedDefinitionBlueprint,
wrapDef,
def,
} from '../functions/util';

const string_agg: OverloadedDefinitionBlueprint = {
Expand Down Expand Up @@ -53,6 +53,6 @@ const string_agg_distinct: OverloadedDefinitionBlueprint = {
export const MYSQL_DIALECT_FUNCTIONS: DefinitionBlueprintMap = {
string_agg,
string_agg_distinct,
...wrapDef('repeat', {'str': 'string', 'n': 'number'}, 'string'),
...wrapDef('reverse', {'str': 'string'}, 'string'),
...def('repeat', {'str': 'string', 'n': 'number'}, 'string'),
...def('reverse', {'str': 'string'}, 'string'),
};
6 changes: 3 additions & 3 deletions packages/malloy/src/dialect/postgres/dialect_functions.ts
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@
import {
DefinitionBlueprintMap,
OverloadedDefinitionBlueprint,
wrapDef,
def,
} from '../functions/util';

const string_agg: OverloadedDefinitionBlueprint = {
Expand Down Expand Up @@ -53,6 +53,6 @@ const string_agg_distinct: OverloadedDefinitionBlueprint = {
export const POSTGRES_DIALECT_FUNCTIONS: DefinitionBlueprintMap = {
string_agg,
string_agg_distinct,
...wrapDef('repeat', {'str': 'string', 'n': 'number'}, 'string'),
...wrapDef('reverse', {'str': 'string'}, 'string'),
...def('repeat', {'str': 'string', 'n': 'number'}, 'string'),
...def('reverse', {'str': 'string'}, 'string'),
};
6 changes: 3 additions & 3 deletions packages/malloy/src/dialect/snowflake/dialect_functions.ts
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@

import {AggregateOrderByNode} from '../../model';
import {
wrapDef,
def,
DefinitionBlueprintMap,
OverloadedDefinitionBlueprint,
arg as a,
Expand Down Expand Up @@ -62,6 +62,6 @@ const string_agg_distinct: OverloadedDefinitionBlueprint = {
export const SNOWFLAKE_DIALECT_FUNCTIONS: DefinitionBlueprintMap = {
string_agg,
string_agg_distinct,
...wrapDef('repeat', {'str': 'string', 'n': 'number'}, 'string'),
...wrapDef('reverse', {'str': 'string'}, 'string'),
...def('repeat', {'str': 'string', 'n': 'number'}, 'string'),
...def('reverse', {'str': 'string'}, 'string'),
};
6 changes: 3 additions & 3 deletions packages/malloy/src/dialect/standardsql/dialect_functions.ts
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@
*/

import {
wrapDef,
def,
DefinitionBlueprint,
DefinitionBlueprintMap,
OverloadedDefinitionBlueprint,
Expand Down Expand Up @@ -69,6 +69,6 @@ export const STANDARDSQL_DIALECT_FUNCTIONS: DefinitionBlueprintMap = {
date_from_unix_date,
string_agg,
string_agg_distinct,
...wrapDef('repeat', {'str': 'string', 'n': 'number'}, 'string'),
...wrapDef('reverse', {'str': 'string'}, 'string'),
...def('repeat', {'str': 'string', 'n': 'number'}, 'string'),
...def('reverse', {'str': 'string'}, 'string'),
};
Loading
Loading