Kusto functions support

[barnuri]

in GetParameters the sql methods type_name, OdbcPrec, OdbcScale, OBJECT_NAME, OBJECT_SCHEMA_NAME not supported

in GetResultElementLists the sql methods type_name, OBJECT_ID is not supported

replace all by join with relevant tables

[ErikEJ]

Does Kusto have functions and stored procedures??

[ErikEJ]

If not, then we should just add a check for Kusto in the scaffolder, and not attempt to scaffold procedures and functions if we are running against Kusto

[ErikEJ]

And mayb revert Kusto related changes in these classes?

[barnuri]

Does Kusto have functions and stored procedures??

no, they have function table, kusto views is function without params, also they have function table with params, so i need the function table support

[ErikEJ]

Wonder if we can discover them - at any reate, we should just refarin from trying to scaffold stored procedure, so changes to SqlServerRoutineModelFactory.cs are not needed, we need to add a check in ReverseEngineerRunner instead

[ErikEJ]

Have you managed to smoke test this with Kusto AND a SQL database with functions?

[barnuri]

Have you managed to smoke test this with Kusto AND a SQL database with functions?

managed to run e2e on kusto, and run the specific query on sql server online emulator (doesnt have real sql server)

[ErikEJ]

@barnuri You can spin up AdventureWorks on an Azure SQL database, it has some SQL functions

[ErikEJ]

@barnuri I will try this PR branch against Adventureworks (or another database with Functions and stored procedures)

[ErikEJ]

Just tested with Adventureworks, and getting errors - I am reluctant to accept this PR, the queries should be left alone for non_Kusto usage, suggest that a variation for Kusto is used instead.

[barnuri]

@barnuri You can spin up AdventureWorks on an Azure SQL database, it has some SQL functions

looks fine :)

[ErikEJ]

Can you find accountnumber ?

[ErikEJ]

Regardsless, there are too many changes to these two queries, that have worked well and contain a lot of knowledge

[barnuri]

accountnumber

[ErikEJ]

These two queries return wildliy dfferent results against AdventureWorks:

SELECT  
    'Parameter' = p.name,  
    'Type'   = COALESCE(ts.name, tu.name),
    'Length'   = CAST(p.max_length AS INT),  
    'Precision'   = CASE 
              WHEN ts.name = 'uniqueidentifier' THEN p.precision  
              WHEN ts.name IN ('decimal', 'numeric') THEN p.precision
              WHEN ts.name IN ('varchar', 'nvarchar') THEN p.max_length
              ELSE NULL
            END, 
    'Scale'   = CAST(p.scale AS INT),  
    'Order'  = CAST(p.parameter_id AS INT),  
    p.is_output AS output,
    'TypeName' = QUOTENAME(s.name) + '.' + QUOTENAME(tu.name),
	'TypeSchema' = tu.schema_id,
	'TypeId' = p.user_type_id,
    'RoutineName' = o.name,
    'RoutineSchema' = s.name
    from sys.parameters p
    inner join sys.objects AS o on o.object_id = p.object_id
    inner JOIN sys.schemas AS s ON o.schema_id = s.schema_id
    left JOIN sys.types AS tu ON p.user_type_id = tu.user_type_id
    left JOIN sys.types AS ts ON p.system_type_id = ts.system_type_id
    ORDER BY p.object_id, p.parameter_id;

	SELECT  
    'Parameter' = p.name,  
    'Type'   = COALESCE(type_name(p.system_type_id), type_name(p.user_type_id)),  
    'Length'   = CAST(p.max_length AS INT),  
    'Precision'   = CAST(case when type_name(p.system_type_id) = 'uniqueidentifier' 
                then p.precision  
                else OdbcPrec(p.system_type_id, p.max_length, p.precision) end AS INT),  
    'Scale'   = CAST(OdbcScale(p.system_type_id, p.scale) AS INT),  
    'Order'  = CAST(parameter_id AS INT),  
    p.is_output AS output,
    'TypeName' = QUOTENAME(SCHEMA_NAME(t.schema_id)) + '.' + QUOTENAME(TYPE_NAME(p.user_type_id)),
	'TypeSchema' = t.schema_id,
	'TypeId' = p.user_type_id,
    'RoutineName' = OBJECT_NAME(p.object_id),
    'RoutineSchema' = OBJECT_SCHEMA_NAME(p.object_id)
    from sys.parameters p
	LEFT JOIN sys.table_types t ON t.user_type_id = p.user_type_id
    ORDER BY p.object_id, p.parameter_id;

[barnuri]

These two queries return wildliy dfferent results against AdventureWorks:

SELECT  
    'Parameter' = p.name,  
    'Type'   = COALESCE(ts.name, tu.name),
    'Length'   = CAST(p.max_length AS INT),  
    'Precision'   = CASE 
              WHEN ts.name = 'uniqueidentifier' THEN p.precision  
              WHEN ts.name IN ('decimal', 'numeric') THEN p.precision
              WHEN ts.name IN ('varchar', 'nvarchar') THEN p.max_length
              ELSE NULL
            END, 
    'Scale'   = CAST(p.scale AS INT),  
    'Order'  = CAST(p.parameter_id AS INT),  
    p.is_output AS output,
    'TypeName' = QUOTENAME(s.name) + '.' + QUOTENAME(tu.name),
	'TypeSchema' = tu.schema_id,
	'TypeId' = p.user_type_id,
    'RoutineName' = o.name,
    'RoutineSchema' = s.name
    from sys.parameters p
    inner join sys.objects AS o on o.object_id = p.object_id
    inner JOIN sys.schemas AS s ON o.schema_id = s.schema_id
    left JOIN sys.types AS tu ON p.user_type_id = tu.user_type_id
    left JOIN sys.types AS ts ON p.system_type_id = ts.system_type_id
    ORDER BY p.object_id, p.parameter_id;

	SELECT  
    'Parameter' = p.name,  
    'Type'   = COALESCE(type_name(p.system_type_id), type_name(p.user_type_id)),  
    'Length'   = CAST(p.max_length AS INT),  
    'Precision'   = CAST(case when type_name(p.system_type_id) = 'uniqueidentifier' 
                then p.precision  
                else OdbcPrec(p.system_type_id, p.max_length, p.precision) end AS INT),  
    'Scale'   = CAST(OdbcScale(p.system_type_id, p.scale) AS INT),  
    'Order'  = CAST(parameter_id AS INT),  
    p.is_output AS output,
    'TypeName' = QUOTENAME(SCHEMA_NAME(t.schema_id)) + '.' + QUOTENAME(TYPE_NAME(p.user_type_id)),
	'TypeSchema' = t.schema_id,
	'TypeId' = p.user_type_id,
    'RoutineName' = OBJECT_NAME(p.object_id),
    'RoutineSchema' = OBJECT_SCHEMA_NAME(p.object_id)
    from sys.parameters p
	LEFT JOIN sys.table_types t ON t.user_type_id = p.user_type_id
    ORDER BY p.object_id, p.parameter_id;

do you have suggestions ?
from looking in the results is look like the new query return things that we didnt found before maybe its ok ?

maybe we can do if kusto use the new query ? what do you think ?

[barnuri]

fixed the first sql

SELECT * INTO #temp1 FROM (
SELECT 
    c.name,
    COALESCE(ts.name, tu.name) AS type_name,
    c.column_id AS column_ordinal,
    c.is_nullable
FROM sys.columns c
inner join sys.types tu ON c.user_type_id = tu.user_type_id 
inner join sys.objects AS o on o.object_id = c.object_id
inner JOIN sys.schemas AS s ON o.schema_id = s.schema_id
LEFT JOIN sys.types ts ON tu.system_type_id = ts.user_type_id
) as a

SELECT * INTO #temp2 FROM (
SELECT 
    c.name,
    COALESCE(type_name(c.system_type_id), type_name(c.user_type_id)) AS type_name,
    c.column_id AS column_ordinal,
    c.is_nullable
FROM sys.columns c

) as b

-- Compare the results
SELECT * FROM #temp1
EXCEPT 
SELECT * FROM #temp2;

-- Compare in the reverse direction to check if both results are identical
SELECT * FROM #temp2 
EXCEPT 
SELECT * FROM #temp1;

[barnuri]

@ErikEJ the first query return the same
for the second added if kusto

[ErikEJ]

Leaving this until the feedback on the EF Core pr has been addressed

[barnuri]

Leaving this until the feedback on the EF Core pr has been addressed

how it's connected ? the only left comment there is on something that i didnt changed here

[barnuri]

Leaving this until the feedback on the EF Core pr has been addressed

update all the places from the prev PR too like i fixed in EF Core pr, found a solution with less changes the efcore team probably will like

validate the change with efcpt.8 with kusto (smoking test)

[ErikEJ]

Cool, will have a look soon

[ErikEJ]

Have you rebased on latest master?

[barnuri]

Have you rebased on latest master?

yeah, i sync fork master and then pull fork master to this branch

[ErikEJ]

LGTM, just a couple of changes needed

[ErikEJ]

Lets revert any changes to this file, the EF Core 6 tools does not support Kusto anyway, and I will remove EF 6 support soon anyway.

[barnuri]

ok, reverted

[ErikEJ]

Thanks @barnuri for a great contribution

[ErikEJ]

@barnuri any plans to share this via a blog post or similar??

[barnuri]

@barnuri any plans to share this via a blog post or similar??

never done that before, if you have an example of something or write by your blog its great too

also can you update here ? i think they waiting for your response

dotnet/efcore#34832