Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Can't import Component Model functions in WASI apps #94513

Closed
mhmd-azeez opened this issue Nov 8, 2023 · 12 comments
Closed

Can't import Component Model functions in WASI apps #94513

mhmd-azeez opened this issue Nov 8, 2023 · 12 comments
Assignees
Labels
arch-wasm WebAssembly architecture area-Build-mono os-wasi Related to WASI variant of arch-wasm
Milestone

Comments

@mhmd-azeez
Copy link

Description

While you can import functions in WASI apps (see #90786), the process breaks down when the imported namespace contains : and /. This is problematic because these are valid characters and are used widely in Wasm Component Model spec. For example, wasi:http/handler is a valid import namespace:

(component
  (import "custom-hook" (func (param string) (result string)))
  (import "wasi:http/handler" (instance
    (export "request" (type $request (sub resource)))
    (export "response" (type $response (sub resource)))
    (export "handle" (func (param (own $request)) (result (own $response))))
  ))
  (import "url=<https://mycdn.com/my-component.wasm>" (component ...))
  (import "relative-url=<./other-component.wasm>,integrity=<sha256-X9ArH3k...>" (component ...))
  (import "locked-dep=<my-registry:sqlite@1.2.3>,integrity=<sha256-H8BRh8j...>" (component ...))
  (import "unlocked-dep=<my-registry:imagemagick@{>=1.0.0}>" (instance ...))
  (import "integrity=<sha256-Y3BsI4l...>" (component ...))
  ... impl
  (export "wasi:http/handler" (instance $http_handler_impl))
  (export "get-JSON" (func $get_json_impl))
)

Reproduction Steps

csproj:

<Project Sdk="Microsoft.NET.Sdk">

  <PropertyGroup>
    <TargetFramework>net8.0</TargetFramework>
    <RuntimeIdentifier>wasi-wasm</RuntimeIdentifier>
    <OutputType>Exe</OutputType>
    <PublishTrimmed>true</PublishTrimmed>
    <WasmSingleFileBundle>true</WasmSingleFileBundle>
    <WasmBuildNative>true</WasmBuildNative>
    <AllowUnsafeBlocks>true</AllowUnsafeBlocks>
  </PropertyGroup>

  <ItemGroup>
    <NativeFileReference Include="$(MSBuildThisFileDirectory)extism.c" />
    <_WasmNativeFileForLinking Include="@(NativeFileReference)" />
  </ItemGroup>

</Project>

Program.cs:

using System;
using System.Runtime.CompilerServices;
using System.Runtime.InteropServices;

public class Program
{
    [DllImport("extism:host/user")]
    public static extern void do_something();

    public static void Main(string[] args)
    {
        
    }
}

extism.c:

#include <string.h>
#include <stdint.h>
#include <assert.h>
#include <stdlib.h>
#include <stdbool.h>
#include <stdio.h>

#define IMPORT(a, b) __attribute__((import_module(a), import_name(b)))

IMPORT("extism:host/user", "do_something")
extern void do_something_wrapper();

void do_something() {
    return do_something_wrapper();
}

Then run:

PS D:\x\dotnet\component-model-repro> dotnet build

And then:

PS D:\x\dotnet\component-model-repro\obj\Debug\net8.0\wasi-wasm\wasm\for-build> wasm-objdump --details --section=import .\ComponentModelRepro.wasm | Select-String -Pattern 'extism'

Full Repro:
https://github.com/mhmd-azeez/component-model-repro

Expected behavior

PS D:\x\dotnet\component-model-repro\obj\Debug\net8.0\wasi-wasm\wasm\for-build> wasm-objdump --details --section=import .\ComponentModelRepro.wasm | Select-String -Pattern 'extism'

 - func[0] sig=0 <extism:host/user.do_something> <- extism:host/user.do_something

Actual behavior

PS D:\x\dotnet\component-model-repro\obj\Debug\net8.0\wasi-wasm\wasm\for-build> wasm-objdump --details --section=import .\ComponentModelRepro.wasm | Select-String -Pattern 'extism'

Notice: extism:host/user.do_something is not imported.

Also, in pinvoke-table.h there is no entry for do_something.

Regression?

No response

Known Workarounds

Simplify the import statemet in Program.cs:

[DllImport("extism")]
public static extern void do_something();

But keep extism.c as is:

#include <string.h>
#include <stdint.h>
#include <assert.h>
#include <stdlib.h>
#include <stdbool.h>
#include <stdio.h>

#define IMPORT(a, b) __attribute__((import_module(a), import_name(b)))

IMPORT("extism:host/user", "do_something")
extern void do_something_wrapper();

void do_something() {
    return do_something_wrapper();
}

Configuration

PS D:\x\dotnet\component-model-repro> dotnet --info
.NET SDK:
 Version:   8.0.100-rc.2.23502.2
 Commit:    0abacfc2b6

Runtime Environment:
 OS Name:     Windows
 OS Version:  10.0.22621
 OS Platform: Windows
 RID:         win-x64
 Base Path:   C:\Program Files\dotnet\sdk\8.0.100-rc.2.23502.2\

.NET workloads installed:
 [wasi-experimental]
   Installation Source: SDK 8.0.100-rc.2
   Manifest Version:    8.0.0-rc.2.23479.6/8.0.100-rc.2
   Manifest Path:       C:\Program Files\dotnet\sdk-manifests\8.0.100-rc.2\microsoft.net.workload.mono.toolchain.current\8.0.0-rc.2.23479.6\WorkloadManifest.json
   Install Type:              Msi


Host:
  Version:      8.0.0-rc.2.23479.6
  Architecture: x64
  Commit:       0b25e38ad3

.NET SDKs installed:
  7.0.403 [C:\Program Files\dotnet\sdk]
  8.0.100-rc.2.23502.2 [C:\Program Files\dotnet\sdk]

.NET runtimes installed:
  Microsoft.AspNetCore.App 6.0.24 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 7.0.13 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 8.0.0-rc.2.23480.2 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.NETCore.App 6.0.24 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 7.0.13 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 8.0.0-rc.2.23479.6 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.WindowsDesktop.App 6.0.24 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
  Microsoft.WindowsDesktop.App 7.0.13 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
  Microsoft.WindowsDesktop.App 8.0.0-rc.2.23479.10 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]     

Other architectures found:
  x86   [C:\Program Files (x86)\dotnet]
    registered at [HKLM\SOFTWARE\dotnet\Setup\InstalledVersions\x86\InstallLocation]

Environment variables:
  Not set

global.json file:
  Not found

Learn more:
  https://aka.ms/dotnet/info

Download .NET:
  https://aka.ms/dotnet/download
PS D:\x\dotnet\component-model-repro> dotnet workload list

Installed Workload Id      Manifest Version                     Installation Source
-----------------------------------------------------------------------------------
wasi-experimental          8.0.0-rc.2.23479.6/8.0.100-rc.2      SDK 8.0.100-rc.2   

Use `dotnet workload search` to find additional workloads to install.

Other information

No response

@ghost ghost added the untriaged New issue has not been triaged by the area owner label Nov 8, 2023
@radical radical added arch-wasm WebAssembly architecture os-wasi Related to WASI variant of arch-wasm labels Nov 8, 2023
@ghost
Copy link

ghost commented Nov 8, 2023

Tagging subscribers to 'arch-wasm': @lewing
See info in area-owners.md if you want to be subscribed.

Issue Details

Description

While you can import functions in WASI apps (see #90786), the process breaks down when the imported namespace contains : and /. This is problematic because these are valid characters and are used widely in Wasm Component Model spec. For example, wasi:http/handler is a valid import namespace:

(component
  (import "custom-hook" (func (param string) (result string)))
  (import "wasi:http/handler" (instance
    (export "request" (type $request (sub resource)))
    (export "response" (type $response (sub resource)))
    (export "handle" (func (param (own $request)) (result (own $response))))
  ))
  (import "url=<https://mycdn.com/my-component.wasm>" (component ...))
  (import "relative-url=<./other-component.wasm>,integrity=<sha256-X9ArH3k...>" (component ...))
  (import "locked-dep=<my-registry:sqlite@1.2.3>,integrity=<sha256-H8BRh8j...>" (component ...))
  (import "unlocked-dep=<my-registry:imagemagick@{>=1.0.0}>" (instance ...))
  (import "integrity=<sha256-Y3BsI4l...>" (component ...))
  ... impl
  (export "wasi:http/handler" (instance $http_handler_impl))
  (export "get-JSON" (func $get_json_impl))
)

Reproduction Steps

csproj:

<Project Sdk="Microsoft.NET.Sdk">

  <PropertyGroup>
    <TargetFramework>net8.0</TargetFramework>
    <RuntimeIdentifier>wasi-wasm</RuntimeIdentifier>
    <OutputType>Exe</OutputType>
    <PublishTrimmed>true</PublishTrimmed>
    <WasmSingleFileBundle>true</WasmSingleFileBundle>
    <WasmBuildNative>true</WasmBuildNative>
    <AllowUnsafeBlocks>true</AllowUnsafeBlocks>
  </PropertyGroup>

  <ItemGroup>
    <NativeFileReference Include="$(MSBuildThisFileDirectory)extism.c" />
    <_WasmNativeFileForLinking Include="@(NativeFileReference)" />
  </ItemGroup>

</Project>

Program.cs:

using System;
using System.Runtime.CompilerServices;
using System.Runtime.InteropServices;

public class Program
{
    [DllImport("extism:host/user")]
    public static extern void do_something();

    public static void Main(string[] args)
    {
        
    }
}

extism.c:

#include <string.h>
#include <stdint.h>
#include <assert.h>
#include <stdlib.h>
#include <stdbool.h>
#include <stdio.h>

#define IMPORT(a, b) __attribute__((import_module(a), import_name(b)))

IMPORT("extism:host/user", "do_something")
extern void do_something_wrapper();

void do_something() {
    return do_something_wrapper();
}

Then run:

PS D:\x\dotnet\component-model-repro> dotnet build

And then:

PS D:\x\dotnet\component-model-repro\obj\Debug\net8.0\wasi-wasm\wasm\for-build> wasm-objdump --details --section=import .\ComponentModelRepro.wasm | Select-String -Pattern 'extism'

Full Repro:
https://github.com/mhmd-azeez/component-model-repro

Expected behavior

PS D:\x\dotnet\component-model-repro\obj\Debug\net8.0\wasi-wasm\wasm\for-build> wasm-objdump --details --section=import .\ComponentModelRepro.wasm | Select-String -Pattern 'extism'

 - func[0] sig=0 <extism:host/user.do_something> <- extism:host/user.do_something

Actual behavior

PS D:\x\dotnet\component-model-repro\obj\Debug\net8.0\wasi-wasm\wasm\for-build> wasm-objdump --details --section=import .\ComponentModelRepro.wasm | Select-String -Pattern 'extism'

Notice: extism:host/user.do_something is not imported.

Also, in pinvoke-table.h there is no entry for do_something.

Regression?

No response

Known Workarounds

Simplify the import statemet in Program.cs:

[DllImport("extism")]
public static extern void do_something();

But keep extism.c as is:

#include <string.h>
#include <stdint.h>
#include <assert.h>
#include <stdlib.h>
#include <stdbool.h>
#include <stdio.h>

#define IMPORT(a, b) __attribute__((import_module(a), import_name(b)))

IMPORT("extism:host/user", "do_something")
extern void do_something_wrapper();

void do_something() {
    return do_something_wrapper();
}

Configuration

PS D:\x\dotnet\component-model-repro> dotnet --info
.NET SDK:
 Version:   8.0.100-rc.2.23502.2
 Commit:    0abacfc2b6

Runtime Environment:
 OS Name:     Windows
 OS Version:  10.0.22621
 OS Platform: Windows
 RID:         win-x64
 Base Path:   C:\Program Files\dotnet\sdk\8.0.100-rc.2.23502.2\

.NET workloads installed:
 [wasi-experimental]
   Installation Source: SDK 8.0.100-rc.2
   Manifest Version:    8.0.0-rc.2.23479.6/8.0.100-rc.2
   Manifest Path:       C:\Program Files\dotnet\sdk-manifests\8.0.100-rc.2\microsoft.net.workload.mono.toolchain.current\8.0.0-rc.2.23479.6\WorkloadManifest.json
   Install Type:              Msi


Host:
  Version:      8.0.0-rc.2.23479.6
  Architecture: x64
  Commit:       0b25e38ad3

.NET SDKs installed:
  7.0.403 [C:\Program Files\dotnet\sdk]
  8.0.100-rc.2.23502.2 [C:\Program Files\dotnet\sdk]

.NET runtimes installed:
  Microsoft.AspNetCore.App 6.0.24 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 7.0.13 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 8.0.0-rc.2.23480.2 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.NETCore.App 6.0.24 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 7.0.13 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 8.0.0-rc.2.23479.6 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.WindowsDesktop.App 6.0.24 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
  Microsoft.WindowsDesktop.App 7.0.13 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
  Microsoft.WindowsDesktop.App 8.0.0-rc.2.23479.10 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]     

Other architectures found:
  x86   [C:\Program Files (x86)\dotnet]
    registered at [HKLM\SOFTWARE\dotnet\Setup\InstalledVersions\x86\InstallLocation]

Environment variables:
  Not set

global.json file:
  Not found

Learn more:
  https://aka.ms/dotnet/info

Download .NET:
  https://aka.ms/dotnet/download
PS D:\x\dotnet\component-model-repro> dotnet workload list

Installed Workload Id      Manifest Version                     Installation Source
-----------------------------------------------------------------------------------
wasi-experimental          8.0.0-rc.2.23479.6/8.0.100-rc.2      SDK 8.0.100-rc.2   

Use `dotnet workload search` to find additional workloads to install.

Other information

No response

Author: mhmd-azeez
Assignees: -
Labels:

arch-wasm, untriaged, area-Build-mono, os-wasi

Milestone: -

@radical radical removed the untriaged New issue has not been triaged by the area owner label Nov 8, 2023
@radical
Copy link
Member

radical commented Nov 8, 2023

cc @maraf

@maraf
Copy link
Member

maraf commented Nov 9, 2023

The DllImportAttribute takes a name of dll/module to import the function from. So it needs to match the name of the .c file in this case extism.c

@lambdageek
Copy link
Member

Imports directly in C# without a .c wrapper will be supported once #93824 is implemented. We should ensure that all valid wasm import names are supported

@lewing lewing modified the milestones: Future, 9.0.0 Nov 9, 2023
@lewing lewing assigned lewing and kg and unassigned lewing Nov 9, 2023
@lewing
Copy link
Member

lewing commented Nov 9, 2023

WasmImportLinkageAttribute is in now, we should add this to the wasm interop work.

@mhmd-azeez
Copy link
Author

The DllImportAttribute takes a name of dll/module to import the function from. So it needs to match the name of the .c file in this case extism.c

The problem is, : is not a valid path character in Windows

@lewing
Copy link
Member

lewing commented Nov 9, 2023

The DllImportAttribute takes a name of dll/module to import the function from. So it needs to match the name of the .c file in this case extism.c

The problem is, : is not a valid path character in Windows

@maraf was suggesting you try [DllImport("extism")] that should resolve to the correct thing given the declarations in the csproj. I believe it should work after that.

@lewing
Copy link
Member

lewing commented Nov 9, 2023

But the wasm import still appears to be being removed. We will take a look

@lewing
Copy link
Member

lewing commented Nov 9, 2023

Sorry, hadn't looked closely enough at this. When you use NativeFileReference you are adding that code to be statically linked and if you specify a ".c" file it will use the filename (without .c) as the module name. What you are effectively doing with extism.c is making a stub library that then calls the wasm import and as @maraf pointed out [DllImport("extism")] works now and is how we intend that to work (mostly). NativeFileReference can also point to an object file and the filename will be the module by default.

If your request is to make [DllImport("extism:host/user")] work without the .c stub and generate a wasm import instead of being resolved statically we intend to allow that with #93824 and the following:

[WasmImportLinkage]
[DllImport("extism:host/user")]
public static extern void do_something();

in the future. I hope that answers your question and sorry for any additional confusion my replies caused.

@mhmd-azeez
Copy link
Author

@lewing thank you for your reply. I had already listed that as a workaround in the issue and have implemented it in our PDK:
extism/dotnet-pdk#28

I opened this issue for 2 reasons:

  1. To make sure the .NET team is aware of the use case and issue so that it gets fixed in the future. I am glad that this is getting fixed with WasmImportLinkage

  2. To see if there are any other workarounds in the meanwhile. Is there any way we can loosen up the constraint of having a c file exactly with exactly the same name as the module name? Maybe StartsWith is enough?

I understand that .NET 8 release is close and you might not want to change anything especially since this will get fixed for good in .NET 9

@lewing
Copy link
Member

lewing commented Nov 10, 2023

#94615 is a quick fix that should resolve the issue. It may not land as is but it works.

@lewing
Copy link
Member

lewing commented Nov 21, 2023

fixed in #94615 any additional issues should be tracked separately.

@lewing lewing closed this as completed Nov 21, 2023
@github-actions github-actions bot locked and limited conversation to collaborators Dec 22, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
arch-wasm WebAssembly architecture area-Build-mono os-wasi Related to WASI variant of arch-wasm
Projects
None yet
Development

No branches or pull requests

6 participants