Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Network endpoints created by CNI NAT plugin is not attached to container #582

Open
yldoge opened this issue Jul 13, 2023 · 0 comments
Open

Comments

@yldoge
Copy link

yldoge commented Jul 13, 2023

The ipconfig inside the container returns nothing.

Beginning state of the network:
image

After code run:

Should the 2 extensions be enabled?

PS C:\Windows\system32> Get-HNSNetwork


ActivityId             : 2D5B4A5C-40C8-4707-9DD9-62AA866A3B7C
AdditionalParams       :
CurrentEndpointCount   : 1
Extensions             : {@{Id=E7C3B2F0-F3C5-48DF-AF2B-10FED6D72E7A; IsEnabled=False; Name=Microsoft Windows Filtering Platform},
                         @{Id=F74F241B-440F-4433-BB28-00F89EAD20D8; IsEnabled=False; Name=Microsoft Azure VFP Switch Extension},
                         @{Id=430BDADD-BAB0-41AB-A369-94B67FA5BE0A; IsEnabled=True; Name=Microsoft NDIS Capture}}
Flags                  : 8
Health                 : @{LastErrorCode=0; LastUpdateTime=133318808183134023}
ID                     : 85D61148-49EF-40BB-A450-3BEBE951D838
IPv6                   : False
LayeredOn              : 24DDF755-8077-4CAD-BEE4-6D6B6B4E015B
MacPools               : {@{EndMacAddress=00-15-5D-2B-FF-FF; StartMacAddress=00-15-5D-2B-F0-00}}
MaxConcurrentEndpoints : 1
Name                   : nat
NatName                : NAT48C0DEC2-ED26-4DC5-A408-9B510A985ABC
Policies               : {@{Type=VLAN; VLAN=1}}
State                  : 1
Subnets                : {@{AdditionalParams=; AddressPrefix=192.168.100.0/24; Flags=0; GatewayAddress=192.168.100.1; Health=;
                         ID=8350EC42-2087-4941-A535-BB4E118B8797; IpSubnets=System.Object[]; ObjectType=5; Policies=System.Object[]; State=0}}
TotalEndpoints         : 1
Type                   : NAT
Version                : 55834574851
Resources              : @{AdditionalParams=; AllocationOrder=2; Allocators=System.Object[]; CompartmentOperationTime=0; Flags=0; Health=;
                         ID=2D5B4A5C-40C8-4707-9DD9-62AA866A3B7C; PortOperationTime=0; State=1; SwitchOperationTime=0; VfpOperationTime=0;
                         parentId=1826BFFB-7D84-461D-B81E-67D7C1BD94B7}
PS C:\Windows\system32> Get-NetIPAddress | Format-Table

ifIndex IPAddress                                       PrefixLength PrefixOrigin SuffixOrigin AddressState PolicyStore
------- ---------                                       ------------ ------------ ------------ ------------ -----------
18      fe80::2026:12f0:fe15:51c8%18                              64 WellKnown    Link         Preferred    ActiveStore
11      fe80::3e4b:b425:85f5:4a92%11                              64 WellKnown    Link         Preferred    ActiveStore
1       ::1                                                      128 WellKnown    WellKnown    Preferred    ActiveStore
18      192.168.100.1                                             24 Manual       Manual       Preferred    ActiveStore
11      10.44.28.245                                              23 Dhcp         Dhcp         Preferred    ActiveStore
1       127.0.0.1                                                  8 WellKnown    WellKnown    Preferred    ActiveStore

Should the generated namespace policy be my portMapping rules, instead of empty?

PS C:\Windows\system32> Get-HnsNamespace

ActivityId       : F1173E93-F496-4595-B99C-E42ED06A2D82
AdditionalParams :
CompartmentGuid  : 79D9FA50-97D7-420B-AF27-864C96489ECC
CompartmentId    : 2
Containers       : {test-container}
Flags            : 0
Health           : @{LastErrorCode=0; LastUpdateTime=133318808092551847}
ID               : B857CF74-2286-416E-A4FE-9B174FB4FDA9
IsDefault        : False
Policies         : {}
ResourceList     : {@{Data=; Type=Endpoint}}
SchemaVersion    : @{Major=0; Minor=0}
State            : 3
Version          : 55834574851
Resources        : @{AdditionalParams=; AllocationOrder=1; Allocators=System.Object[]; CompartmentOperationTime=0; Flags=0; Health=;
                   ID=F1173E93-F496-4595-B99C-E42ED06A2D82; PortOperationTime=0; State=1; SwitchOperationTime=0; VfpOperationTime=0}
PS C:\Windows\system32> Get-VMSwitch

Name SwitchType NetAdapterInterfaceDescription
---- ---------- ------------------------------
nat  Internal
PS C:\Windows\system32> Get-HnsEndPoint


ID                 : 64dd3052-e077-4d11-951c-8fdb73f2ebd5
Name               : test-container_nat
Version            : 55834574851
AdditionalParams   :
Resources          : @{AdditionalParams=; AllocationOrder=4; Allocators=System.Object[]; CompartmentOperationTime=0; Flags=0; Health=;
                     ID=6FB7AB65-8CB2-4623-B444-1ED5425A2CFC; PortOperationTime=0; State=1; SwitchOperationTime=0; VfpOperationTime=0;
                     parentId=2D5B4A5C-40C8-4707-9DD9-62AA866A3B7C}
State              : 2
VirtualNetwork     : 85d61148-49ef-40bb-a450-3bebe951d838
VirtualNetworkName : nat
Policies           : {@{ExternalPort=8886; InternalPort=8888; Protocol=TCP; Type=NAT}}
MacAddress         : 00-15-5D-2B-F8-1C
IPAddress          : 192.168.100.170
PrefixLength       : 24
GatewayAddress     : 192.168.100.1
IPSubnetId         : 2b2fda3e-a14e-439d-8de8-92ff2395ef31
DNSServerList      : 10.50.4.32,10.50.4.33,10.48.4.1
DNSSuffix          :
Namespace          : @{ID=b857cf74-2286-416e-a4fe-9b174fb4fda9}
SharedContainers   : {test-container}



PS C:\Windows\system32> ping 192.168.100.170

Pinging 192.168.100.170 with 32 bytes of data:
Reply from 192.168.100.170: bytes=32 time<1ms TTL=128
Reply from 192.168.100.170: bytes=32 time<1ms TTL=128
Reply from 192.168.100.170: bytes=32 time<1ms TTL=128
Reply from 192.168.100.170: bytes=32 time<1ms TTL=128

Ping statistics for 192.168.100.170:
    Packets: Sent = 4, Received = 4, Lost = 0 (0% loss),
Approximate round trip times in milli-seconds:
    Minimum = 0ms, Maximum = 0ms, Average = 0ms

Is able to ping the endpoint from host namespace

  TCP    127.0.0.1:53562        Dev0027986:53563       ESTABLISHED
  TCP    127.0.0.1:53563        Dev0027986:53562       ESTABLISHED
  TCP    127.0.0.1:53616        Dev0027986:53619       ESTABLISHED
  TCP    127.0.0.1:53619        Dev0027986:53616       ESTABLISHED
  TCP    [::1]:53918            Dev0027986:5985        TIME_WAIT
  TCP    [::1]:53921            Dev0027986:5985        TIME_WAIT
PS C:\Windows\system32> curl http://localhost:8886
curl : Unable to connect to the remote server
At line:1 char:1
+ curl http://localhost:8886
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : InvalidOperation: (System.Net.HttpWebRequest:HttpWebRequest) [Invoke-WebRequest], WebException
    + FullyQualifiedErrorId : WebCmdletWebResponseException,Microsoft.PowerShell.Commands.InvokeWebRequestCommand

My external port is 8886, no port 8886 is established.

My CNI config:

{
    "cniVersion": "0.2.0",
    "name": "nat",
    "type": "nat",
    "master": "Ethernet",
    "ipam": {
        "subnet": "192.168.100.0/24",
        "routes": [
            {
                "gateway": "192.168.100.1"
            }
        ]
    },
    "capabilities": {
        "portMappings": true,
        "dns": true
    }
}

CNI plugin log:
image

My code logic:

        containerId := "test-container"
        l, err := gocni.New(
		gocni.WithPluginConfDir(cniConfDir),
		gocni.WithPluginDir(cniPluginDir),
	)
	l.Load(gocni.WithDefaultConf)
        netNs, err := netns.NewNetNS("")
	result, err := l.Setup(ctx, containerId, netNs.GetPath(), nsOpts...)

	// ============================ create container ============================
	img, err := h.conn.GetImage(ctx, image)
	c, err := h.conn.NewContainer(ctx, containerId,
		containerd.WithImage(img),
		containerd.WithNewSpec(
			oci.WithDefaultSpec(),
			oci.WithImageConfig(img),
			oci.WithHostname("test-container-hostname"),
			oci.WithMounts(mounts),
			oci.WithWindowNetworksAllowUnqualifiedDNSQuery(),
			oci.WithWindowsIgnoreFlushesDuringBoot(),
			oci.WithWindowsNetworkNamespace(netNs.GetPath()),
		),
		containerd.WithNewSnapshot(containerId, img),
	)

	task, err := c.NewTask(ctx, cio.NewCreator(cio.WithStdio))
        task.Start(ctx)

Containerd version:

Client:
  Version:  v1.7.2
  Revision: 0cae528dd6cb557f7201036e9f43420650207b58
  Go version: go1.20.4

Server:
  Version:  v1.7.2
  Revision: 0cae528dd6cb557f7201036e9f43420650207b58
  UUID: c775f801-1980-4709-82da-fd2a591e7be3

OS: Windows Server 2022

@yldoge yldoge changed the title Network endpoints created by CNI NAT plugin is not attached to container. Network endpoints created by CNI NAT plugin is not attached to container Jul 13, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant