TcpClient and NetworkStream do not react to network changes

[borrrden]

Steps to Reproduce

1. Build and install the attached project
2. Note the logcat output indicating (on a 5 second loop) that the TcpClient is connected, and its network stream is read-writable
3. Put the device into airplane mode

NetworkStreamBug.zip

Expected Behavior

The TcpClient would no longer show a connected status, and/or the NetworkStream would either have CanRead/CanWrite be false or throw an exception when a read or write is attempted (not applicable to this demo project, but I have a more sophisticated project that shows how writing stuff seems to write into a void never to be seen again if you are interested. It just requires more setup)

Actual Behavior

The TcpClient stays in a connected state, and reads / writes from NetworkStream succeed despite actually doing nothing. Furthermore, on connection restoration this situation continues.

Version Information

Microsoft Visual Studio Enterprise 2017
Version 15.5.7
VisualStudio.15.Release/15.5.7+27130.2036
Microsoft .NET Framework
Version 4.7.02556

Installed Version: Enterprise

Architecture Diagrams and Analysis Tools 00369-60000-00001-AA422
Microsoft Architecture Diagrams and Analysis Tools

Visual Basic 2017 00369-60000-00001-AA422
Microsoft Visual Basic 2017

Visual C# 2017 00369-60000-00001-AA422
Microsoft Visual C# 2017

Visual C++ 2017 00369-60000-00001-AA422
Microsoft Visual C++ 2017

Visual F# 4.1 00369-60000-00001-AA422
Microsoft Visual F# 4.1

Application Insights Tools for Visual Studio Package 8.10.01106.1
Application Insights Tools for Visual Studio

ASP.NET and Web Tools 2017 15.0.31129.0
ASP.NET and Web Tools 2017

ASP.NET Core Razor Language Services 1.0
Provides languages services for ASP.NET Core Razor.

Azure App Service Tools v3.0.0 15.0.31106.0
Azure App Service Tools v3.0.0

Common Azure Tools 1.10
Provides common services for use by Azure Mobile Services and Microsoft Azure Tools.

JavaScript Language Service 2.0
JavaScript Language Service

JavaScript Project System 2.0
JavaScript Project System

JavaScript UWP Project System 2.0
JavaScript UWP Project System

JetBrains ReSharper Ultimate 2017.3.1 Build 111.0.20171221.145902
JetBrains ReSharper Ultimate package for Microsoft Visual Studio. For more information about ReSharper Ultimate, visit http://www.jetbrains.com/resharper. Copyright © 2018 JetBrains, Inc.

Merq 1.1.17-rc (cba4571)
Command Bus, Event Stream and Async Manager for Visual Studio extensions.

Microsoft Continuous Delivery Tools for Visual Studio 0.3
Simplifying the configuration of continuous build integration and continuous build delivery from within the Visual Studio IDE.

Microsoft JVM Debugger 1.0
Provides support for connecting the Visual Studio debugger to JDWP compatible Java Virtual Machines

Microsoft MI-Based Debugger 1.0
Provides support for connecting Visual Studio to MI compatible debuggers

Microsoft Visual C++ Wizards 1.0
Microsoft Visual C++ Wizards

Microsoft Visual Studio VC Package 1.0
Microsoft Visual Studio VC Package

Mono Debugging for Visual Studio 4.8.4-pre (3fe64e3)
Support for debugging Mono processes with Visual Studio.

NuGet Package Manager 4.5.0
NuGet Package Manager in Visual Studio. For more information about NuGet, visit http://docs.nuget.org/.

SQL Server Data Tools 15.1.61710.120
Microsoft SQL Server Data Tools

TypeScript Tools 15.5.11025.1
TypeScript Tools for Microsoft Visual Studio

Visual Studio Code Debug Adapter Host Package 1.0
Interop layer for hosting Visual Studio Code debug adapters in Visual Studio

Visual Studio Tools for CMake 1.0
Visual Studio Tools for CMake

Visual Studio Tools for Universal Windows Apps 15.0.27130.2036
The Visual Studio Tools for Universal Windows apps allow you to build a single universal app experience that can reach every device running Windows 10: phone, tablet, PC, and more. It includes the Microsoft Windows 10 Software Development Kit.

VisualStudio.Mac 1.0
Mac Extension for Visual Studio

Xamarin 4.8.0.760 (fc93f3f5b)
Visual Studio extension to enable development for Xamarin.iOS and Xamarin.Android.

Xamarin Designer 4.8.188 (c5813fa34)
Visual Studio extension to enable Xamarin Designer tools in Visual Studio.

Xamarin.Android SDK 8.1.5.0 (HEAD/75f8c6838)
Xamarin.Android Reference Assemblies and MSBuild support.

Xamarin.iOS and Xamarin.Mac SDK 11.6.1.4 (db807ec)
Xamarin.iOS and Xamarin.Mac Reference Assemblies and MSBuild support.

Log File

Gist link

[borrrden]

Ok so after reading a bit more the title sounds like expected behavior, but the write / read should still cause an exception (not demonstrated here), and polling on the Socket connected to the TCP client should return false (it returns true even in airplane mode)

[grendello]

The behavior is, indeed, as you noted, expected. This is a by-design feature of the TCP/IP stack. There's no "live" status update going on between the endpoints and as long as you don't write to the other endpoint, the local socket is connected. The connected status means that the IP port was bound, an initial communication was established and the state hasn't changed due to either local conditions (file descriptor/handle for the socket, closed. I/O error, local timeout etc) or remote conditions (data was sent to the other endpoint and the packets weren't acknowledged/received because either the other party closed the connection or the network went down). Polling is, indeed, the only way to check whether the connection is still alive - but polling means sending data over the connection in this case. The networking stack, as the one in the BCL, should not send any unsolicited data behind the scenes without the application's consent and knowledge. It is the application's responsibility/privilege to send the data. You may want to implement such behavior (and indeed, the "connected" status of mobile devices is ascertained by a simple and "primitive" ping sent to some well-known IP address most of the time) but I'd rather recommend reacting to events raised when your data write fails rather than polling in the described manner. If you need to know the device's network state, using the OS-provided APIs is the best course of action.

[borrrden]

That’s the thing though. The write doesn’t fail. It succeeds despite being in airplane mode. I am reacting to write failures in general but they never come.

[borrrden]

As a quick update to the project I sent, you can add some polling in and it also returns true despite being in airplane mode.

[grendello]

@borrrden What happens if you use the native Java APIs?

[borrrden]

What would the equivalents be? Can I use one of them to write to a networksteam or does it have to be all changed classes all the way up?

[grendello]

https://developer.android.com/reference/java/net/Socket.html would be the direct equivalent. You could also use our AndroidHttpClientHandler with HttpClient - it uses the native Android stack (https://developer.android.com/reference/java/net/HttpURLConnection.html) for actual networking and you can request a stream from the returned content object. The objective here would be to bypass .NET BCL stack to see if the native Android one behaves in the same way. If it does then the issue is specific to the OS, if the native part works as expected then we have a bug in Mono's BCL.

[borrrden]

Here is the updated test using Java classes:

// Socket creation
_socket = new Socket(InetAddress.GetByName("www.google.com", 80);

// Test connection loop
if(_socket.IsConnected) {
    Log.Info("Test", "Still connected!");
}

try {
    _socket.OutputStream.Write(new byte[1] { 0 }, 0, 1);
    Log.Info("Test", "Writeable");
} catch(Exception e) {
    Log.Error("Test", $"Error writing {e}");
}

When I put the device into airplane mode, I see something that I can work with:

Still Connected!
Error writing Java.Net.SocketException: Broken pipe
(stack trace)

[borrrden]

Note that "Still Connected!" always prints despite the write failures though. That won't bother me that much as long as writing to the stream throws an exception.

[grendello]

OK, that gives us some information. It might be a bug in our networking stack, but it's just occurred to me that it might be also buffering in the network stream on our side. What happens if you flush the managed network stream? Do you get an error/exception?

[borrrden]

I tried adding a flush call before (FlushAsync technically) but it didn't seem to have any noticeable effects.

[borrrden]

Also some more information is that even after bringing the device back online, the bytes are never sent over the wire (which I would expect if it were buffering)

[borrrden]

Sorry for spamming. More information. System.Net.Sockets.Socket also throws an exception immediately on send:

SocketException (0x80004005: The socket has been shut down)

[grendello]

OK, so buffering is still on the table. The issue might be in NetworkStream (very likely)

[borrrden]

Now this is interesting....calling Write in this repro project produces an exception. I wonder if getting this exception is dependent on the manifest capabilities....

[grendello]

What capabilities do you have in mind?

[borrrden]

ACCESS_NETWORK_STATE or ACCESS_WIFI_STATE come to mind.

[grendello]

That might be important indeed. Although a write error exception should still be thrown regardless of whether the permissions are there or not.

[borrrden]

Hmmm I don't particularly get it. The other difference is that the project using NetworkStream in the failing case is a .NET Standard 2.0 library while the repro case is just straight Xamarin. Could that be a factor?

EDIT That is to say, I made sure those two were active in the failing project, but it didn't help.

[borrrden]

Oh I think i figured out the difference! The failing project is using a LAN IP address, and the repro project is using www.google.com

Using a LAN address on the repro project causes the issue. Let me check to see if Java has the same issue.

[grendello]

No, netstandard is not an issue, it's just a bunch of type forwarders and PCL-like facade assemblies. Under the hood it's still Mono and its BCL.

The permissions you mentioned deal with access to network information, not the network itself. The connected state of the network is not synonymous with the socket's connected state. The former basically checks if one can reach some server, the latter deals with the socket (which is an open file descriptor on a very high level) state - whether it's bound (has an IP and a port assigned) and/or connected (was the connection established at some point in the past and no error has occurred until this moment). I strongly suspect an issue with buffering or state maintenance in NetworkStream. One problem here is that the BCL must not query the network state from the OS - because of permissions the app might not have.

[borrrden]

Java socket has identical behavior here 👎 It doesn't have to be a LAN IP address, just an IP address in general (DNS is a factor perhaps?).

[grendello]

DNS does send a packet on address resolution to the server, which would trigger the "poll" and thus the error. I wonder if Android does some low-level buffering (probably limited) to retransmit the packets after the network is back, if possible. The Linux TCP/IP stack has buffering (as any TCP stack) on the socket level (I think the default is 32k for Linux, at least on desktop, not sure on mobile devices) but that is probably not the problem here. Either way, it appears to be a fact of life that has to be dealt with... :)

[borrrden]

It seems pretty hard to deal with. The library I am working on is deployed to Xamarin iOS, Xamarin Android, UWP, and .NET Core so it's awkward to inject platform specific functionality inside. Is there anything you can think of that could help here, inside the constraints of NS 2.0? I filed an issue on the Android tracker to see what they have to say about it, but it seems like the issue as far as Xamarin is concerned can be closed.

[grendello]

The only idea I have is to attempt DNS name resolution, at least periodically. The downside is that it can be slow :( You could also try ICMP ping to a known IP, but ICMP can be blocked by firewalls, proxies etc so it's not a fireproof solution. Networking is an inherently tough nut to crack and high-level cross-platform solutions to such problems might be impossible. With non-trivial cross-platform applications you will always need to deal with a little bit of OS-specific code, and that's ok IMO. Platform-specific code often makes stuff simpler, and that's a good thing.

[borrrden]

I don't need to test for network connectivity in general. All I need is "can I reach the endpoint I am trying to reach" and react based on a yes or no answer. This was working well in 1.x of our product when we used HTTP to do everything, but now we have switched to web socket messaging. Thanks for the ideas though!