Repository navigation
fix(libudx): keep receiving after an undelivered datagram on Windows - #34
Closed
ColbySerpa wants to merge 1 commit into
Closed
ColbySerpa wants to merge 1 commit into
ColbySerpa wants to merge 1 commit into
Conversation
On Windows, a UDP socket reports on its next receive that a datagram it sent earlier was not delivered: WSAECONNRESET after an ICMP "port unreachable", and WSAENETRESET after an ICMP "time to live exceeded". The socket itself still works. The receive loop of `UdxSocket` ended at the first receive error. So one datagram to a peer that had gone, or one low-TTL holepunch probe, left the socket deaf: it delivered no more datagrams, and no more packets to the streams registered on it, until another stream was registered. A DHT node shares one socket between its RPC and its streams, so on Windows a node stopped hearing replies and its streams stalled. The loop now carries on after those two errors, as libuv does (`uv__process_udp_recv_req`), and still ends on any other error. Other systems report no such error on an unconnected socket, so nothing changes for them. Tests: a socket, and a stream, still receive after a datagram from their socket was refused by a closed local port. On Windows both fail without this change, where the receive times out, and pass with it. Elsewhere they pass either way. Unit tests cover which errors the loop carries on after.
3 tasks done
3 tasks done
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
UdxSocketstopped receiving after one of its datagrams went undelivered. Its receive loop ended at the first receive error, and Windows reports the ICMP error that such a datagram draws on the next receive of that socket.WSAECONNRESET,WSAENETRESET), as libuv does. Other systems are unaffected.Motivation
Windows hands the ICMP error for a datagram that a UDP socket sent earlier to the next receive on that socket (
recvfrom):WSAECONNRESET: "a previous send operation resulted in an ICMP Port Unreachable message", so a datagram to a peer that has gone;WSAENETRESET: "the time to live has expired", so a low-TTL holepunch probe (SocketRef::send_holepunch).Neither says anything about the socket, which still sends and receives. But the receive loop in
UdxSocketInner::ensure_recv_loopwaswhile let Ok(..) = udp.recv_from(..).await, so the first such error ended it. After that the socket delivered no more datagrams, and no more packets to the streams registered on it, until another stream was registered and started the loop again.A DHT node shares a socket between its RPC and its streams, and routinely sends to peers that have gone. So on Windows a node stopped hearing replies, and its open streams stalled.
libuv, which the Node side runs on, ignores exactly these two errors for UDP and keeps receiving (
uv__process_udp_recv_reqinsrc/win/udp.c: "A previous sendto operation failed; ignore this error").Changes
libudxConnectionReset, and on Windows OS error 10052 (WSAENETRESET, whichstdgives noErrorKind). It still ends on any other error.libudxlibudxNo public API changes.
Testing
cargo test --workspace(Linux: 1086 passed, 0 failed, 15 ignored)cargo test -p peeroxide-cli --test live_commands -- --ignored(Linux: 6 passed)cargo clippy --workspace --all-targets -- -D warnings(Linux, and-p libudxon Windows)On Windows 11,
cargo test -p libudx --test socket:socket_receives_after_a_refused_datagramandstream_receives_after_a_refused_datagramfail in 3 runs of 3: the receive times out;On Linux they pass either way, since no error is reported there.
Notes
WSAENETRESETis handled from the documentation, not from a reproduction. A TTL-1 probe from my network drew no ICMP answer, so I could not observe it.WSAECONNRESETis reproduced by the tests.windows_icmp_errors_report_an_undelivered_datagram, and the Windows behaviour of the two integration tests, only run locally.stream_write_read_10mbandthroughput_sustainedtime out there with and without this change. The rest of thelibudx,peeroxide-dhtandpeeroxidesuites pass, Node interop included. That is a pacing matter, which I will raise separately.