I ran several more tests and have come up with some results.
First, I modified the toy app a bit. In psuedo code:
wait for client to connect
do forever
do n times
generate large_message // (1400 bytes)
send(large_message)
if error on send
// looks like a disconnect happened!
exit
end
end
sleep
end
Note that the above pseudo code is a bit inefficient. Generating the data multiple times will hurt our throughput. However, in the actual implementation the generated data is a counter to help distinguish what packets make it to the client. For this reason, it is is generated on each iteration of the loop. The point of the code is to try to max out whatever buffering is happening at lower levels of the networking stack. We want to keep the buffer as full as possible at all times.
I ran tests with n = 20, 5, 2
The test was really simple:
1 Start server
2 Start ethereal capture
3 Connect to server with client
4 Wait for a couple of messages to come in from server
5 Yank network cable from client
6 Wait a bit and plug network cable back into client
7 Yank network cable from client and wait until server socket times out
Here is what I found out:
A: send() will block if the internal socket buffer is full. The server
stopped generating messages as it waited for the send() call to return.
B: The socket buffer appears to be about 12600 bytes (12KB or so?) I came
up with this number by subtracting the last message the client had
received [RM] before being removed from the network (each message was
numbered) from the last message that the server attempted [SM] to send()
before send() blocked and multiplying by the size of the message [1400B].
E.g. for one of my traces, SM = 23, RM = 14
–> (SM – RM)*1400B = (23-14)*1400B = 12600B –> 12600/1024 = 12.3KB
(Sorry about my notation, and hopefully my math isn’t horribly wrong…)
C: The server will stop sending retransmissions after a few tries and
resort to looking for the disconnected client with arp requests
D: If reconnected, the server will flush its buffer and client will
receive all messages. In particular, if the server has ceased to attempt
retransmits, if it is successfully able to find the client via arp it will
send another retransmit which will then trigger the flush.
E: Eventually, the server will time out completely, seems to be in the
range of 15 minutes or so.
Questions I still have:
What happens if we use the gserver model (have the server application
queue data) and fill the socket buffer while disconnected from the client?
Will that data (socket buffered) disappear when we reconnect after a
timeout or will it be flushed to the client? (I need to run the tests on a
modified version of gserver1) I’m thinking that it will be gone forever,
but if it isn’t, I think I might be able to make a minor change to the
disconnect detection code to make things work how we want.
Is there anyway to get the state of the socket so that we might be able to
reconcile what’s in the socket buffer with what’s in our application’s
queue?
Is there anyway to get unflushed data from the socket buffer back into our
application? If there is, then we could wait for the normal timeout to
occur (which ought to trigger the disconnect/reconnect mechanism on the
server) and transfer all the data in the socket buffer back into our
application’s queue.
It still feels like I’m being a bit naive about this, but I guess progress
is being made 