Comments: Linux Sockets

Use twisted matrix and get rid of this problem once and for all.

Posted by Valentino at October 11, 2007 10:32 AM

This is not a "weirdness" or even really a Linux peculiarity.

Its part of the POSIX standard -- and there is actually a good reason why a send() or a write() for that matter may not deal with as many bytes as requested. Basically, it allows the kernel to make its life easier by "walking away" if something happens (maybe a certain interrupt on the underlying device -- doesn't have to be a failure case per se).

What tends to bite people is that empirically it will appear that all bytes do get sent/written most of the time -- except once in a blue moon.

IMO, Python and its raw socket library do the right thing by giving direct exposure to this behaviour from the stdlib, and not isolating you from this behaviour.

Yes in 99% of times, probably you just want all bytes to be sent. In which case, use a higher-level pattern or abstraction (like twisted) or just a simple helper function.

But there are cases where you want to know that the system call got turfed before all your bytes were dealt with. For example, things that happened in the mean time may make the rest of the send()/write() irrelevant and the protocol may allow you to escape or frame out actually transmitting these.

Posted by Charl at October 11, 2007 12:47 PM

There are good reasons for this API: it allows you to write highly efficient applications. The downside is that the API is more complicated, and the fix is to use a higher-level library for (like twisted).

I was going to suggest MSG_WAITALL as a flag to send/recv to remove the need for looping, but the documentation says there are still cases where you may get less data than you asked for. Also, MSG_WAITALL (defined in POSIX.1-2001) is not as portable as plain send/recv.

Posted by Marius Gedminas at October 11, 2007 12:57 PM
MT::App::Comments=HASH(0x622520) Subroutine MT::Blog::SUPER::site_url redefined at /kunden/homepages/26/d89284986/htdocs/jotsite/cgi-bin/mt/lib/MT/Object.pm line 125.
MT::App::Comments=HASH(0x622520) Subroutine MT::Blog::SUPER::archive_url redefined at /kunden/homepages/26/d89284986/htdocs/jotsite/cgi-bin/mt/lib/MT/Object.pm line 125.
MT::App::Comments=HASH(0x622520) Subroutine MT::Blog::SUPER::archive_url redefined at /kunden/homepages/26/d89284986/htdocs/jotsite/cgi-bin/mt/lib/MT/Object.pm line 125.