When syncookies are enabled, the existing code does not drop any entries from the syncache, choosing to send a syncookie response instead. However, in practice this leads to the syncache being full of bogus entries from a SYN flood, and forces all legitimate connections to be handled by syncookies. Essentially, the system ends up behaving as if there is no syncache, which is not an ideal situation.
An alternate approach that may prove feasible is to use a syncookie as the ISN for all connections, instead using arc4random() in the syncache case. This would permit the replacement mechanism of entries within the syncache to operate as normal, as the returning ACK could be accepted by either by virtue of passing the syncookie check, or by matching an existing syncache entry. This approach is currently under investigation; one issue that needs to be addressed is whether the reduced entropy of a syncookie ISN provides adequate protection from remote attackers as compared to one from arc4random().
In this paper, we have seen that an unmodified machine provides unacceptable response times under a simple 10Mb/s SYN flood attack. Two approaches to handing this load are presented and evaluated, and we show that both are able to extensively migitate the effects of a SYN flood and allow the system to continue operating. This goal is reached by the dual approach of reducing memory consumption and state on the server side, and the use of better algorithms to handle a large number of incompleted connections. With the new code, the same hardware is now able to withstand a SYN flood attack while maintaining an acceptable level of service to legitimate clients.