As wireless local area networks (WLANs) become more widely deployed due to its transmission rate higher than that of cellular systems and important flexibility, security of WLANs has become a serious concern for an increasing number of organizations because the wireless medium is open for public access within a certain range[1, 2]. In order to provide secure data communication over WLANs, wired equivalent privacy protocol (WEP) has been proposed. However, significant deficiencies have been identified in both the encryption and the authentication mechanism [3, 4]. To enhance the security functionality in the WLANs, IEEE 802.11i  has been proposed to provide confidentiality, integrity and mutual authentication. The 802.11i authentication protocol consists of several components including an 802.1X authentication phase, a 4-way handshake to establish a fresh session key, and an optional group key handshake for group communication. One of the key components to defend against various wireless attacks is the 4way handshake protocol. The 4-way handshake protocol is designed a) to verify the liveness of the peers by randomly generated Nonces, b) to confirm the existence of the pair-wise master key (PMK) between the supplicant and the authenticator to prevent man-in-middle attack, and c) to establish the pairwise transient key (PTK) for subsequent data transmissions between the supplicant and the authenticator. However, due to one design flaw of the protocol that the Message 1 has not been
protected by the MIC field, an attacker can easily
impersonate the authenticator and compose a fake Message 1 to send to the
supplicant, causing the Message 1 DoS attacks [1, 2] or Message 1 DoS flooding
attacks . Some of the research works have been found to provide solutions
against various malicious DoS attacks. However, some of their results [1, 2]
may not sound convictive without enough security and performance evaluation.
Moreover, some of the proposals [6, 7] require too much change on the message
formats or the devices currently installed which make the proposals
unfeasible. In this paper, we examine
the degradation of the 4-way handshake protocol in the presence of the Message
1 DoS attacks and DoS flooding attacks and propose two solutions to protect
WLANs from malicious DoS attacks. CPN is adopted as a formal verification tool
to analyze the security functionality of the original 4-way handshake scheme
and the proposed solutions under the DoS attacks. Based on our security
analysis and formal verification on our proposals by using CPN, we can confirm
that our proposed solutions outperform the previous proposed solutions [1, 2,
6, 7] from two aspects. a) It requires less changes in the current hardware and
software installation of the WLAN devices and thus it is more feasible, b) It
has better performance in the scenario under the Message 1 DoS flooding
attacks. The rest of this paper is structured as follows. In section II, the
two DoS attacks and the existing solution to prevent them will be briefly
reviewed. In section III, our proposed solutions, namely, Message 1
Authentication and Supplicant Active Protection are presented in detail. The
modeling and verification by CPN is presented in section IV. The conclusion is
drawn in section V finally.
One of the key components of IEEE 802.11i authentication protocol to defend against various malicious attacks is its 4-way handshake scheme. However, since Message 1 in the handshake scheme has not been protected by the Message Integrity Code (MIC), the original 4-way handshake scheme specified by the IEEE standard is vulnerable to the Denial of Service (DoS) attacks and DoS flooding attacks. Several countermeasures have been proposed in the literatures to prevent these attacks. However, they have ignored the feasibility of the solutions and the performance of them against the DoS flooding attacks. Motivated by these, in this paper, two security schemes, namely Message 1 Authentication and Supplicant Active Protection are proposed to protect wireless LAN networks free from the DoS attacks. Based on the analysis and formal verification by Colored Petri Nets (CPN), our proposed schemes could provide stronger secure functionality and outperform the existing schemes in the scenario under the Message 1 flooding attacks.
SECURITY IMPROVEMENTS AND ANALYSIS
Message 1 Authentication Although the enhanced two-way handshake protocol can effectively prevent DoS attacks, it requires too many changes on the current message format and hardware. Motivated by the intention to overcome the drawbacks, we have designed a new solution on Message 1 authentication, which has the similar functionality as the enhanced two-way handshake with much less changes in the current message format and the installed hardware as shown in Figure 5. Instead of inserting a new encryption field into Message 1 , the proposal reuses the authentication field MIC, which is also used for the other three messages, to prevent the unauthorized changes to Message 1. Because the authenticator can only access SNonce after receiving Message 2, we make changes on the calculation of the temporary PTK value as follows.
Supplicant Active Protection All the above mentioned proposals cannot perform well in the DoS flooding attacks because the supplicant either needs to store too many nonce values, which will cause the memory exhaustion, or perform too many computation of the MICs, which will cause the CPU exhaustion. In order to fill this gap, we have designed the “Supplicant Active protection” scheme.
The 4-way handshake protocol specified in the standard is vulnerable to the Message 1 DoS attacks and Message 1 DoS flooding attacks. In this paper, we have proposed two solutions to enhance the 4-way handshake protocol. Based on the security analysis and performance evaluations, it can be confirmed that our proposals are more reliable and require less change in the current message format and hardware than the existing solutions. Finally, we have performed an integrated formal verification by the model checking approach with CPN toolkit. The verification results confirm the vulnerability of the original 4-way handshake protocol under the DoS attacks and the effectiveness of our proposals in preventing such attacks.