Skip directly to content

Minimize RSR Award Detail

Research Spending & Results

Award Detail

Awardee:UNIVERSITY OF SOUTHERN CALIFORNIA
Doing Business As Name:University of Southern California
PD/PI:
  • Jelena Mirkovic
  • (310) 448-9170
  • mirkovic@isi.edu
Co-PD(s)/co-PI(s):
  • Genevieve Bartlett
Award Date:08/19/2013
Estimated Total Award Amount: $ 479,487
Funds Obligated to Date: $ 479,487
  • FY 2013=$479,487
Start Date:10/01/2013
End Date:12/31/2016
Transaction Type:Grant
Agency:NSF
Awarding Agency Code:4900
Funding Agency Code:4900
CFDA Number:47.070
Primary Program Source:040100 NSF RESEARCH & RELATED ACTIVIT
Award Title or Description:TWC: Option: Small: FRADE: Model Human Behavior for Flash cRowd Attack DEfense
Federal Award ID Number:1319215
DUNS ID:072933393
Parent DUNS ID:072933393
Program:Secure &Trustworthy Cyberspace
Program Officer:
  • Nina Amla
  • (703) 292-7991
  • namla@nsf.gov

Awardee Location

Street:University Park
City:Los Angeles
State:CA
ZIP:90089-0001
County:Los Angeles
Country:US
Awardee Cong. District:37

Primary Place of Performance

Organization Name:University of Southern California
Street:4676 Admiralty Way, Ste 1001
City:Marina del Rey
State:CA
ZIP:90292-6611
County:Marina del Rey
Country:US
Cong. District:33

Abstract at Time of Award

Application-level, aka ``flash-DDoS'' attacks are the most challenging form of distributed denial of service (DDoS). They flood the victim with legitimate-like service requests generated from numerous bots. There is no defense today that is even remotely effective against flash-DDoS attacks, thus such attacks are today a serious and unmitigated threat to any server. Our project works on developing defenses against flash-DDoS attacks that can pinpoint traffic sent by automated bots and differentiate it from human-generated traffic. Bot IPs are then blacklisted and their traffic filtered protecting the server under attack without any damage to legitimate users. Our project develops novel technologies called ASTUTE (pASsive TUring TEsts) to distinguish bots from human users, by modeling three aspects of human user behavior: (1) dynamics of human-server interaction, (2) human preference for server content, and (3) human processing of visual and textual cues. IP addresses of detected bots will be blacklisted and their traffic will be dropped during server overload. ASTUTE technologies model human behavior without conscious human participation, thus performing Turing tests (human vs machine differentiation) transparently to humans. We will implement all our code as extensions of popular open-source server platforms, such as Apache (for Web) and bind (for DNS). At the end of this work we will deliver working prototypes of these extensions, thus our research will directly transition into practice for any interested party at no cost to them. All our code will be released as open-source under the GNU GPL v3 license.

Publications Produced as a Result of this Research

Note: When clicking on a Digital Object Identifier (DOI) number, you will be taken to an external site maintained by the publisher. Some full text articles may not yet be available without a charge during the embargo (administrative interval).

Some links on this page may take you to non-federal websites. Their policies may differ from this site.

Simon S Woo, Jelena Mirkovic "Improving Recall and Security of Passphrases through Use of Mnemonics" Passwords Conference, v., 2016, p..


Project Outcomes Report

Disclaimer

This Project Outcomes Report for the General Public is displayed verbatim as submitted by the Principal Investigator (PI) for this award. Any opinions, findings, and conclusions or recommendations expressed in this Report are those of the PI and do not necessarily reflect the views of the National Science Foundation; NSF has not approved or endorsed its content.

Application-level DDoS attacks, also known as Layer 7 attacks or “flash-crowd attacks” flood servers with service requests. These attacks are particularly difficult to handle because the attack requests are highly similar to those that a legitimate client would send. Application-level attacks are also effective at a much lower rate than attacks that simply consume bandwidth, and thus can be launched by smaller botnets. Application-level DDoS attacks waste resources, rack up unnecessary billing charges for CDN and remote-hosted customers, and deny or slow down service to legitimate users. Arbor Networks Report for 2016 shows that 97% of respondents experience application-level attacks.

In our FRADE project, we have developed three models that enable a server under attack to differentiate legitimate human users from bots, and to blacklist bot addresses: the dynamics, the semantics and the deception model. We have specifically focused on protecting Web servers from DDoS, because Web is currently the most popular service on the Internet.

Our dynamics model consists of three sub-models that capture the dynamics of a client’s interaction with the server. Our DYN1 model captures the count of requests sent by a client, for non-embedded objects, our DYN2 model captures the count of requests for embedded objects and our DYN3 model captures the time the server spent processing requests for that specific client. All these measures are trained from server request logs, and captured over different time intervals, modeling burstiness at small time scales and the intermittent nature of human-generated requests. Bots that send requests aggressively, regularly or that generate costly requests will be detected by our dynamics model.

Our semantics model learns probabilities of request sequences, and is trained from server request logs. Bots that generate requests from a hard-coded sequence, random requests,  or requests for non-existing pages will be detected by our semantics model.

Our deception model plants invisible hyperlinks into Web pages and blacklist any client that accesses them. Humans will not click on these links as they cannot see them, and they are planted at parts of a page where they are unlikely to be clicked by mistake. Bots that mine hyperlinks from pages and select the next link at random would be detected by our deception model.

FRADE’s modules that implement dynamics and semantics models are passive modules that work on Web server logs. They are thus easily adopted by any platform. FRADE’s deception module can process a variety of pages and insert invisible hyperlinks. When our models flag a client as a bot, FRADE communicates this information to iptables or another firewall technology, where the bot gets blacklisted.

FRADE was tested with real attacks and human users. We have replicated three copyright-free and diverse server contents – Imgur, Reddit and Wikipedia – in the DeterLab testbed. We then recruited Amazon Mechanical Turk participants to visit these servers for a while, so we could capture human behavior and train our dynamics and semantics models. In the next phase of testing we have generated attacks on our servers, using several hosts in DeterLab, and virtualizing many attackers on each host, each with a different IP address. Concurrent with attacks, we have generated synthetic legitimate traffic that fit our human user models. FRADE was able to quickly and accurately identify bot addresses and blacklist them. Although our attacks lasted for 10 minutes, FRADE was able to eliminate all attack traffic within 1-2 minutes and restore good service quality for legitimate users. Our FRADE modules were tested for up to 16,000 requests per second, from 16,000 different IPs, and blacklisting solution was tested for up to 1,000,000 IP addresses. Neither our modules nor the blacklisting solution imposed any perceptible delay to client request processing.

Figures enclosed with this report show the illustration of FRADE's functionality, and how attack and legitimate traffic are handled by FRADE over time in a 4,000 request per second attack launched by 4,000 sources for over 10 minutes. FRADE takes 2 minutes to identify all the bot addresses at the server and to blacklist them. After the first 2 minutes almost no attack traffic gets served and all legitimate traffic gets served. We also show the time it takes to serve a legitimate request. During the first 2 minutes of attack it may take more than 10 seconds to serve each request. Prior to the attack, and after 2 minutes, the time per request is low - around 0.3 seconds. This shows that FRADE can quickly restore service quality for legitimate users and remove the effect of the ongoing attack.

More information about outcomes of our project can be found at the project Web page: http://steel.isi.edu/projects/frade/

 


Last Modified: 03/10/2017
Modified by: Jelena Mirkovic

For specific questions or comments about this information including the NSF Project Outcomes Report, contact us.