welcome to the big city

Incident Reporting Helps the CERT™ Coordination Center
Keep Pace with a Rapidly Expanding Internet

by Jeffrey J. Carpenter
<jjc@cert.org>

Jeffrey J. Carpenter is a senior Internet security technologist at the CERT Coordination Center (CERT/CC) in the Software Engineering Institute at Carnegie Mellon University, and he is a lead developer of an incident knowledgebase now in the works. Previously he acted as the incident response team leader, managing staff providing technical assistance to Internet sites that have experienced a computer security incident.

The Internet has been around in some form for the past 30 years. Security has always been a concern but has historically been more of an afterthought than a requirement. The history of Internet security can be compared to life in a town. When the town is small, people are likely to know and trust one another, doors are left unlocked, and generally there is little or no crime. However, as the town grows, security and crime become greater concerns.

The Internet of today has grown to something in orders of magnitude greater than the largest of metropolitan cities. Consequently, there is an increase in crime and concern about security. Moreover, dependency on the Internet as a communications infrastructure has increased. At the same time, however, intruder technology has become significantly more sophisticated, and the expertise required to be a successful intruder has decreased. Fortunately, security awareness among system administrators and Internet users has increased, but much work remains.

The CERT Coordination Center Is Born

The Internet was still very much a small city in November 1988, when a Cornell University graduate student let loose the notorious Internet worm that brought down much of the Internet and demonstrated the growing network's susceptibility to attack. Once a group of researchers drawn from government and the academic community successfully contained the worm, the National Computer Security Center (part of the National Security Agency) initiated a series of meetings to discuss how to prevent and respond to such occurrences in the future.

Shortly thereafter, the Defense Advanced Research Projects Agency (DARPA) announced its intention to fund development of the CERT Coordination Center (CERT/CC). DARPA chose the Software Engineering Institute (SEI) on the campus of Carnegie Mellon University, in Pittsburgh, Pennsylvania, as the new center's home. The SEI was charged with establishing the capability to quickly and effectively coord-inate communication among experts during security emergencies in order to prevent future incidents, and with building awareness of security issues in the Internet community at large.

Since its inception in 1988, the CERT/CC has responded to more than 17,800 security incidents that have affected over 235,000 government, academic, and corporate sites. Consequently, the time required to resolve computer security incidents and repair computer-system vulnerabilities has decreased. The resulting incident-response and security-improvement practices developed have led to networked computing systems that are more resistant to attack and less likely to be compromised.

The Incident Reporting Process

Input from the community is critically important to the CERT/CC. The community provides us with the necessary raw data in the form of incident reports, vulnerability reports, alerts from intrusion-detection systems on networks, and discussions with other response teams and experts. Public mailing lists and security Web sites are also monitored.

The majority of this input data arrives in the form of incident reports from system and network administrators. An incident report is a collection of data that has been identified by someone as an attack. The format and transport for an incident report is typically unstructured text sent by email to <cert@cert.org>, or by a telephone call to the CERT Hotline (412 268-7090).

Since the CERT/CC was established, its goals have been identifying Internet security trends, detecting attacks spanning multiple administrative domains, and handling attacks targeted against or affecting the Internet infrastructure. The mechanisms used to accomplish these goals have continued to be developed and refined over time. Data from multiple sources is processed in an attempt to positively identify attacks targeted at, or affecting significant portions of, the Internet infrastructure.

When an incident is reported to the CERT/CC, an analyst works to determine its priority. This involves interpreting, extracting data from, analyzing, and recording information on the basis of the incident report. Part of the analysis that occurs with a new incident report is correlation with data that has been received from all input sources and past incidents. Collections of data representing related attacks are referred to as an incident. One incident can represent something minor, such as a single probe to a single site, or something significant, such as the Melissa virus. Additional analysis involves identifying instances of known methods or signatures of attack. Any attack method identified that cannot be represented by a common, well-known signature is investigated by the incident analyst until the details of the attack can be determined. When novel methods of attack are uncovered, the analyst records a signature for that attack and determines what the likelihood of attack is and what the threat is for the use of the attack on the Internet infrastructure. If the threat exceeds a certain threshold, a CERT Advisory is issued.

On the basis of the information received from the Internet community, critical information about specific threats is disseminated through security alerts, such as CERT Advisories, Incident Notes, Vulnerability Notes, and Vendor-Initiated Bulletins. CERT Advisories address Internet security problems. They offer an explanation of the problem, information that helps determine if a site has the problem, fixes or workarounds, and vendor information. Among the criteria for developing an advisory are the urgency of the problem, potential impact of intruder exploitation, and the existence of a software patch or workaround. CERT Summaries are published as part of our ongoing efforts to disseminate timely information about Internet security issues. The summary is typically published four to six times a year. The primary purpose of the summary is to call attention to the types of attacks being reported to us.

We also publish two Web documents, Incident Notes and Vulnerability Notes, as an informal means for giving the Internet community timely information relating to the security of its sites. Incident Notes describe current intruder activities that have been reported to the CERT/CC incident-response team (<https://www.cert.org/currnet/>). Vulnerability Notes describe weaknesses in Internet-related systems that could be exploited but that do not meet the criteria for advisories.

Lessons learned from incident handling and vulnerability analysis are made available to users of the Internet through a Web site and FTP archive of security information and products. These include answers to frequently asked questions, a security checklist, tech tips for system administrators, security tools such as TCP wrappers, research and technical reports, and a handbook for new computer security incident-response teams (CSIRTs). Members of the Internet community can subscribe to receive advisories by email. Subscription information is available on the CERT Web site: <https://www.cert.org>. At present there are more than 100,000 addresses on the public mail subscriber listing with many of those as mail exploders (a higher-level address that forwards mailings to its own individual subscribers). As a result, it is estimated that CERT mailings reach over half a million addresses.

The most up-to-date information about ongoing attacks can be found on the Current Activity section of our Web site. Other outputs include educational documents and vulnerability alerts to affected vendors.

The Benefits of Incident Reporting

Ultimately, incidents reported to the CERT/CC benefit all parts of the Internet community. In more severe cases, our staff may provide direct assistance in resolving an incident. A minor or seemingly insignificant piece of incident data at a reporting site may represent part of a much larger and more significant attack affecting multiple sites. Receiving data from multiple sources helps us to have a more accurate understanding of the current state of the Internet. This information is of fundamental importance for detecting attacks affecting significant parts of the Internet infrastructure and preventing the spread of such attacks by producing Advisories. We receive many valuable reports from sites that need no specific assistance. These sites let us know about the activity they are seeing or when they see new vulnerabilities or types of attacks. We encourage sites to report information to us even if they do not need assistance, because incident reporting adds to the body of attack knowledge that will find its way back to the community in the form of our advisories and other documents.

In addition to helping detect and scope the significance of attacks, incident reports provide a basis to determine trends and statistics in Internet security. This type of information is compiled regularly in our publications and frequently discussed in public forums such as conferences.

From Reactive to Proactive

Over the years, the CERT/CC has evolved from a handful of technical staff reacting to computer security incidents to a multidisciplinary team of professionals working to prevent future incidents as well as respond to them.

Based on this experience, our research agenda has been structured around several major components, including security evaluation and improvement, simulation of interconnected systems and infrastructures, assisted and automated detection and response techniques, and advanced vulnerability analysis.

In the area of automated incident-detection and response, we continue to work toward automation of incident reporting and automated incident data sharing. Tools are also under development within the program to support the automated upgrade and patching of large heterogeneous networks. In conjunction with the result of vulnerability handling and prioritization, these automated upgrade tools can assist in the protection of large distributed networks such as those found in railway systems, power distribution, and telecommunications.

Keeping Pace with the Threat

The Internet is an environment in which intruders form a well-connected community and use network services to distribute information quickly on how to maliciously exploit vulnerabilities in systems. Intruders dedicate time to developing programs that exploit vulnerabilities and to sharing information. They have their own publications, and they regularly hold conferences that deal specifically with tools and techniques for defeating security measures in networked computer systems.

In contrast, the legitimate, often overworked system administrators frequently find it difficult to take the time and energy from their normal activities to stay current with security and vulnerability information, much less design patches, workarounds, tools, policies, and procedures to protect the computer systems they administer.

In helping the Internet community work together, the CERT/CC and other incident-response teams face policy and management issues that are perhaps even more difficult than the technical issues. Most important, the Internet community needs to work together closely to keep pace with an emerging threat and to ensure that future products and services are able to survive.