What is all of this?

Global aggregation systems are central repositories where internet users can upload their own computer logs. The logs are analyzed by the system and examined for possible attacks that occurred on the user's system. This alerts the user that someone has attempted to attack their computer and also alerts other internet users of the particular type of attacks occurring on a global internet scale. So in addition to the user being alerted to the attack as they would by a personal Intrusion Detection System or firewall, a large community is notified of the attack so that they can be better prepared. Other features offered by the systems varies. For example, some systems allow sending reports to the ISP of the offending attacker.

Who are the big names?

There are multiple companies that have global aggregation systems. None of the system implementations are identical, however the purpose of the system in each case is similar. Reviewed below are 3 major players. Each section includes their primary goal, how log data is collected, availablility of aggregated/analyzed data, examples of the type of information found, and information about their privacy policy.

DShield

Purpose: "DShield.org is an attempt to collect data about cracker activity from all over the internet. This data will be cataloged and summarized. It can be used to discover trends in activity and prepare better firewall rules."

Impression: DShield seems like an honest, "grassroots" attempt to help improve the security and awareness of internet users. The work is licensed under Creative Commons Attribution-NonCommercial-ShareAlike 2.5 License, which means the work can be freely distributed and make derivative works as long as credit is given to the source, it is not used for commercial purposes, and derivative works must also be licensed in the same way.

Collecting Data: Data collection is done in numerous ways. Firewall logs from common firewalls such as ZoneAlarm, Symantec Personal Firewall, Windows XP Internet Connection Firewall, IPFW, and others can be used. DShield also recommends a collection of clients/plugins that support submitting information to DShield automatically (example: ZoneLog), however logs can be submitted manually through their web interface. Additionally, some router configurations (those with the Kiwi Syslog Daemon) can also be used to collect data and submit reports. Of the systems reviewed, DShield has the most flexible system to submit logs for analysis.

Data Availability: For an unregistered user, data available is for the most part general. Information given is the most popularly attacked port, most "malicious" internet users (those who have been shown up in the most attacks), and other summary statistics. For registered users, more detailed information can be obtained about threats through public mailing lists.

Example Data: Summary statistics for the amount of threats detected for a given IP address (malicious attacker), or targeted port/service are two examples of data given. For example, information regarding win-rpc can be found here.

Privacy: System allows users to submit data "anonymously", and data is made available to public at no charge. Reports and searchable data attempts to mask the IP address of the person being attacked. Other information collection (such as website cookies) are done for website only--not for any 3rd parties. (More information here)

Other Cool Stuff: Has a "FightBack" feature for users who submit data. This forms a message to the ISP of the attacker with evidence collected from system logs for the purposes of reporting the malicious user. (More Information)

MyNetWatchMan

Purpose: "myNetWatchman collects, analyzes and reports malicious access attempts to ISPs, who can then take action against the offending machines." In essence, this seems like a similar purpose to DShield, except it is slanted more towards reporting. A good overview Vision can be found on their website.

Impression: Part of the time I tried to access the site, it failed to load. When the site was available, a great deal of information was available to the public, without needing to register. Things such as Top Port Targets, recent attacks, and incidents by ISP can be browsed/searched (these in particular were very cool). Additionally, specific IP addresses can be searched to see if there are any records in the database relating to that IP (as an attacker).

Collecting Data: Data collection is done exclusively through a program made for the myNetWatchMan service. This is a small client application that monitors system firewall logs and periodically sends data to myNetWatchMan servers for analysis. Analysis is done on the server end so that similar reports can be compared and the likelihood of false positives occurring is diminished. When attacks are reported, escalation reports are sent to the offending ISPs based on the severity of the attacks detected.

Data Availability: Data is available to registered and non-registered users. A variety of data is available, for example, you can query specific IP addresses to see what sort of previous activity has been recorded for that address, or see reports for most attacked ports (useful for tracking outbreaks) and recent ISP reports/incidents. Data from the last 7 days can be seen on their website.

Privacy: Data can be seen by anyone who "accesses their web based reporting system."

Other Cool Stuff: You can view the result of incident reports sent to ISPs. For example, many ISPs will reply to the incident report sent that they are investigating/have investigated the issue and have dealt with it appropriately. I think this is neat how this server empowers a user a little bit more than they would otherwise by helping them track down malicious users. Obviously, not all ISPs will be cooperative of the reports sent.

Symantec DeepSight

Purpose: "...a free service that gives you the ability to track and manage attacks on your computer. Analyzer automatically correlates attacks from various Firewall and network based Intrusion Detection Systems, giving you a comprehensive view of your computer or general network."

Impression: This is the most closed website of the 3 reviewed. No data other than very generic data such as "internet threat level", number of "attacking IPs" and number of "Events" are shown to unregistered users. This system like MNWM requires registration and installation of a custom program for the purposes of uploading data to the DeepSight servers. However, this service seems the most professional of the three, offering the cleanest website, and most variety of features. For example, the following list is given:

  • Report incidents. We look up the appropriate contacts for the offending organization and their upstream provider, allow you to select which incidents you wish to report, and draft a report for you with all the pertinent information.
  • Access descriptions about what the attack was that your firewall or IDS spotted. This includes links into the Bugtraq database where appropriate, as well as articles and exploit code so you can see if the compromise was successful or not.
  • See how many other DeepSight users your attacker has attacked. This can help you determine whether or not you are being targeted for individual attack, in case that factors into your decision on whether to report or not.
  • Track your incidents through our system. You can keep track of which attacks a particular IP has used against you.
  • Correlate reports from different firewall and IDS brands. This is especially helpful when you utilize more than one type of firewall or IDS.

Collecting Data: Data collected is parsed out from logs created on the local machine by a variety of compatible firewalls (including firewalls other than Symantec's own). The DeepSight system has essentially 2 distinct parts, data harvesting (from local logs) and processing (on Symantec's servers). After processing is done, information is sent back to the client computer with any information regarding recent attacks on their computer.

Data Availability: Data is available only to registered members. I am unsure how much "global" data is available, but there is a wide variety of feedback given to a user regarding their own logs submitted to DeepSight.

Example Data: Unable to access data without registration.

Privacy: Symantec has taken actions to limit the risk of storing person firewall logs and such on a network location. For example, account information and IDS logs are stored in separate locations. A full privacy statement can be found here. In general, the privacy seemed pretty good--if you trust Symantec.

What have these companies found?

Generally all of these companies have provided two types of information to the public--conglomerate data and incident specific data.

  • Conglomerate data is generated from many users submitting data of their own personal logs which may contain information about attacks carried out on their computers. This type of data gives users a general feel for wide-scale changes/attacks that could be happening in certain locations (or widespread) on the internet.
  • Incident specific data is generated from knowing a lot about what is going on in a larger scale. For example, if a lot of reports have recently been published of unnecessary access to a specific port, that could be indication of an outbreak of a virus or other type of malware.