Friday, October 10, 2008

Information Pathways in Social Networks

The first talk presented in the social network session of KDD 2008 was for an interesting paper by G. Kossinets, J. Kleinberg, and D. Watts titled The Structure of Information Pathways in a Social Communication Network (PDF). Although I was not at KDD I was able to watch it online at videolectures.net.
Kleinberg, the presenter, made some interesting observations having to do with our "rhythmic" everyday conversations. The approach to analyzing communication within these social networks is focused on the frequency of correspondence, rather than the content conveyed.

They measure "distance" between individuals by measuring the minimum time required for information to pass from one node to another. A methodology based on Lamport's work and vector clocks in the area of distributed computing.

Using this metric they are able to filter a busy network (one having edges for all communication packets) in a simplified network that contains only the edges that are minimum-delay paths between a pair of nodes. They call this simplified network view the network backbone. Below is an example of such a network (along with the caption) taken from the paper.
The nodes further outside of the center of the graph are more "out-of-date" with respect to node v, since they communicate less frequently.

I found the approach to be novel and useful. As with nearly any analysis technique, caution should be used in selecting the time-period and group size to be studied. Recency and frequency issues come into play as correspondence is aggregated. However, this pursuit offers another approach for more fully understanding information flow.

Originally published by Matt on his blog at: http://dmine.blogspot.com

Thursday, October 9, 2008

AMIA Competition Finalists!

Jun, Yao and Matt participated in the 2008 Data Mining Competition: Discovering Knowledge in NHANES Data, sponsored by the AMIA Knowledge Discovery and Data Mining Working Group, and were selected as one of the finalists by the judging panel. They will be presenting their results in a dedicated session of the AMIA Annual Symposium in Washington, DC, in November 2008. Congratulations!