Students Changing the World, One Bug at a Time

I am so impressed by the seriousness of purpose our Computer Sciences students bring to this project. In many ways their grasp of what we are doing exceeds that of our journalists. This shouldn’t be thought of as a criticism of our journalists; our programs are quite modest and our students are all under-grads, much more adept at wrestling with cameras, microphones, and dangling participles than they are with theoretical approaches to news and the computer’s place in journalism.

Nonetheless, the questions that these young people have raised (cheered on by their mentor, Dr. Abbot) have profound implications for the practice of journalism.

I fear not being able to communicate the significance of this effort. On one hand, it could be an interesting and challenging way to earn a degree (four of our five students are pursuing graduate degrees, one an undergrad degree). Clearly, what we are trying to do is tame a bit a very wild and unruly beast, but this could be said of any massive corporate software project.

But somehow, news is different.

The product of all our journalistic efforts is knowledge (not simply information; we’re awash in a sea of that over-hyped commodity).

And like scientific or cultural knowledge, journalism influences society in sometimes profound ways. It can, of course, trivialize human activity. We find ourselves hypnotized by celebrity, impressed by nonsense, and unable to pull away from dross.

In a larger sense, we have become slaves to ‘mere novelty’; in fact, I coined the term ‘the tyranny of novelty’ to describe how meaningless drivel sucks the intellectual oxygen out of the air.

But it also remains true that while we may get the media we deserve, journalism remains the solid bedrock upon which societies operate. Rarely will you find a good society with a bad press, and where you find a vibrant, engaging press existing in a bad society.


One of the key ideas with which we are wrestling is how to devise a search algorithm that is not premised on simple popularity. One of the key functions of a vibrant press is to frequently turn away from popularity and conventional wisdom to direct society’s gaze on that which is not merely novel, but rather has profound implications for society.

Conventional search engines are based on the idea that the number of links between sites is a good indicator of what people want to see. In essence, they are big popularity contests. There is a certain egalitarian aspect to this reality: every site, every blog, every portal is measured simply by its popularity.

This popularity can be measured in a number of ways: the total number of links to a site, total number of site visitors, total number of unique visitors, page views, etc.

But all these measures overlook the fact that mere popularity can’t accurately predict what is important for society.

Journalists ignore this popularity at their peril, but surely, they have to look elsewhere as well.

In short, search engines work on measures of quantitative value. This is often at odds with what good journalists do. Often, journalists – the good ones at least – must ignore what is merely novel or popular to find out the real meaning behind trends or developments.

At this point, our search technology can’t make subjective judgments as to what will be or actually is important. We still relay on human brains to look over the haul from search engines to determine what is good and valuable and what is not.

I am not enough of a Luddite to believe that computers won’t some day be able to search more smartly in this context; rather I simply acknowledge that we aren’t there yet.

So one of my conclusions at the beginning of the marriage of computers and news is that it is quite possible our first round of projects will be less successful than we dream.

That would be fine. The process of wrestling with, probing and poking at this old beast is an honorable undertaking, and could very possibly change the way we process civic knowledge.