Friday, August 24, 2007

Students Changing the World, One Bug at a Time

I am so impressed by the seriousness of purpose our Computer Sciences students bring to this project. In many ways their grasp of what we are doing exceeds that of our journalists. This shouldn’t be thought of as a criticism of our journalists; our programs are quite modest and our students are all under-grads, much more adept at wrestling with cameras, microphones, and dangling participles than they are with theoretical approaches to news and the computer’s place in journalism.

Nonetheless, the questions that these young people have raised (cheered on by their mentor, Dr. Abbot) have profound implications for the practice of journalism.

I fear not being able to communicate the significance of this effort. On one hand, it could be an interesting and challenging way to earn a degree (four of our five students are pursuing graduate degrees, one an undergrad degree). Clearly, what we are trying to do is tame a bit a very wild and unruly beast, but this could be said of any massive corporate software project.

But somehow, news is different.

The product of all our journalistic efforts is knowledge (not simply information; we’re awash in a sea of that over-hyped commodity).

And like scientific or cultural knowledge, journalism influences society in sometimes profound ways. It can, of course, trivialize human activity. We find ourselves hypnotized by celebrity, impressed by nonsense, and unable to pull away from dross.

In a larger sense, we have become slaves to ‘mere novelty’; in fact, I coined the term ‘the tyranny of novelty’ to describe how meaningless drivel sucks the intellectual oxygen out of the air.

But it also remains true that while we may get the media we deserve, journalism remains the solid bedrock upon which societies operate. Rarely will you find a good society with a bad press, and where you find a vibrant, engaging press existing in a bad society.

One of the key ideas with which we are wrestling is how to devise a search algorithm that is not premised on simple popularity. One of the key functions of a vibrant press is to frequently turn away from popularity and conventional wisdom to direct society’s gaze on that which is not merely novel, but rather has profound implications for society.

Conventional search engines are based on the idea that the number of links between sites is a good indicator of what people want to see. In essence, they are big popularity contests. There is a certain egalitarian aspect to this reality: every site, every blog, every portal is measured simply by its popularity.

This popularity can be measured in a number of ways: the total number of links to a site, total number of site visitors, total number of unique visitors, page views, etc.

But all these measures overlook the fact that mere popularity can’t accurately predict what is important for society.

Journalists ignore this popularity at their peril, but surely, they have to look elsewhere as well.

In short, search engines work on measures of quantitative value. This is often at odds with what good journalists do. Often, journalists – the good ones at least – must ignore what is merely novel or popular to find out the real meaning behind trends or developments.

At this point, our search technology can’t make subjective judgments as to what will be or actually is important. We still relay on human brains to look over the haul from search engines to determine what is good and valuable and what is not.

I am not enough of a Luddite to believe that computers won’t some day be able to search more smartly in this context; rather I simply acknowledge that we aren’t there yet.

So one of my conclusions at the beginning of the marriage of computers and news is that it is quite possible our first round of projects will be less successful than we dream.

That would be fine. The process of wrestling with, probing and poking at this old beast is an honorable undertaking, and could very possibly change the way we process civic knowledge.

Friday, August 10, 2007

Broadcastvoice: Pleasant Chastisement

CoolState Vidz

Pleasant Chastisement

In two fairly inspiring conversations this week, colleagues of mine at Cal State LA planted some very interesting ideas in my brain; ideas that may seem routine in the worlds of systems and programming, but ideas nonetheless, which could revolutionize – or at least dramatically change the way we currently make news content.

First, Dr. Chengyu Sun listened ever so politely as I rattled on about our plans for using computer resources to do the job of gathering and evaluating news more thorough and more efficient.

The major topic of our discussion was on the topic of databases. I’m not talking about a couple hundred records of names and addresses in a so-called ‘flat file’, but rather a potentially massive media database that would make available to users vast quantities of photos, audio, video, text, flash animation, sound-slides, and other media content.

As I saw it, there were three broad scenarios available to us:

First, the use of fairly high-end consumer computers (in our case Mac Pro G5s, with dual Intel processors) installed in our department. We are already using this set up to manage some small text/number databases accessible by faculty, students and staff. There would be some obvious storage and bandwidth limitations on this set up, but it could work as a rudimentary arrangement.

The second scenario I envisioned was the purchase of a commercial server; in our case an Apple Xserve dual core Intel Xeon server with up to 3GHz of processing power. This server would be installed on campus server racks and would eliminate our need to manage back-ups, crashes, updates, etc., but would not only cost us a bit of our very limited budget, could potentially pass on band width problems to the campus. This was not a great solution, because there is an existing history of faculty installing servers here that essentially jeopardized the entire campus system. That faculty member was reprimanded and the server shut down. This wasn’t the kind of arrangement we wanted.

The third scenario I proposed was the leasing of server space at commercial sites off campus, which would handle bandwidth problems and could expand with our needs. The problems, of course, were with money. While server space is relatively cheap at the moment, the cost of band width alone could quickly eat up our entire budget.

Dr. Sun endorsed a scenario that we had pursued as a quick fix solution: embedding media players in our site so that we could filter media through our own editorial process, but still take advantage of the commercially existing media databases online, such as You Tube or Google Video (related sites, but not the same thing…). I had (incorrectly) assumed that we would lose control of our ‘branding’ of the page. That is to say, we wanted to feature videos which has passed through an editorial process in our enterprise before posting them online.

Our graduate programmers, Dhaval Joshi and Manish Sharma came up with a very simple and elegant solution, based on the ideas Dr. Sun endorsed (though Dhaval and Manish had started this work as an interim solution prior to Dr. Sun’s recommendation). That page, under construction, not ready for comments, and definitely not open for business is located at:

Dr. Sun was so patient with me. I am a journalist, not a programmer or systems expert. His smiling and kindly insistence that taking advantage of other commercial site’s storage space and band width was a revelation.

Today, meeting with Dr. Russ Abbot, I laid out an over-view of three projects I had spoken with graduate students about recently. It is my hope that these three students will take on these projects, which I will explain in a later post.

While the details of each of those projects are interesting in and of themselves, features of which could have a dramatic effect on how we gather and distribute news content, Dr. Abbot was more concerned about my overall lack of vision on how to integrate the disparate and unruly parts of our enterprise. While I have prided myself on the tireless work I’ve done getting phones installed, replacing old furniture, and setting up infrastructure and manual work-flow systems – sort of rambling on and on, Dr. Abbott patiently listened to my blathering and finally asked me “Yes, but how do all these parts work together?”

In the rush to get a news paper published, to get a web site launched, to produce and develop novel and innovative new news delivery systems, I’d sort of missed the big picture: all of these elements have to work together as a whole, not just in the theoretical or editorial sense, but in the system sense as well.

I’ve got some serious think-time ahead of me. It’s not at all clear that I can come up with answers for any of Dr. Abbott’s questions, but it’s probably a good thing that I am at least asking those questions.

It’s Fiday night. I’m still on campus at 9:00 pm. My head hurts. But in a good way.