jodischneider.com/blog

What if newspapers published not just stories but databases? Dan Conover’s vision for the future of newspapers is inspired in part by his first reporting job, for NATO:

When we spotted something interesting, we recorded it in a highly structured way that could be accurately and quickly communicated over a two-way radio, to be transcribed by specialists at our border camp and relayed to intelligence analysts in Brussells.

The story, says Conover, is only one aspect of reporting. The other part? Gathering structured metadata, which could be stored in a database—or expressed as linked data.¹

Newspapers already have classification systems and professional taxonomists. The New York Times’ classifications system, in use since 1851, now aggregates stories from the archives in Times Topics, a website and API.²

What if, in addition to these classifications, each story had even more structured metadata?
Capturing metadata ranges from automatic to manual. Some automatic capture is already standard (timestamps) or could be (saving GPS coordinates from a photo or storing timestamps), and some information needing manual capture (like the number of alarms of a fire) is already reported.

Dan compares the “old way” with his “new way”:

The old way:

Dan the reporter covers a house fire in 2005. He gives the street address, the date and time, who was victimized, who put it out, how extensive the fire was and what investigators think might have caused it. He files the story, sits with an editor as it’s reviewed, then goes home. Later, he takes a phone call from another editor. This editor wants to know the value of the property damaged in the fire, but nobody has done that estimate yet, so the editor adds a statement to that effect. The story is published and stored in an electronic archive, where it is searchable by keyword.

The new way:

Dan the reporter covers a house fire in 2010. In addition to a street address, he records a six-digit grid coordinate that isn’t intended for publication. His word-processing program captures the date and time he writes in his story and converts it to a Zulu time signature, which is also appended to the file.

As he records the names of the victimized and the departments involved in putting out the fire, he highlights each first reference for computer comparison. If the proper name he highlights has never been mentioned by the organization, Dan’s newswriting word processor prompts him to compare the subject to a list of near-matches and either associate the name with an existing digital file or approve the creation of a new one.

When Dan codes the story subject as “fire,” his word processor gives him a new series of fields to complete. How many alarms? Official cause? Forest fire (y/n)? Official damage estimate? Addresses of other properties damaged by the fire? And so on. Every answer he can’t provide is coded “Pending.”

Later, Dan sits with an editor as his story is reviewed, but a second editor decides not to call him at home because he sees the answer to the damage-estimate question in the file’s metadata. The story is published and archived electronically, along with extensive metadata that now exists in a relational database. New information (the name of victims, for instance) automatically generates new files, which are retained by the news organization’s database but not published.

And those information fields Dan coded as “Pending?” Dan and his editors will be prompted to provide that structured information later — and the prompting will continue until the data set is completed.

- Dan Conover in The “Lack of Vision” thing? Well, here’s a hopeful vision for you

And that data set? It might even be saleable, even though each individual story had perhaps been given away for free. Dan highlights some possibilities, and entire industries have grown around repackaging free and non-free data (e.g. U.S. Census data, phone book data). I think of mashups such as Everyblock and hyperlocal news sites like outside.in.

1. Some news organizations, like the New York Times (see Linked Open Data) and the BBC (overview, tech blog) are already embracing linked data. [↩]
2. I delved into Times Topics’ taxonomy and vocabulary in an earlier post. [↩]

Will Google Books create a monopoly? Some¹ people think² so. Brin claims it won’t:

If Google Books is successful, others will follow. And they will have an easier path: this agreement creates a books rights registry that will encourage rights holders to come forward and will provide a convenient way for other projects to obtain permissions.

-Sergey Brin, New York Times, A Library To Last Forever

Brin is wrong: the proposed Google Books settlement will not smooth the way for other digitization projects. It creates a red carpet for Google while leaving everyone else at risk of copyright infringement.

The safe harbor provisions apply only to Google. Anyone else who wants to use one of these books would face the draconian penalties of statutory copyright infringement if it turned out the book was actually still copyrighted. Even with all this effort, one will not be able to say with certainty that a book is in the public domain. To do that would require a legislative change – and not a negotiated settlement.

– Peter Hirtle, LibraryLawBlog: The Google Book Settlement and the Public Domain.

Monopoly is not the only risk. Others include³ reader privacy, access to culture, suitability for bulk and some research users (metadata, etc.). Too bad Brin isn’t acknowledging that!

Don’t know what all the fuss is with Google Books and the proposed settlement? Wired has a good outline from April.

1. “Several European nations, including France and Germany, have expressed concern that the proposed settlement gives Google a monopoly in content. Since the settlement was the result of a class action against Google, it applies only to Google. Other companies would not be free to digitise books under the same terms.” (bolding mine) – Nigel Kendall, Times (UK) Online, Google Book Search: why it matters [↩]
2. “Google’s five-year head start and its relationships with libraries and publishers give it an effective monopoly: No competitor will be able to come after it on the same scale. Nor is technology going to lower the cost of entry. Scanning will always be an expensive, labor-intensive project.” (bolding mine) – Geoffrey Nunberg, Chronicle of Higher Education, Google’s Book Search: A Disaster for Scholars (pardon the paywall) [↩]
3. Of course there are lots of benefits, too! [↩]

I love it when paper and digital formats are both used for what they do best. Like the Incidental:
“The Incidental is [a] feedback loop made out of paper and human interactions – timebound, situated and circulating in a place.” [Schulz and Webb]

annotated incidental 4/25/09

“Over in Milan at the Salone di Mobile they’ve created a thing called The Incidental. It’s like a guide to the event but it’s user generated and a new one is printed every day. When I say user generated, I mean that literally. People grab the current day’s copy and scribble on it. So they annotate the map with their personal notes and recommendations. Each day the team collect the scribbled on ones, scan them in and print an amalgamated version out again. You have to see it, to get it. But it’s great to see someone doing something exciting with ‘almost instant’ printing and for a real event and a real client too.

The actual paper is beautiful and very exciting. It has a fabulous energy that has successfully migrated from the making of the thing to the actual thing. Which is also brilliant and rare. [Ben Terrett as quoted by Schulz and Webb]

The Incidental was created at and for Milan’s furniture/design fair with funding by The British Council.

Last night I popped into Second Life for a poster session. JCDL 2009 is going on in Austin this week, and several of the posters were on display in the Digital Preserve region of SL. Chris Beer asked for some screenshots.

Here’s the whole poster space from outside. (Click each image for the ginormous full-size screenshot.)

My avatar (TR Telling) is in a bright orange UIUC GSLIS T-shirt, thanks to a class tour Richard Urban led last year. With a closer look, you can spot the screen that was used to project MinuteMadness.

Here are two posters, “Finding Centuries-Old Hyperlinks” and “Toward Automatic Generation of Image-Text Document Surrogates to Optimize Cognition”.
Poster numbers were used for the best poster competition, I believe.

Large text-sizes really help viewing from afar; deft users can get a closer view with ‘mouse look’. I took a second screenshot of the “Finding Centuries-Old Hyperlinks” poster since it was my favorite. Xiaoyue (Elaine) Wang and Eamonn Keogh suggest cross-referencing manuscript pages using icon similarity.
Handouts could be really useful for a SL poster session — I had to settle for taking screenshots. Clicking on the poster could give a copy of the poster, which could include links to more information. A mailbox could facilitate sending messages to the presenters.

One presenter ‘attended’ from New York. Several people are gathered around her poster, which generated a lot of discussion.

In the left corner you can see one of the more visually striking posters, a study of LIS students’ impressions of the Kindle, after using it for something like 3 weeks.

To the right of the entrance is a sign that says “What did you think?”, which linked to a comment form to be completed on the Web. I succeeded at that box, but wasn’t able to figure out how to submit a second, in-world comment form.

My avatar is just stepping down from a rotating lazy-susan which held a striking comment box. Getting a comment form and filling it out was straightforward. However, dragging and dropping the form back onto the box, as suggested, didn’t work for me.

I had several interesting conversations, most notably a chat outside in the Poster Garden with Javier Velasco Martin who helped build and furnish the Preserve. Ed Fox was easily identifiable: his avatar’s first name is EdFox. For social gatherings, handles are useful, but for professional gatherings it can be reassuring to know who you’re talking with.

Here’s one last look at the dome from the outside. I love the bright aqua JCDL lettering. And, what trip to Second Life would be complete without some flying?
With a closer look, you can see the large comment box in the center of the dome.

A call to action from BYU English professor Gideon Burton: Stop intellectual apartheid!

Let me illustrate how academic institutions enforce Intellectual Apartheid through a simple experiment you can perform right now. Let’s say that you are researching lingering effects of South Africa’s apartheid and you discovered (as I did using Google Scholar) a recent article, “Fantasmatic Transactions: On the Persistence of Apartheid Ideology” (published in Subjectivity in July, 2008 by D. Hook). Now for the experiment: click on this link to the full text of the article.

One of two things just occurred. Either you just gained immediate access to a PDF version of the full article; or, more likely, an authentication window popped up requesting your login credentials. It turns out that Palgrave-Macmillan publishes Subjectivity, and through their website one can get access to this article for a mere $30. Alternatively, one may subscribe to the journal for $503 per year.

You really don’t need to go to the developing world to recognize that advanced knowledge is a big club with stiff entrance fees. Even middle class Americans will think twice before throwing down $30 for a scholarly article. How likely will this knowledge ever reach scholars in Mexico or India? And just how broadly can the editors of Subjectivity expect it to reach when subscribing costs $503/year?

Gideon also gives suggestions for scholars, librarians, and administrators.

via Cameron Neylon on friendfeed

Clay Shirky writes of newspapers in an age of revolution: 15 years of anticipated problems* viewed optimistically, patched with one-size-fits-all solutions. Those solutions don’t attack the main issue: “the core problem publishing solves — the incredible difficulty, complexity, and expense of making something available to the public — has stopped being a problem.” It’s a revolution, he says, drawing on the print revolution of the early 1400s, and no one knows what will happen.

The old stuff gets broken faster than the new stuff is put in its place. The importance of any given experiment isn’t apparent at the moment it appears; big changes stall, small changes spread. Even the revolutionaries can’t predict what will happen. Agreements on all sides that core institutions must be protected are rendered meaningless by the very people doing the agreeing. (Luther and the Church both insisted, for years, that whatever else happened, no one was talking about a schism.) Ancient social bargains, once disrupted, can neither be mended nor quickly replaced, since any such bargain takes decades to solidify.

And so it is today. When someone demands to know how we are going to replace newspapers, they are really demanding to be told that we are not living through a revolution. They are demanding to be told that old systems won’t break before new systems are in place. They are demanding to be told that ancient social bargains aren’t in peril, that core institutions will be spared, that new methods of spreading information will improve previous practice rather than upending it. They are demanding to be lied to.

There are fewer and fewer people who can convincingly tell such a lie.

Shirky sees the future of journalism as “overlapping special cases” with a variety of funding and business models. It’s a time for experimentation, and while he sees failure and risk, he has hope, too:

Many of these models will fail. No one experiment is going to replace what we are now losing with the demise of news on paper, but over time, the collection of new experiments that do work might give us the reporting we need.

Society needs reporting, not newspapers. That need is real, and worth restating:

Society doesn’t need newspapers. What we need is journalism. For a century, the imperatives to strengthen journalism and to strengthen newspapers have been so tightly wound as to be indistinguishable. That’s been a fine accident to have, but when that accident stops, as it is stopping before our eyes, we’re going to need lots of other ways to strengthen journalism instead.

When we shift our attention from ’save newspapers’ to ’save society’, the imperative changes from ‘preserve the current institutions’ to ‘do whatever works.’ And what works today isn’t the same as what used to work.

Go read the whole essay, then let it stew with other thoughts on the future of publishing.

*Circa 1993: “When a 14 year old kid can blow up your business in his spare time, not because he hates you but because he loves you, then you got a problem.”

Via John Dupuis’ post in Confessions of a Science Librarian.

Yesterday, Steven Berlin Johnson spoke at SXSW about the information ecosystem and the future of news. Fortunately, for those of us playing at home, he blogged a transcript.

Johnson adds international and war reporting to investigative reporting as the areas at risk due to the implosion of news funding. Johnson envisions a bright future in other areas, citing a well-developed information ecosystem in technology, and comparing coverage of the 2008 and 1992 U.S. Presidential elections.

Extending his ecosystem metaphor, Johnson introduces technology journalism as the “old-growth forest” of web journalism. Ecologists use (real-world) old growth “to research natural ecosystems”, so by extension, Johnson says, “it’s much more instructive to anticipate the future of investigative journalism by looking at the past of technology journalism”. While this argument holds no water, it’s certainly suggestive.

in the long run, we’re going to look back at many facets of old media and realize that we were living in a desert disguised as a rain forest. … most of what we care about in our local experience lives in the long tail. We’ve never thought of it as a failing of the newspaper that its metro section didn’t report on a deli closing, because it wasn’t even conceivable that a big centralized paper could cover an event with such a small radius of interest.

But of course, that’s what the web can do. … As we get better at organizing all that content – both by selecting the best of it, and by sorting it geographically – our standards about what constitutes good local coverage are going to improve.

As Johnson envisions, “Five years from now, if someone gets mugged within a half mile of my house, and I don’t get an email alert about it within three hours, it will be a sign that something is broken.”.
This is all by way of introduction to his new company, outside.in, which provides geographic search and alerting.

Johnson concludes, in part, by examining the filtering problem, and turning it into an opportunity:

Now there’s one objection to this ecosystems view of news that I take very seriously. It is far more complicated to navigate this new world than it is to sit down with your morning paper. There are vastly more options to choose from, and of course, there’s more noise now. For every Ars Technica there are a dozen lame rumor sites that just make things up with no accountability whatsoever. I’m confident that I get far more useful information from the new ecosystem than I did from traditional media along fifteen years ago, but I pride myself on being a very savvy information navigator. Can we expect the general public to navigate the new ecosystem with the same skill and discretion?

Johnson expects (future) newspapers to function as filters, aiding the public in getting the news:

Information Ecosystem, as envisioned by Steven Berlin Johnson

Johnson does not address who’s going to pay for the filtering. He’s ready for a new model, but leaves that to the industry to discover for itself. “Measured by pure audience interest, newspapers have never been more relevant.” When he acknowledges the short-term pain of the newspaper industry today, he worries:

we’re going to spend so much time trying to figure out how to keep the old model on life support that we won’t be able to help invent a new model that actually might work better for everyone. The old growth forest won’t just magically grow on its own, of course, and no doubt there will be false starts and complications along the way.

The entire transcript is well worth a read.

Via Steven Johnson on twitter.

Timothy Burke is my new hero. The death* of newspapers, he says, is a problem mainly because somebody’s got to pay for investigative reporting:

We don’t need newspapers to have film criticism or editorial commentary or consumer analysis of automobiles or comic strips or want ads or public records. It might be that existing online provision of those kinds of information could use serious improvement or has issues of its own. It might be that older audiences don’t know where to find some of that information, or have trouble consuming it in its online form. But there’s nothing that makes published newspapers or radio programming inherently superior at providing any of those functions, and arguably many things that make them quite inferior to the potential usefulness of online media. So throw the columnists and the reviewers and the lifestyle reporters off the newspaper liferaft.

So it comes down to independent, sustained investigation of public affairs. The argument that online media cannot provide this function comes down to money…

Burke gives more details and examples, and calls for new funding models, including philanthropic and/or foundation money. He concludes that the “The end of the newspaper model of the last century doesn’t have to be the end of independent investigative reporting.”

Go read the whole thing.
*It seems like death and rebirth, to me, especially with some major newspapers reinventing themselves online. But that’s another matter.

Burke first came to my attention last year, from a talk he gave to the LC Working Group on the Future of Bibliographic Control at March’s meeting on the Users and Uses of Bibliographic Data. Burke represented and reflected upon the user perspective, as an academic who searches catalogs outside his area of expertise.

Via John Dupuis’s friendfeed.