Semantic Web: The Missing Pieces

Kate Ray put together an excellent short film (15 min) on the semantic web. See “Web 3.0″ on her blog. The topics covered are spot on, and the people interviewed are thought leaders in field, including Clay Shirky, David Weinberger and Tim Berners-Lee.

The arguments closely mirror a recent presentation I gave at the IA Summit in Warsaw called “The Navigation Layer: Making Sense of it All.” It was comforting for me to hear others making these same arguments, for one. But there seemed to be a big piece of the puzzle missing in Kate’s Web 3.0 video, which I explain at the end of this post.

Below you’ll find a correlation of themes from the Web 3.0 video and my presentation at the Polish IA Summit:

WEB 3.0 VIDEO: “The core problem is that our ability to create information has far exceeded our ability to manage it.”

MY PRESENTATION: Yup – information overload is the primary challenge we face online. What’s more, to paraphase David Weinberger, the solution to too much information is to add more information; namely, metadata. I then identify three sources of explicit metadata: expert created (e.g., controlled vocabularly), algorithmically created (e.g., entity extraction), and user created (e.g, tagging)

But metadata alone isn’t helpful. It needs some kind of structure.

WEB 3.0 VIDEO: “What if I could start to put things together in all that flow of information? In order to do that you need some sort of structure.”

MY PRESENTATION: Just as there are three types of metadata, there are three types of structures: expert created (e.g., a taxonomy or thesaurus), algrithimically created (e.g., tag cloud), and user created (e.g., social groups on bookmarking sites).

Which types of metadata and structures are best? I strongly believe it’s not an either/or question.

WEB 3.0 VIDEO: Are ontologies needed for the semantic web? Many of the experts in the field believe so. Others dissent. David Weinberger: “We are always going to be filtering the filters that filter our filters that filter our filters.”

MY PRESENTATION: We will need ALL types of metadata and all types of structures to deal with information overload. Solutions may mix top-down ontologies with algorithmically created metadata that users can tag as they see fit. The filters needed can (and probably) will be a mix of types of metadata and types of structures. See my article in the ASIST Bulletin, “Navigating The Long Tail,” for more on this.

It’s all about relationships:

WEB 3.0 VIDEO: “With semantic web technologies, the links mean something.” “If I have enough of those relationships it builds context. And context is what it’s all about.”

MY PRESENTATION: Amen. We need to move beyond just findability. We need to help people make sense what they find. To do this, you need to expose relationships between metadata in the navigation layer. Conext is critical here.

WEB 3.0 VIDEO: Clay Shirky: “Content, as it turns out, is not king”…”How do you filter things to create more value than you currently get?”… “If I were going to start a news business tomorrow, I’d start a news business designed to produce not one new bit of news. But instead, to aggregate news for individuals in ways that matter to them.”
John Hebeler: Google doesn’t scale to meet the challenges of trillions and trillions of information objects on the web. Instead, “you have to do all of the integration in your head.”

MY PRESENTATION: I show a screen shot of the LexisNexis Analytics Dashboard, a producted I helped design. This solution provides a media analyst or a reputation manager, for instance, the big picture of what’s happening in the news without having to read one news article. It aggregates metadata and presents it in charts and graphs that show relationships and patterns. Of course, you can link through to the content if needed, but very often it’s the picture of aggregated metadata that’s most insightful. So the curse of information overload, or more specifically metadata overload, becomes a blessing if you can harness the patterns it shows in a way that’s ready for human consumption.

WEB 3.o VIDEO: Clay Shirky: “Does the world make sense, or do we make sense of the world?”

MY PRESENTATION: We make sense of the world. Hence, sense making as a field of inquiry is absolutely required.

The Missing Pieces

The video has an overall techno-philosophical tone to it. What’s missing from the core of the discussion are human factors, I believe. In a previous post, I point to a quote from Ora Lassila, semantic web guru. He says:

After 10+ years of work into various aspects of the Semantic Web and its constituent technologies, I am now fully convinced (read: no longer in denial) that most of the remaining challenges to realize the Semantic Web vision have nothing to do with the underlying technologies involving data, ontologies, reasoning, etc. Instead, it all comes down to user interfaces and usability. Somehow, I repeatedly run into a situation where some use of Semantic Web technologies that would make a nice end-user application is “blocked” by the fact that the user interface is the real challenge.

How true.

WEB 3.0 VIDEO: “There’s a massive amount of potential, but no real tools to harness it.”

MY PRESENTATION: Beyond the technical tools that will harvest data and metadata, we need interfaces into the data that a.) mortal human beings can easily use and b.) help people make sense of the data they are viewing.

In my presentation I then point to four key challenges the design of interfaces for semantic technologies face:

• Representation – How information is displayed affects how it’s found and understood.
• Interaction – The ability to interact and manipulate information is important in sense making, particularly in digital environments.
• Context – The topical and thematic structures applied to metadata define scope and the boundaries between domains and niche markets.
• Time – Showing trends over time can in many situations provide the most meaningful insight.

So, it seems to me we’ve bashed around the technical and philosophical aspects of the semantic web long enough. I say until we can figure out how the promise of the semantic web can help average people in the daily lives, we need to shift focus to the human factors involved.

This is where information architects can potentially help. Unfortunately, not only do I not see different groups reaching out to each other across the aisle, I don’t see IAs moving towards solutions for the semantic web. Instead, the field is drifting off in the direction of genric user experience design (which is also important, but perhaps too broad to directly address the challenges of the semantic web and sense making). Sense making is an area of interest that has been around for decades. It’s had recent growth in interessant with things like sense making workshops at the CHI conferences.

To reiterate a point previously made:

The success of next-generation information systems depends much more on human factors than on more sophisticated technologies.

But beyond the human factors, it’s also not clear what the business value of the promise of the semantic web is. So in addition I also see a need to shift focus to business aspects of semantic technologies on the wbe.

In my opinion, the missing pieces in the current discussions of the semantic web and semantic technologies are twofold: 1.) users and 2.) business value:

The general semantic web discussion needs a shift in focus to the design of user interfaces that help people make sense of massive amounts of aggregated metadata while at the same time bringing value to businesses.

This is why I find projects like Pivot so interesting, which I mentioned in a previous post. The creators talk about joy and information experience, rather than data and technology.