From Reddit to Quora, discussion forums can be equal parts informative and daunting. We’ve all fallen down rabbit holes of lengthy threads that are impossible to sift through. Comments can be redundant, off-topic or even inaccurate, but all that content is ultimately still there for us to try and untangle.
Sick of the clutter, a team from MIT’s Computer Science and Artificial Intelligence Laboratory (CSAIL) has developed “Wikum,” a system that helps users construct concise, expandable summaries that make it easier to navigate unruly discussions.
“Right now, every forum member has to go through the same mental labor of squeezing out key points from long threads,” says MIT Professor David Karger, who was senior author on a new paper about Wikum. “If every reader could contribute that mental labor back into the discussion, it would save that time and energy for every future reader, making the conversation more useful for everyone.”
The team tested Wikum against a Google document with tracked changes that aimed to mimic the collaborative editing structure of a wiki. They found that Wikum users completed reading much faster and recalled discussion points more accurately, and that editors made edits 40 percent faster.
Karger wrote the new paper with PhD students Lea Verou and Amy Zhang, who was lead author. The team presented the work last week at ACM’s Conference on Computer-Supported Cooperative Work and Social Computing in Portland, Oregon.
How it works
While wikis can be a good way for people to summarize discussions, they aren’t ideal because users can’t see what’s already been summarized. This makes it difficult to break summarizing down into small steps that can be completed by individual users, because it requires that they spend a lot of energy figuring out what needs to happen next. Meanwhile, forums like Reddit let users “upvote” the best answers or comments, but lack contextual summaries that help readers get detailed overviews of discussions.
Wikum bridges the gap between forums and wikis by letting users work in small doses to refine a discussion’s main points, and giving readers an overall “map” of the conversation.
Readers can import discussions from places such as Disqus, a commenting platform used for publishers like The Atlantic. Then, once users create a summary, readers can examine the text and decide if they want to expand the topic to read more. The system uses color-coded “summary trees” that show topics at different levels of depth and lets readers jump between original comments and summaries.
“Our aim is to harness collaborative summarization to save the reader from sifting through hours of unorganized forums to find the content they want,” says Zhang.
Editors have a similar interface to readers, with options for marking comments as unimportant, summarizing and grouping comments and replies, and citing or quoting key parts of comments. The researchers used an automatic highlighting algorithm that picks out important sentences for editors, who are limited to 250-word summaries.
"Discussion is a timeless medium—some of what we do today on Reddit looks like discussion on USENET in the early 1990s,” says Amy Bruckman, a professor at the Georgia Tech who was not involved with the paper. “Innovating in this space is hard, and that’s what makes Wikum impressive. They’ve made the first serious advance in helping people quickly understand long conversations. Their approach to recursive crowd sourced summarization has intriguing potential."
In the future, the goal would be to have a diverse group of contributors organize and moderate discussions. The team plans to explore how to best motivate users to summarize content.
The team is also collaborating with the Wikimedia Foundation to deploy Wikum with their editors and further improve the system. Specifically, they’re seeing how editors can use it to summarize large discussions about important changes to Wikipedia pages like Requests for Comment (RFCs), in order to more easily make final decisions.
They also want to examine using Wikum in a chat structure to see if real-time summaries could improve digital messaging systems like the types used on apps such as Slack and Trello.
“At a time of great enthusiasm for machine learning and data mining, it's worth remembering that human beings can be excellent processors of information if they are given the right tools to leverage their intelligence,” Karger says.
