Have you ever bought a book from Amazon.com that was recommended by another Amazon customer you’ve never met using their automatic referral system? If you have succumbed to temptation and purchased a book based on what other readers have enjoyed, then you’ll understand the value of the article ranking and readers choice feature in PressDisplay.com.
About a year ago, the PressDisplay.com product team thought it would beneficial to our readers to know what articles in newspapers were the most interesting to other readers. So we started to experiment with user-driven article ranking.
The challenge was how to calculate the value of a story on a site which aggregates multiple sources and multiple audiences. How do you measure “interesting” and who decides what’s interesting – readers or editors or both?
So we started by devising an algorithm to determine “top stories” from an editor’s perspective. Some of the attributes used to determine if a story was an “editors’ pick” included: where the article was located (e.g. front page or front of section), the length of the story, the number, size and color of pictures, title font size and the number of sources that covered the story.
Next we experimented with different methods of evaluating how readers ranked articles. At first, we considered measuring click-throughs, but quickly abandoned that idea. Stories originally placed at the top of the home page naturally received more hits, regardless of their quality. Enticing titles tweaked the curiosity of readers, but people often clicked on interesting titles and then closed the story when they discovered the content wasn’t what they were expecting. No, clicks didn’t cut it, so then we looked at voting as an option.
Certainly voting is a popular activity on the internet on sites like Digg or YouTube, but the problem with voting is that 1) few visitors actually vote and 2) often people vote based on their attitude about the subject of the story, not on the quality of the journalistic work.
So after eliminating click counting and voting, we decided that the best way to determine a reader’s interest in a story was to keep track of how well that article retained the interest of readers on a consistent basis. What is the likelihood of a new person who opens that article will continue to read it?
We started keeping track of the reading times for all articles. We even measured which parts of a page in a newspaper received the most attention – e.g. the text of articles, related pictures and even advertisements.
Once article ranking by editors and readers was avaialble in PressDislay.com, the next question was how to display the top stories on the home page. With over 450 publications from around the world being published 24/7 it was a challenge deciding which stories should receive the highest page position, given that some stories had high rankings from readers because they were published 12 hours earlier, while other stories had no ranking yet because they may have only been published 5 minutes before.
To address this issue, we decided to offer our readers 3 options. Readers could choose to see stories ranked highest by editors (Editors’ Picks), stories ranked highest by other readers (Readers’ Choice) or a blend of the two.
Editors’ Picks
When readers first enter the PressDisplay.com site, they will see the stories that editors thought were the most interesting. Then throughout the day, PressDisplay.com will rotate stories as new issues are published, so the most recent developments in stories are promoted on the site, keeping readers up to date on breaking news and story updates. In this respect PressDisplay.com behaves in a very similar way to Google News or Yahoo!.
Readers’ Choice
If visitors to PressDisplay.com want to see the stories that other readers found the most interesting they need only slide the red bar along the new “Readers versus Editors” control in top right corner of the window, and then watch the whole display change to show the stories that retained readers’ attention the longest.
For this “Readers’ Choice” view, we start by treating all stories as equal, so there is no initial preference of one story over another. We still give higher priority to the news that has just been published, but other than that, all stories are considered equal until people start reading them. And then every 10 minutes, the PressDisplay.com site is refreshed using the latest reader data, ensuring the all article rankings are up to date.
A Combination of Editors’ Pick and Readers’ Choice
For visitors who want to see a blended view of these two perspectives, they can just move the slider to the center between the two choices.
Customizing PressDisplay.com
For visitors who want to see how other readers with similar interests to them ranked articles, they can use the customization features of PressDisplay.com to tailor the site to their preferences. By creating a favorites list of newspapers, showing only stories from those sources and choosing articles written specific languages, visitors can create an environment for themselves that provides insights into how others who read the same newspapers in the same languages evaluate stories.
However, this doesn’t limit the reader’s view of our global audience. They can choose at any time to create a home page that shows stories from all sources or a combination of all sources and their favorites using the slider just below the editors’ picks versus readers’ choice slider control
There is also a one-week back issue calendar so they can look back at the different perspectives of readers and editors over the past 7 days. And as it does with all other user preferences, PressDisplay.com remembers your last choice of view (readers versus editors) and so the next time you login to PressDisplay.com you will be presented with that perspective of the news.
It is pretty interesting to toggle back and forth between Readers’ Choice and Editors’ Picks to see the difference between what editors thought readers would read and what they actual did.
Take a look at the January 31st view of our readers’ choice in sports stories from 18 popular US newspapers randomly selected.
Notice there are lots of green cells highlighted in these news stories – the more green cells a story has, the higher its rank by readers. Up to 5 green cells are awarded based on readers’ article ranking.
Now here’s the editors’ picks for sports stories using the same selection of papers.
A few green cells are visible here and there, but not nearly as many as the Readers’ Choice stories.
Now look at the story titled “Anger Management”. It was rated as a top sports story in the selection of 18 newspapers by the editors and given prominence on the front page of the sports section of the Miami Herald.
Isn’t it interesting that it never made it to the list of readers’ top sports stories. Also, the 15 related stories in other papers didn’t receive much interest either from our readers. Meanwhile, a story buried inside the Washington Post on page E8 was given the highest ranking by all of the readers.
One story that grabbed the attention of readers that week was found in the January 28th issue of the Los Angeles Times on page B4 – “Next Idea: a key-ring Breathalyzer”.
This story was a “one-off” article carried by no other newspaper in all of our 450+ editions in PressDisplay.com. In the 18 selected US newspapers, it was ranked #1 by readers under the News section but never came close to an “Editors’ pick”. It would be interesting to know if the advertiser on this page found this to be a successful campaign for them. Could anyone predict that this page would receive so much attention? Some may call this kind of thing an anomaly, but you’d be surprised how often obscure stories win the eyeballs of readers.
An interesting twist on the wisdom of crowds…no matter what the people in power do to try and influence behavior, everyday people still have an important say when it comes to democratic choices – even in the news.
You can check out PressDisplay.com for free. It’s worth your time. You will need to register to see inside a publication, but you can read two articles from every title on the site every day, without paying anything. It’s a unique opportunity for you to see what other readers are actually reading and how it differs from what editors predicted, and it is fun to discover gems of stories you might otherwise never see.
Check out the press release where you can see more examples and a video on how it works. Enjoy!
