HSDent - by Charles Sizemore⋅ July 27, 2010 - “The Internet’s sheer scale means that listening to all of the noise would result in information overload. Before it can be understood effectively, it needs to be harnessed into usable data.”

–Dalrymple and Chrysafis, 2010

The quote above is particular true in the financial markets, where we at HS Dent operate.  The Information Revolution of the 1990s unleashed incredible, exponential amounts of new information.  The rise of social media and standardized blogging platforms ten years later has kicked this revolution into overdrive.

Even so, I still at times find the shear volume of financial information at my disposal to be overwhelming.  This is why I was particular interested in Dalrymple and Chrysafis’s 2010 white paper, “Using the Internet as a Market Research Database.”

Joel Dalrymple is an old friend from my days in London.  After getting his master’s degree from the London School of Economics he became a business consultant and currently works for a London firm that specializes in internet data mining.

In the July issue of the HS Dent Forecast, we highlighted how Rick Davis of Consumer Metrics uses his proprietary data mining techniques to forecast consumer spending weeks if not months before official government statistics are released.  We’ve incorporated Davis’s work into the HS Dent Forecast and have thus far been very satisfied with the results.

Dalrymple and his colleagues at Onalytica approach their research in a somewhat similar manner.  They point out that more information is not necessarily beneficial or desirable.  What is valuable is the ability to convert all this data into actionable intelligence.  What they do, essentially, is mine news and social media sites on the internet to determine which voices are most influencing a given debate and then use this knowledge to enhance the accuracy of predictions.

As an example, they used the recent election in the United Kingdom, which resulted in David Cameron becoming the new prime minister.

Interestingly,  Dalrymple and his colleagues found that, despite the explosion of blogging and social media, traditional media sources (i.e. established newspapers and their online editions) continue to hold far more sway than the upstarts. Dalrymple and Chrysafis write,

There is evidence that Social Media reflected the wider election debate, increased the speed and intensity of debate and in some cases was used effectively as a communication tool to engage with a network of local voters. However, a low influence measurement indicates that Social Media does not frequently generate discussion that is cited outside of those networks. Consistent with findings elsewhere, the content is more typically generated by the larger media agencies. This finding means that to gain attention the political parties could take actions to ensure they will be discussed by the ‘traditional media’ as the weight of this media in the overall discussion is significant

If we think back to the 2008 U.S. presidential election, it’s not hard to remember all of the talk about then-candidate Barrack Obama’s use of social media sites like Facebook to energize voters.  But if Dalrymple’s quantitative analysis were applied, it is highly likely that we would find that the traditional media was far more influential in shaping public opinion.

Sure, Facebook didn’t hurt Obama’s chances of getting elected, nor did the legions of bloggers writing favorable posts on his behalf (or negative posts on McCain or Palin).  But if future candidates want to draw the right conclusions from the 2008 election, it would not be that social media is essential to getting into the White House.  After all, Republican/Libertarian candidate Ron Paul had legions of online supporters yet, because he was largely ignored by the traditional media, he got very few votes in the primaries.  If you want to get elected, you need to have opinion makers in the mainstream press on your side.

At any rate, the implications here are enormous.  For marketers, traditional media sources continue to be most important for branding purposes.  For company executives, identifying the news sources that are most influential can be essential for damage control purposes (think the recent British Petroleum oil spill debacle). For investors or traders trying to gauge market sentiment or sentiment towards a particular stock, you have to know where to look.  News on the market or company as reported by the Wall Street Journal or Bloomberg continues to carry a lot more weight than news reported on a financial blog–and I say this as I’m writing a post for a financial blog!

Consider this the next stage of the Information Revolution.  First, the internet “democratized” the media, allowing virtually everyone to have a presence as a provider of information.  Next, came the overload stage in which far too many voices were added to the mix, making the new media hard to use in a practical manner.  Now, we are entering the organizational stage in which data mining techniques are allowing us to separate the wheat from the chaff.  We’re moving from “more information” to “better information.”  I, for one, consider it a step in the right direction.