Research Supports Conversational Index

Some research by a University of Amsterdam researcher and Buzzmetrics support the principles behind the Conversational Index, namely that having lots of comments (they didn’t look at trackbacks, lamentably) is an indicator of a blog’s importance:

[from ResourceShelf]

New Research Paper (DRAFT): Large Scale Study Looks at Comments Posted on Weblogs (8 pages; PDF)

This time the prolific Dr. Gilad Mishne from the University of Amsterdam and Natalie Glance from Buzzmetrics have written (what we’re linking to is a draft) about their research into commentary left of weblogs. This paper will be presented at a blogging ecosystem workshop during the WWW2006 conference next month. We’ve posted several other new papers by Dr. Mishne here and here during the past couple of months including one focused on blog search.

And the results?

These tables show the relationship of comments and popularity.

Their conclusions:

I can’t translate this data to the CI model (because of the lack of trackbacks), it strongly supports the metric (although, I didn’t consider the size of comments — since I don’t have an easy way to get those numbers). And I am suprised that the authors didn’t come up with a formula like the Conversational Index, themselves.

19 April 2006 Permalink

New Voices: Leisa Reichelt

I am resurrecting a project that I started some time ago at Get Real to try to find new voices, based on a call from Halley Suitt. I am not going to be too much of a stickler about race, gender, and geography — Halley was trying to counter the American white male dominance of the blogosphere. I have no quarrel with that, but it’s not exactly what I am trying to do.

No, I am simply trying to uncover people who have been blogging for a short time, less than a year, to hear some new thoughts, experience new perspectives, and — honestly — just to smell that new mown hay.

Today’s choice revealed herself to me through the Oracle at Technorati: a link to the Conversational Index meme in a recent post exposed the Disambiguity blog of Leisa Reiche, who is an Australian woman, working in web design. Here’s one of her observations after blogging for a few months:

[from blogging milestones (2 months) at disambiguity]

Something I’ve noticed in my last two months of blogosphere immersion (and that includes a bit of mailing list immersion too) - there are some really mean people out there. Not many - nice people are definitely winning - but those nasty ones… eh. I’m still working out how to deal with them best. They’re not quite like trolls that you can just ignore…. but there’s no reasoning with them either….

At the end of the second month, the most exciting thing has been having a sense of engaging in a community - of getting to ‘meet’ people who thing about similar things to me, to swap notes on ideas and experiences, and just generally to be a part of the flow of ideas and conversations that is the energy of the blogosphere (sorry…. couldn’t think of a better word to describe it).

No apology is necessary, Leisa. Thank you for the unadorned and direct sense of excitement.

28 March 2006 Permalink

Power Laws, Popularity, Authority, A-Lists and the Rest

Robert’s advice to the bloglorn is a bit superficial, focusing on eBay-ish features like adding a picture to your Technorati profile, or catchy headlines. Some of the tips are useful, like using lots of descriptive tags (as that will help search engines index your posts better).

However, here’s my list of what to do to improve your blog, so that your sphere of influence will widen and various rankings will increase. Maybe it will push you into the so-called A-List. [Note: /Message a lowly, lowly 7,379 at Technorati this morning, which is nothing like Robert’s 74, and the best T’rati rank I have ever hit is somewhere around 1200, for the Get Real blog. But still, the techniques I have used to climb from one million plus to 7,379 in the past 35 days (chronicled in the Starting From Zero series) are very different from what Robert is talking about.]

1. True Voice — The absolutely, indispensible, central core of all great blogs is authentic and empassioned writing, clearly expressing a consistent and value-based perspective. If you do not possess this, work hard to see how others do it, and emulate their techniques.
2. Throw Yourself Into Dialog — Do not write in a corner, looking at the walls. Most great posts are a response to the writing of others. You read something (as I read Robert’s post this morning), it sparks some thoughts, and you add to the thread. Then continue on: see if those involved in the thread respond to your addition to the discussion. Repeat.
3. Draw The Line, Over And Over Again — At any given time, successful, engaged bloggers are pursuing a set of themes or topics. These are like an investigative series in conventional journalism, topics that you return to, time and again, successively elaborating your view or arguments. Keeping tabs on the censorship in China, or posting consistently on why certain forms of marketing is immoral, or whatever. State your position and defend it. Howl at the inequities in the world. Shake your finger at the idiots.
4. The Big Idea — Every once in a while, work on one of those big posts, that outlines an idea that may have big implications. This could be asking a hard question, or debunking conventional wisdom, or defining the outlines of a new, emerging market. I recently introduced the Conversational Index, which led to a large cascade of commentary and thinking by others. In past years, I have been lucky enough to click that way with other notions, like last fall’s RSS Readering meme. This is a function of invention, and is hard to channel or predict. But the effect, even of just asking a really hard, important question, can be enormous.
5. Sharpen Your Pencil, And Then Write. The Polish polymath Ignace Paderewski once said, “before I was a genius I was a drudge.” Writing skills sharpen with use, and the sphere of influence also increases through frequency. You should write — at a minimum — every day.
6. Courage — You have to be willing to be called an idiot by some if you intend to be considered an authority by most on the topics you are interested in. Accept the occasional (or even consistent) vitriol from detractors and nay-sayers. If you stand up and say something is great, or pointless, or the most likely trend for the future, you can be sure that there are others that will disagree, and they will be happy to say so. Fine. But you can’t hedge, and middle-of-the-road platitudes or cautious optimism — which may come naturally after a diet of television news and mainstream journo-babble — will simply not break you out of the pack.
7. Technology — By all means arm yourself with technology. Learn how search engines work, and do the obvious things. Expressive titles, especially with people’s and products’ names help greatly. Tagging with detailed terms helps search engines and people alike. By all means, make your blog visually pleasing, accessible, and easy to read. Use graphics when appropriate, such as screen shots or diagrams. Link to all the people and stories you reference, and include people discussed as tags.
8. Timing Matters — I am not suggesting blowing hot and cold on themes, but rather try to build on stories when they are still new and in people’s thoughts. I saw this post of Robert’s, and I am using it as a springboard to collate a bunch of my thoughts on the topic that he opened. If I had waited a week, a much smaller number of people would read it, because next week this will be one of last week’s hot themes. So timing matters.
9. Human Sized Pieces — People are busy, and so your posts should generally not be 20 page dissertations. How long do you expect people to spend reading your thoughts? Can you condense? An occasional “Interesting piece from Robert, check it out!” may be ok, but a steady diet of link-blogging is too low fat for most of us. We need more juice. But only a plateful at a time. Not every thing needs to be a three course meal.
10. Respond to comments — People that comment on your blog are most likely those that are most interested in the topics you are writing about (leaving aside your mom, who just comments to make you feel better). Engage them when they come. But never feed the trolls.

I recently fired myself from an Amercian Marketing Series on social media, because I sensed that a high proportion of the folks that were attending the seminars were approaching the whole idea of blogging tactically: “How little of this do I have to do to be doing an adequate job?” My problem is I only want to talk to people who approach the subject strategically, working backward to the various elements from an analysis of excellence. I bet that those who buy in on that approach will at least find an echo of their own thoughts in these recommendations, and the rest will simply think I am a monomaniacal windbag with too much time on my hands.

15 February 2006 1 Permalink

The Conversation About The Conversational Index

There has been so much buzz about the Conversational Index idea, that many (if not most) people have lost the original context:

[from Blog Conversational Index: say what? by Jon Lebkowsky]

• This is an incentive to end the war against comment spam, because the more I get, the better my comment index. *8^)
• Am I more effective because I draw more attention from vocal people, and other bloggers? How do we measure the lurkers? How do we measure the quality of responses, and the cluefulness of responders?

[from Weblogsky: Blog Conversational Index: say what? by Nancy White]

Ed Vielmetti chimed into Jon’s comments with a good observation:

Some online conversations happen through blog comments, others through IM and email and in-person sideband and voice and … and …

No one index covers any meaningful part of it.

Stats lie. Using stats to measure utility just encourages people to game the stats. Not a game I want to play…

I added my agreement, noting that we’ve seen this metric come up in the “old days” of forum based online communities. Quantity did not equate to quality. It’s like confusing signal and noise. Unless you are going for pageviews for advertising models. And I thought we were “talking” about “conversations!” wink wink

The quality of a conversation is entirely contextual. Reducing it to a number does not give an indication of quality.

This goes to another assumption. Are all blogs about conversation? Are all conversations bloggy? Naw… we know they aren’t. But it is chic to glorify conversation the way we used to glorify “community.” (Yes, I’m getting snarky. IT must be all the SuperBowl testosterone floating around my house. I can’t help it!)

I’m not totally dissing the idea here. I find it very interesting to look at ways to discern patterns in blogs. There is value in looking at the ratio of post to responses. But it cannot stand alone as a measure of value. Conversation is still, thank goodness, a quirky human act that cannot be reduced to a metric.

Lots of good points. Yes, people will game any system. Yes, comment quality varies. Yes, people have tried to quanitfy the quality of conversation before. Yes, yes, yes.

But I do maintain that a healthy degree of conversational interaction is a necessary precondition for a successful blog, which was the original context of the idea. I noted that all the blogs that became successful at Corante — had steady link counts, regular readers, engaged authors — displayed a high Conversational Index (using Dodge’s variant of CI, CI=(Comments+Trackbacks/Posts)) from the very start, and maintained it.

Note that is may indiciate all sorts of causes:

• The author(s) of the blog may be tapping into an existing personal network or an existing community, and the blog meets the needs of that group.
  
  
• The topics being convered in the blog are extremely topical.
  
  
• The author(s) may be unusually gifted writers.
  
  
• The author(s) may be celebrities or may be involved in some controversy or scandal that leads to high traffic from the very start.

Not a comprehensive list, but good indiciators. Examples: Guy Kawasaki’s new blog has taken off, and the reason: a good writer who is a celebrity, talking about important topics. But the average unknown blogger, writing in the dark, divorced from a larger community of interested people? Hopeless. A relatively unknown genius, deeply thoughtful on the search for life balance and meaning, swept into the public eye by the Tsunami? Evelyn Rodriguez.

And of course, at least I hope, /Message, where the CI has moved up from 1.27 the day I wrote the first post about it to today’s 1.75, thanks to the lively conversation around the idea of the CI itself, and other posts, like the Cocomment release.

So, to put back into context: The Conversational Index is intended as a leading indicator of present and future blog viability and vitality. I don’t know whether an index of 3 would be twice as good as an index of 1.5. Perhaps there is some sort of reverse log scale involved, where 3 only indicates a slightly more engaged and active community than 1.5. But I do know, empirically, that those with subpar CI, where there are way more posts that comments or trackbacks, are unlikely to be successful in the long run.

8 February 2006 1 Permalink

Blogs its all about the conversation

Wow. What I thought was a modest post with a neat and helpful small idea — The Conversational Index — took off in a big way yesterday, getting picked up, and picked on, by a long list of smart people. I thought I would aggregate various comments and try to address them in one place.

The basic idea?

[from The Social Scale of Social Media: The Conversational Index by Stowe Boyd]

While working at Corante, I had the opportunity to peer at the stats for all sorts of blogs that we had going. And one thing that became really obvious is that sucessful blogs — ones that were currently viable and vibrant, and those that were on a growth trajectory from their start — shared a common characteristic: The ratio between posts and comments+trackbacks (posts/comments+trackbacks) was less than one. Meaning that there was more conversation — as indicated by the number of comments and track backs offered by readers — than posting articles. I will call this the Converation Index, just to put a handle on it.

Go Flock Yourself was the first to trackback on the topic, and giving me the best laugh I’ve had recently:

I’m very happy to pronounce to the world that GFY sports a Conversational Index of 0.135 (233 posts/1724 comments), a number that should make Stowe’s beret flip up off of his head and fly around the room like a frisbee, and reduce Richard MacManus to tears on the floor of his mother’s basement.

Doc Searls comments:

Oh shit. My ratio sucks. I think I run less than a comment a day, vs. half a dozen posts or so.

Of course, my blogging environtment doesn’t encourage comments.

And, I look for comments back on other blogs. I think I’d rather have them there anyway.

So… I dunno.

Don Dodge suggests that the CI is useful, but should be inverted, which I agree with, so from this point forward I will use Don’s variant, where CI = (Comments+Trackbacks)/Posts. This means the CI gets larger as the conversation gets richer:

[from The Conversational Index for blogs]

Note, I have calculated the number differently than Stowe, but the meaning and measurement is the same. Using my formula and Stowe’s blog stats his blog has a CI of (71+31)/80=1.27. Stowe Boyd has several blogs, and is a very well known writer, so my guess is that these numbers are from one of his newer, and lesser known, blogs.

Don is right: I was using the figures from /Message, which is less than 30 days old. My bet is that my CI is small because /Message is new, and the index will go up over time, as more people find /Message.

Peter Caputa has a CI (Dodge variant) of 1.53, and thinks this is because…

I ask a lot of questions and people humor me with answers. And because I say a lot of stupid things, and people yell back at me. And because I ask people for their feedback sometimes on specific posts.

Zoli Erdos builds on the discussion and points out that we are losing track of the other half of the conversation: the dark comments out there.

Tracking The COMPLETE CONVERSATION - Part 3]

we really are losing track of half the conversation in the Blogosphere.

As Stowe points out, for truly vibrant blogs the CI will be <1, which means there are far more comments than blog posts (I am cheating a little, ignoring trackbacks). This will likely be the case for all the Technorati top 100 or even 500 bloggers – from their viewpoint most of the conversation happens on / around their own blog. However, for the the rest of us, the other 26 million (?) bloggers chances are the conversation really takes place outside our own blog, and I for one certainly can’t keep track of all comments I left on other blogs.

The current crop of tracking / linking services all have a top-down publisher-centric view, where everything revolves around a blog and its related posts, totally missing this other, “bottom-up” half of the conversation. So please, somebody give Stowe his badge, but we also badly need a way to show by subject matter an integrated view of all conversations where we are participating whether we started the thread or someone else.

Zoli’s right in a way. After all, if I leave a comment on his blog, I am enhancing his CI and not influencing mine at all. However, if I write a post at my blog and trackback to his blog, I am influencing both CIs: his goes up (Dodge variant), and mine goes down. Of course, I am likely to get a trackback or a comment back from Zoli, or others involved in the conversation, so I personally bellieve that it is best to make that post and trackback. (And of course, there is cocomment, which I am profiling today, that intends to bring those dark comments out there back into the light. See the widget in the right margin?)

Michael Parekh says

[from ON STOWE BOYD’S CONVERSATIONAL INDEX]

Where do I come out on this? Well, I’ve been long-convinced of the value of comments in blogs as the next logical step of blog mining evolution.  (see Comments Search: the next big mother lode of user-generated content (UGC) and this post last June)

But Stowe’s idea of creating a mathematical formula has a Google like simplicity at it’s core. He even visualizes it as a living, breathing thing:

“I hope someone out there — some bored toolsmith, or a computer science student looking for an interesting project — will build a tool that will scan a blog, determine the CI, and provide the result as a chicklet that we can embed on our blogs. Even better would be a 30 day
graph, like Tufte’s sparklines, that shows the social interaction ebbing and flowing.”

It all sounds mesmerizing.

Until you realize that if the Conversational Index (CI) did in fact take off, both as yet another way to rank blogs on the Internet, and then actually as a tool to commercialize said rankings into real dollars for the bloggers, then might not this lead to the next logical step?

The overnight reversal of seeing Trackback and Comment Spam as BAD things, to actually GOOD, welcome things?

That is to say, we may need a mechanism to independently verify that

the Trackbacks and Comments reported as a component in the CI calculation are Good and Pure of the Spammy Stuff.

Not to mention the inevitable emergence of “Comment-fraud” and “Trackback-fraud” to take their place along-side of Click-fraud

Despite these possible negatives, thinking about the Conversational Index at least gets folks to start thinking about the value in comments and trackbacks.

A number of other folks commented that comment spam and trackback spam would artificially enlarge the CI, and I agree. But on the other hand, I was operating under the assumption that all sensible people would delete that junk. Still, it’s a relevant observation.

Easton Ellis deconstructs the CI pointing out that

1. comment quality varies [yes, I agree]
   
   
2. many blogs start out conversation-poor and gradually pick up speed as they gain a consistent following [true, but as I said in the first post, I saw at Corante that the successful blogs started out on the good foot: a good CI from the beginning]
   
   
3. what about the authors comments at her own blog, or trackbacks she sends to her own entries? [comments responding to comments is a good conversation, isn’t it? And tracking back to earlier posts is good ettiquette too, helping readers find subsequent posts that elaborate on earlier thoughts.]
   
   
4. Could this statistic be meshed with a particular individual’s CI? [Sure, you can average your CI from multiple blogs. Why not? We can do whatever we want here in the matrix.]
   
   
5. Is a comment always equal to a trackback? [I don’t know. But for simplicity they are in the formula.]
   
   
6. What about the number of commenters on or trackbackers to a blog? [I don’t know if a conversation is better when there are a smaller number of people involved, making serial comments/trackbacks, or if there is a larger group, where each individual comments/tracksback less. So for now, we don’t care. Also might be hard to get that number.]
   
   
7. My head hurts. I feel like a geek. [Me too.]
   
   

Mathew Ingram calls me a geek, too, but in a nice way:

[from Blogs — it’s all about the conversation]

what makes most blogs interesting isn’t so much the great things that the writer puts on there (as much as I like to hear the sound of my own voice), but what kind of response it gets, and how that develops, and who carries it on elsewhere on their own blog. And I agree that it would be nice if someone like technorati.com or memeorandum.com could track that kind of thing and make it part of what brings blogs to the top.

I like to see what people are talking about — not just what a blogger has to say, but what others have to say about what they say. That’s why I also agree with Steve Rubel that it would be nice to have a way of tracking comments, other than by subscribing to a feed of comments, or bookmarking posts you’ve commented on with del.icio.us or some other tool.

The latter problem is perhaps solved by cocomment (about which more later), and the former, by the blogpulse conversation tracker, which does a memeorandum-ish snapshot of the cascade of posts emanating from an initial “converation seed”. Here’s the picture that they draw from the initial Conversational Index post:

Note that they don’t track comments at the blog, though. Ultimately, I would like to have all that wrapped up into one representation.

Sadly, though, no one has yet stepped forward to build a tool that would yeild the number: we still have to do it manually.

As of this morning, my CI (Dodge variant — (C+T)/P) is (88+42)/89 = 1.46.

5 February 2006 Permalink

Don Dodge on The Next Big Thing: The Conversational Index for blogs

Don Dodge had apparently come up with the Conversation Index on his own, although he inverts the calculation:

[from Don Dodge on The Next Big Thing: The Conversational Index for blogs]

Note, I have calculated the number differently than Stowe, but the meaning and measurement is the same. Using my formula and Stowe’s blog stats his blog has a CI of (71+31)/80=1.27. Stowe Boyd has several blogs, and is a very well known writer, so my guess is that these numbers are from one of his newer, and lesser known, blogs.

Hmmm. Since the arithmetic is barely a day old, I am willing to be swayed. I guess it makes more sense that as the conversation increases the ratio gets bigger. So I will switch over to Don’s formula, instead of my first pass.

4 February 2006 Permalink

I demand the beanie off of Stowe Boyds head.

I think it was Brian Dennis Ritchie, one of the inventors of Unix and C, who said “All large systems that work start as small systems that work.” This is an enormously powerful thought, one that can be carried into all corners of life, business, and technology, and which resonates with today’s notions about small, focussed, and quickly evolving Web 2.0 apps. I also think that this insight also is deeply relevant to social media.

While working at Corante, I had the opportunity to peer at the stats for all sorts of blogs that we had going. And one thing that became really obvious is that sucessful blogs — ones that were currently viable and vibrant, and those that were on a growth trajectory from their start — shared a common characteristic: The ratio between posts and comments+trackbacks (posts/comments+trackbacks) was less than one. Meaning that there was more conversation — as indicated by the number of comments and track backs offered by readers — than posting articles. I will call this the Converation Index, just to put a handle on it.

Here’s the current picture for /Message, a CI of 80/102 = 0.784.

The down side? Those blogs that we started at Corante that did not take off, and subsequently went dormant, or were shuttered, had a Conversational Index greater than one: too much speech, not enough banter. And those that started badly seldom pulled out of the problems.

Perhaps it’s all unsurprising, really. But this conversational metric is not hidden, per se — I mean you can go to a blog and do the arithmetic — but our expereince of this differential between blogs is generally just sensed rather than explicitly measured. That may be a mistake. Perhaps it should be as relevant to determining whether something is worth reading as Google juice and Technorati rank, but it is not reported, and not used by those search tools, as far as I know. These search engines simply count links, and who is linking — which is a useful metric — but these are relatively crude measures, and don’t adequately measure the level of interactivity going on at blogs, I don’t think. They are industrial grade metrics, good for comparing the top 5000 blogs, perhaps. But they don’t really help a blogger with relatively light traffic to determine week to week, month to month, if what they are doing is satisfying to some group.

No, the Conversational Index is much better for the artisanal level of blogger. So I hope someone out there — some bored toolsmith, or a computer science student looking for an interesting project — will build a tool that will scan a blog, determine the CI, and provide the result as a chicklet that we can embed on our blogs. Even better would be a 30 day graph, like Tufte’s sparklines, that shows the social interaction ebbing and flowing.

And of course, my thesis: any successful blog starts in a small way, but from the very beginning is highly social. It is a place, a shared space, not a container for articles. The best predictor of blog success — aside from previous success in other blogs — is the Conversational Index, because it contains the outcomes of many other small things done right.

[Update - Doc Searls corrected the Brian Dennis error in the first paragraph, and says he can’t find the quote anywhere. Neither could I, but the C Programming Language text is not indexed on line. So who knows.]

3 February 2006 Permalink