Saturday, October 29th, 2005, 3:27 am
Moderate the Moderators
Can the men/women behind CCTV be trusted?
EMEBER the a popular phrase “Who is watching the watchers?”
Well, it appears as though, under the umbrella of Web 2.0, where visitors’ involvement is perpetually encouraged, we face yet another challenge: how can comments be moderated, articles ranked and statistics assembled reliably?
Can you truly trust a moderator? Need you ever moderate the moderator? If so, are you not getting into a cyclic moderation trap here? Slashdot have introduced the idea of meta moderation, where moderators can be penalised for unfair treatment of comments. For example, some would be aware of the effects a terrible day has on personality. Even in a peer review process, people are more likely to punish others passionately due to their own personal problems.
Why has this idea sprung to my mind and awoken in my consciousness? WordPress has recently seen some collaborative comment spam filtering, currently known as Akismet. I was involved in testing the plug-in (see my entry on ending comment spam) and I can finally give more details about it.
Comments are marked as “genuine” or “spam” and their status is shipped to a central repository where filtering is administered. At present, API keys are given to trusted bloggers. This is a handicapping property of the service, which may exclude many. It may even be interpreted as insulting to some. The API keys are intended to keep spammers out of any ability to flag comments badly, thus collapsing the system and breaking everybody’s long-taught filters. Essentially, there is the possibility of filling the engine with noise, which makes it utterly unusable.
Finally, the scenario above begs for the question: how can the moderators — the test set which flags spam — ever be moderated if there is no trust or moderation atop the moderators? We are yet to discover where it will all end up. I believe they may be an exclusive ‘army’ of flaggers while the remainder will be just client of the Akismet filtering engine.