__/ [ Lance ] on Wednesday 10 May 2006 12:46 \__
> When a person is given an awful diagnosis by a physician (e.g., "You
> have cancer and about six months to live") many patients will seek a
> second or a third opinion. The implicit argument the patient is making
> in seeking second and third opinions seems to be a statistical one
> using the binomial distribution: The probability of n independent
> judges reaching the same wrong conclusion is q^n where q is the
> probability of a individual diagnosis being wrong. If we have
> confidence in physicians we may think q is small to begin with, and q^n
> will then become very small as n increases.
>
> This seems to be an argument for the importance of consensus amongst
> experts if a lay-person is to judge the truth of some opinion. But is
> this position - that consensus amongst experts is what we should look
> for to determine the truth of an opinion - correct?
The physicians might have some commonalities though. They may have studied
using the same literature or attended the same school. In practice (and
*at* the practice), their patients must have had an impact as well. The
same applies to a whole plethora of matters, including peer reviews in
academia. We /are/ what we eat and drink, or put differently, or opinions
our dependent upon what we are exposed to. When was the last time that you
hear flattery in the context of Bush (I am *not* defending him here)?
> Notice that the same kind of problem arises when journals assess
> manuscripts (different reviewers may disagree and the mere fact of
> disagreement is often used to reject papers) and in many other areas
> where people seek expert guidance.
>
> Some points that occur to me:
>
> - We need to be sure that we are dealing with real experts (though How?
> Is a question here!). Paul Meehl, for example, once wrote: "On a
> disputed point in quantum physics I would rely on Dirac's judgement
> rather than on the pooled judgements of ten psychologists."
> - The binomial argument above depends on the various experts being
> consulted offering independent opinions. But in fact the second and
> third doctors consulted may well just pick up their phone and discuss
> the issue with the first physician resulting in non-independence. A
> second form of non-independence may occur in that all the physicians
> concerned may share identical training and therefore rely on the same
> (perhaps faulty) knowledge and rules in arriving at their conclusion.
> - The binomial argument seems to require a rather well developed area
> in which expertise has been acquired. We assume that the physicians
> really do have real knowledge of the domain and that the information
> available to the experts is stable. But in fact medical and other
> domains of knowledge are often quite unstable, with radically new
> theories and findings emerging all the time. If the latter is true is
> it wise to expert people grappling with integrating this body of
> findings and theory to agree amongst themselves?
> - Expert judgements should perhaps be thought of as having error
> margins, and perhaps we should seek not only the "yea and nay"
> judgement, but some understanding of how wide the margin of error could
> be
> - The judgements we get from experts perhaps are wrongly constricted as
> yea or nay judgments: perhaps they cover a range of interconnected
> issues and so agreement or disagreement between experts might be about
> any of these issues
>
> If consensus is not the best way to assess expertise how should a lay
> person or consumer of expertise go about assessing expertise?
Expertise, after many years of practice, is often judged and measures by
familiarity with the written 'facts'. In the case of the medical domain,
there are some immutable truths. There are not always blacks and white
however. Even when it comes to the diagnosis, don't neglect the fact that
probability is involved (you could take this further to realms of
dicussions on causality). So, the doctor, as in your example, might face a
situation wherein the patience plays a role in correctness of the
diagnosis.
I hope my drivel makes some sense...
--
Roy S. Schestowitz
http://Schestowitz.com | SuSE GNU/Linux ¦ PGP-Key: 0x74572E8E
4:00pm up 12 days 22:57, 8 users, load average: 0.87, 0.73, 0.68
|
|
