Home Messages Index
[Date Prev][Date Next][Thread Prev][Thread Next]
Author IndexDate IndexThread Index

Re: Why does Roy ignore IBM?

* Tim Smith fired off this tart reply:

> In article <Wl%aj.21416$k27.10088@xxxxxxxxxxxxxxxxxxxxxx>,
>  Linonut <linonut@xxxxxxxxxxxxx> wrote:
>> Actually, I feel I'm rational, and I have an objection.  Combining this
>> 
>>    >> Seattle Area Bloggers Needed for Microsoft User Research Study
>> 
>> with this
>> 
>>    > studies?  ... usability studies?
>> 
>> raises the hairs on my back.  I don't care who does it, it is not a
>> valid study to have "Seattle Area Bloggers" getting goodies for using a
>> product.
>
> Why not?  You want people to come down to your office, try out some new 
> software for a couple hours while you have people watch them do it and 
> take notes, and then you want them to answer questions about it 
> afterwards, and you don't want to give them some freebies for their time?
>
> Here's a description of the process:
>
>    <http://www.microsoft.com/usability/studies.mspx>
>
> What's invalid about that?

   1. The entity doing the testing is highly interested in the outcome.

   2. The tested entity must sign a non-disclosure agreement.

   3. The test engineer includes subjective information in his
      observations.

   4. The payment is a company product, as opposed to the small sum of
      money that is customary payment in lab studies.

   5. The criterion for the assessment is not explained (perhaps it is
      in the follow-on links)

   6. There are no hypotheses stated (to be capable of refutation).

   7. The statistical analysis (data plot layouts and analytical
      methods) is not explained.

While this may be a somewhat valid method of provide some guidance in
product development, it is hardly a study.  Microsoft may well find
itself fooled by its own biased observations.

-- 
Tux rox!

[Date Prev][Date Next][Thread Prev][Thread Next]
Author IndexDate IndexThread Index