[Heartlogic-dev] Re: statistics (was Re: Open Heart Logic)

[William L. Jarrold]

On Mon, 27 Oct 2003, Joshua N Pritikin wrote:

> On Sun, Oct 26, 2003 at 11:59:50PM -0600, William L. Jarrold wrote:
> > On Wed, 22 Oct 2003, Joshua N Pritikin wrote:
> > > Can we assume a single methodology or are many different
> > > statistical approaches possible?
> >
> > It is very likely that many different statistical approaches
> > are possible.
> >
> > I think we can hack our way through the first 100 survey responses
> > just to learn more about the issues.  As Mr Bottom Up Man, I am
> > a firm believer in learning via experience.  Plus, it allows me
> > to indulge procrastination.
>
> Fair enough.
>
> > > What I imagine is that we'll ask a question like:
> > >
> > >   Tracy wants a banana.
> > >   Mummy gives Tracy an apple.
> > >   ->
> > >   Tracy is sad because she wants a banana.
> > >
> > >   Believable?  (Yes) (Somewhat) (Not really) (No)
> >
> > *Exactly*....Except for the minor detail that I think it is
> > better to have a Likerat rating scale.  That is,
> > "Please rate the believability of the above scenario on a scale
> > from one to five"....There should be an example in the dissertation.
>
> Likerat, yah.  No problem.

Sorry, my bad, Likert.

>
> > If not or if you wanna see more examples, I can send you a survey soon.
>
> Sure, more examples would be good.

Okay I have attached it.  For my purposes I am writing a note to myself
that I got this from:

address@hidden:~/ArchivedFiles/GradSchool/Dissertation/Items/Study3Items/After2002/SurveysForMon/grp1.tex.

>
> > > Then we'll show the user stats about how many people voted
> > > for each evaluation, adjusting for ablation.  (?)
> >
> > Ugh.  More or less.  To really get that across how that works
> > will take a long time (not to mention a much better understanding
> > of stats than I have now).
> >
> > Hmm, should I attempt to explain statistical inference to you?
>
> No, but you should try to point me in the right direction.
> What I did (in a panic) is to read about half of this book:
>
>   http://www.stat.lsa.umich.edu/~faraway/book/
>
> That helped, but I am still feeling a little bit uneasy
> about the statistics.  Should I invest $100 on what seems
> to be the standard reference?

I doubt it.

>
>   "Statistical Analysis With Missing Data" by Little, Rubin
>
> Or is this a waste of money?  Can you send me a copy of
> this article?
>
>   Graham, J.W., Hofer, S.M., & MacKinnon, D.P. (1996). Maximizing
>   the usefulness of data obtained with planned missing value
>   patterns: An application of maximum likelihood procedures.
>   Multivariate Behavioral Research, 31(2), 197-218.

Not worth it.

>
> The other question is, am I taking this too seriously?

Yes.

> Are you confident that you can wave the magic wand and
> do robust statistical inference of the highest scientific
> standards?

Not confident but with a little elbow grease we can work it
out....If we want to publish in an AI journal, they won't care.
A psych journal will care.  If I re-read my diss (and hopefully
even if I don't) it will come back to me...BUT, we can get
good important results by departing from statistical stricture.
In fact, it is very very likely we will have to.

>
> > > What other testing formats do we want to accomadate?
> > > A good prediction about this can save us redesign later.
> >
> > Hmm, not sure.  I mean, I am sure we will eventually want
> > other testing formats than items like in my dissertation.
>
> Can I assume that there will always be these components:
>
>   given assumptions
>   appraisal (from the KR model)
>   believability rating of the appraisal (from the user)

Yes.  Subject to these mods....
The given assumptions I call the scenario cue.  Whatever.
The appraisal may or may not come from the KR model.  I used
human generated appraisals too.

Oh, we might also want them to optionally add some commentary text
explaining why they thought the scenario was believable/unbelievable.

>
> Or is this too restrictive?  Can you predict anything
> about the structure of our questioning?

Certainly for the beginning, yes, that is what we want.

But, one of the nearest term mods I can see us wanting is the following:
I am also interested in just having people write their own appraisals
of situations.  E.g. we might have them select from n emotions and add in
their own brief text explanation.

It would also be cool to just have users rate stuff.  E.g. "How
interesting[common sensical, logical, etc] is the following
assertion."...I'd like to compare ratings of interestingness (etc)
as a funciton of a) was the assertion generated via Cyc or Thought
Treasure or Open Mind....But this drifts away from open heart.

That's all I can think of now.  And I have GOT to get to bed!!

>
> > > One side benefit is
> > > that we probably don't have to worry about getting hosted on
> > > a super powerful computer.
> >
> > Hmm.  Does OpenMind have a super powerful computer at their
> > disposal?
>
> It is certainly more powerful than the computer we will
> start with.  ;-)
>
> > > Once we have something basic working then we will be able
> > > to publish an article such as "The [Open] Heart Logic
> > > Initiative" with a call-for-participation.
> >
> > Yipee.  But, I'd rather get a lot of friends to try it out first.
>
> Yah, obviously.  :-)
>
> > Also, if we have something soonish, maybe we can get some subjects
> > from the UT subject pool to try it.  At the end of the semester
> > there are usually a bunch of kids who have procrastinated and missed
> > their chances to participate in required research.  As a result
> > they must do some dumb assigment which some poor slob grad student
> > must grade.  Well, we can save poor slob grad student IF our
> > system is working in time.  I'll need to check with Diane
> > if this is an option, but if you think that you can have something
> > which more or less replicates my dissertation ready by say Nov 25th,
> > then I should ask Diane soon if we can qualify for the study.  They
> > will have to make special exceptions since we did not go through
> > the usual application process back in Aug/Sept.
>
> Nov 25.  Let's see what I can do.

No guarantee that is what the date is.  Soon, I'll try to ping Diane.
It might hit harder if we actually have a simple survey on the web.
Then she'll know that we mean business are ready to go.  She knows me
too well to know that I tend to do things chaotically at the last minute.
If she sees that thing works NOWISH or SOONISH, she'll realize that
Bill's last minute chaos is not going to be an issue.  Thus, she'll be
more likely to flex her political muscle to get us in the door.  Then
gain maybe the IRB (internal review board) has super strict rules and
that the idea is already dead.

Bill

p.s. I thikn your gimp picture is great!  No need to make it better.
Imperfection is the spice of beauty.  (Hmm, sounds like some corny
quote but I swear I just made it up).

>
> --
> A new cognitive theory of emotion, http://savannah.nongnu.org/projects/aleader
>

grp1.tex
Description: TeX document