Re: Proposal for an Emacs User Survey

[Jean Louis]

* Ivan Yonchovski <yyoncho@gmail.com> [2020-10-26 21:14]:
> 
> Jean Louis writes:
> 
> > Both ways how the surve have been presented with the proprietary
> > Javascript and with the download of the Org file, are not user
> > friendly and cannot reach large sample. One also needs marketing
> > skills.
> 
> What number of responders will be considered large enough sample?

That is not easy to determine. I would say that 1000 people would be
minimum in each case and it would never be enough, as that comes from
personal experience from work in two organizations, one was
specifically a survey company and other was conducting always surveys
for its own humanitarian purposes.

From:
https://www.cloudresearch.com/resources/guides/statistical-significance/determine-sample-size/

As you can see, even when a population is large, researchers can often
understand the entire group with about 1,000 respondents.

  Population Size  Sample Size Based on ±3%  Sample Size Based on ±5%  Sample 
Size Based on ±10%    
    Margin of Error  Margin of Error  Margin of Error    
  500  345  220  80    
  1,000  525  285  90    
  3,000  810  350  100    
  5,000  910  370  100    
  10,000  1,000  385  100    
  100,00+  1,100  400  100    
----- end of quote --------

Every survey should have its purpose, usually to improve the product
or service, in this case Emacs. Improvements are constantly made and
public opinion is looked upon.

To discuss issues with people is human, friendly and vibrant activity
and is way of taking opinions (bug reports, emails, opinions, package
contributions) and putting them in reality (patches, improvements,
ELPA...).

To ask people questions online without opportunity for feedback is
one-way, dead end type of a communication.

There are few questions that look like there is question about
accessibility yet we will see later what comes out of it. If website
is not made well accessible those questions may not be productive.

For solving usability issues, instead of 1000 people one could as well
follow feedback from 5 users, reference:
https://www.nngroup.com/articles/usability-101-introduction-to-usability/

Article refers to web, it may be applied on Emacs user interface.

Quote:

Usability is a quality attribute that assesses how easy user
interfaces are to use. The word "usability" also refers to methods for
improving ease-of-use during the design process.

Usability is defined by 5 quality components:

- Learnability: How easy is it for users to accomplish basic tasks the
  first time they encounter the design?

- Efficiency: Once users have learned the design, how quickly can they
  perform tasks?
  
- Memorability: When users return to the design after a period of not
  using it, how easily can they reestablish proficiency?
  
- Errors: How many errors do users make, how severe are these errors,
  and how easily can they recover from the errors?
  
- Satisfaction: How pleasant is it to use the design?


-- 
Jean Louis