I find that (i) paying participants as soon as they finish the questionnaire yields a lower answer quality than waiting for all or an intermediate procedure (ii) having participants enter their names for receipt preparation does not reduce answer quality (and does not increase the social desirability of answers) (iii) a higher overall payment increases answer quality, while (iv) framing the fixed part of participants’ payment as being “for completing the questionnaire” as well as (v) progress feedback do not affect answer quality. Thus the accuracy and consistency of survey/questionnaire forms a significant aspect of research methodology which are known as validity and reliability. It is important for self-report scales and measures used by educators and health care professionals (such as the Physical Self-Description Questionnaire-Short. I run an experiment on five aspects that are specific to the laboratory context. The main objective of questionnaire in research is to obtain relevant information in most reliable and valid manner. A developed scale needs to be valid and reliable, validity is a measure of whether a scale measures what it intends to measure, and reliability is a measure of its consistency between measures. A short survey among experimental economists documents substantial heterogeneity in the procedures used, and in expectations about the effects of procedural details. While a lot is known about how to run experiments and about how to construct and run field surveys, much less is known about how to administer such surveys in a post-experimental context. The test-retest reliability statistics showed a kappa coefficient of 0.5460.897 (all P<0.001) and percentage agreement of 8098.3 indicating high temporal stability in the target population. Questionnaires eliciting personality traits and other characteristics of a person are important tools for many experimental economists.

Reliability of questionnaire