Alpha level – The probability of incorrectly rejecting the null hypothesis that is used by a researcher to decide whether an outcome of a study is statistically significant (most commonly, researchers use a probability of .05) See also, Type I error.

Analysis of variance (F test) – A statistical significance test for determining whether two or more means are significantly different. F is the ratio of systematic variance to error variance.

Bar graph – A visual presentation that uses bars to depict frequencies of responses, percentages, or means in two or more groups.

Beta level – The probability of incorrectly accepting the null hypothesis. See also, Type II error

Central tendency – A single number or value that describes the typical or central score among a set of scores.

Concurrent Validity – The construct validity of a measure is assessed by examining whether groups of people differ on the measure in expected ways.

Confidence interval – An interval of values within which there is a given level of confidence (e.g. 95%) where the population value lies.

Construct Validity – The degree to which a measurement device accurately measures the theoretical construct it is designed to measure.

Content Validity – An indicator of construct validity of a measure in which the content of the measure is compared to the universe of content that defines the construct.

Convergent Validity – The construct validity of a measure is assessed by examining the extent to which scores on the measure are related to scores on other measures of the same construct or similar constructs.

Correlational coefficient – An index of how strongly two variables are related to each other.

Criterion variable – The variable/score that is predicted based upon an individual’s score on another variable (the predictor variable). Conceptually similar to a dependent variable.

Cronbach’s alpha – An indicator of internal consistency reliability assessed by examining the average correlation of each item (question) in a measure with every other question.

Decision logic

  1. There is a Research Hypothesis
  2. Set up the Null Hypothesis
  3. Set the alpha level (decision rule)
  4. Collect the data
  5. Compute how likely the results could have come out the way they did due to chance alone
  6. Decide whether to reject the Null Hypothesis

We reject the Null Hypothesis when: the outcome is rare (less than α) if only chance is at work. If the result is rare enough we say it is not just due to chance.

Degrees of freedom (df) – A concept used in tests of statistical significance; the number of observations that are free to vary to produce a known outcome.

Descriptive Statistics – Statistical measures that describe the results of a study; descriptive statistics include measures of central tendency (e.g. mean), variability (e.g. standard deviation), and correlation (e.g. Pearson r)

Discriminant Validity – The construct validity of a measure is assessed by examining the extent to which scores on the measure are not related to scores on conceptually unrelated measures.

Effect size – The extent to which two variables are associated. In experimental research, the magnitude of the impact of the independent variable on the dependent variable.

Error variance – Random variability in a set of scores that is not the result of the independent variable. Statistically, the variability of each score from its group mean.

Face Validity – The degree to which a measurement device appears to accurately measure a variable.

Frequency distributions – An arrangement of a set of scores from lowest to highest that indicates the number of times each score was obtained. Note: in frequency distributions, the dependent variable is plotted on the horizontal axis, and the frequency is plotted on the vertical axis (the dependent variable is otherwise typically plotted on the vertical axis).

Frequency polygons – A graphic display of a frequency distribution in which the frequency of each score is plotted on the vertical axis, with the plotted points connected by straight lines.

Histogram – Graphic representation of a frequency distribution using bars to represent each score or group of scores.

Inferential statistics – Statistics designed to determine whether results based on sample data are generalization to a population.

Inferential Z-score: a standard score for the sampling distribution

Internal consistency reliability – reliability measured with data collected at one point n time with multiple measures of a psychological construct. A measure is reliable when the multiple measures provide similar results.

Interrater reliability – An indicator of reliability that examines the agreement of observations made by two or more raters (judges).

Interval scale – A scale of measurement in which the intervals between numbers on the scale are all equal in size.

Item-total correlation – The correlation between scores on individual items with the total score on all items of a measure.

Mean – A measure of central tendency, obtained by summing scores and then dividing the sum by the number of scores.

Measurement error – The degree to which a measurement deviates from the true score value.

Median – A measure of central tendency; the middle score in a distribution of scores that divides the distribution in half.

Mode – A measure of central tendency; the most frequent score in a distribution of scores. Note: if there are two most frequent scores, this is called a “bimodal” distribution.

Multiple correlation – A correlation between on variable and a combined set of predictor variables.

Negatively-skewed distributions – the mode is more towards the right side of the graph

Nominal scale – A scale of measurement with two or more categories that have no numerical (less than, greater than) properties.

Null hypothesis – The hypothesis, used for statistical purposes, that the variables under investigation are not related in the population, that any observed effect based on sample results is due to random error.

Ordinal scale – A scale of measurement in which the measurement categories form a rank order along a continuum.

Partial correlation – The correlation between two variables with the influence of a third variable statistically controlled for.

Pearson product-movement correlation coefficient – A type of correlation coefficient used with interval and ratio scale data. In addition to providing information on the strength of relationship between two variables, it indicates the direction (positive or negative) of the relationship. (Symbolized as r.) r varies from -1 through 0 to +1. When r is 0, there is no relationship.

Pie chart – Graphic display of data in which frequencies or percentages are represented as “slices” of pie.

Population – the entire group we want to find something out about. A datum from a population is referred to as a parameter.

Positively-skewed distribution – the mode is more towards the left side of the graph

Power – The probability of correctly rejecting the null hypothesis.

Predictive validity – The construct validity of a measure is assessed by examining the ability of the measure to predict a future behavior.

Predictor variable – A variable that is used to make a prediction of an individual’s score on another variable (the criterion variable). Conceptually related to an independent variable.

Probability –  The likelihood that a given event (among a specific set of events) will occur.

Range – The difference between the highest score and the lowest.

Ratio scale – A scale of measurement in which there is an absolute zero point, indicating an absence of the variable being measured. An implication is that ratios of numbers on the scale can be formed (generally, these a physical measures such as weight or timed measures such as duration or reaction time.)

Reactivity – A problem of measurement in which the measure changes the behavior being observed.

Regression equations – A mathematical equation that allows prediction of one behavior when the score on another variable is known.

Reliability – The degree to which a measure is consistent.

Research hypothesis – The hypothesis that the variables under investigation are related in the population – that the observed effect based on sample data is true in the population.

Restriction of range – A problem when scores on a variable are limited to a small subset of their possible values; this makes it more difficult to identify relationships of the variable to other variables of interest.

Sample – a subsection of a population, the portion of a population which is actually studied, in order to find out more information about the population. A datum from a sample is a statistic

Sampling distribution – Theoretical distribution of the frequency of all possible outcomes of a study conducted with a given sample size. It has a normal shape; one of the reasons the normal distribution is good to know. The SD of the sampling distribution is known as the standard error.

Scatterplot – Graphic representation of each individual’s scores on two variables. The score on the first variable is found on the horizontal axis and the score on the second variable is found on the vertical axis.

Split-half reliability – A reliability coefficient determined by the correlation between scores on half of the items on a measure with scores on the other half of the measure.

Standard deviation – The square root of the average deviation of scores from the mean (i.e. the square root of the variance).

Standard error – The standard deviation of the sampling distribution. Calculated differently from standard deviation (see forumulae page)

Standard Score (Z-score) – An individual raw score, minus the mean, the difference of which is then divided by the standard deviation.

Statistical significance – Rejection of the null hypothesis when an outcome has a low probability of occurrence (usually .05 of less) if, in fact, the null hypothesis is correct.

Structural equation modeling (SEM) – Statistical techniques that are used to evaluate a proposed set of relationships among variables.

Systematic variance – Variability in a set of scores that is the result of the independent variable; statistically, the variability of each group mean from the grand mean of all subjects.

t test – A statistical significance test used to compare differences between means.

Test-retest reliability – A reliability coefficient determined by the correlation between scores on a measure given at one time with scores on the same measure given at a later time.

True score – An individual’s actual score on a variable being measured, as opposed to the score the individual obtained on the measure itself.

Type I error – An incorrect decision to reject the null hypothesis when it is true (alpha level).

Type II error – An incorrect decision to accept the null hypothesis when it is false (beta level).

Variability – The amount of dispersion of scores about some central value.

Variance – A measure of the variability of scores about a mean; the mean of the sum squared deviations of scores from the group mean.