Definition Regression analysis

The regression analysis is an analytical method which allows us to calculate a regression as a "straight line" or as a "polynomial line". The regression informs us about the directed dependence between two or more variables. In the case of only one independent variable, the regression is defined as univariate. In the case of multiple independent variables, the regression is defined as multivariate. The so-called coefficient of determination (R²) expresses the quality of a representation of the relationship between the independentand dependent variable through the regression function. The values of R² lie between 0 and 1, where R²=1 means that all observed data points are perfectly located on the regression function. R²=0, on the contrary, means that the observed data points are all "nowhere near" the regression function.

Together with the R², other important parameters for the regressions are the F-test for the joint significance of the independent variables of the model in predicting the dependent variable, and the t-test to assess the significance of the single variables' coefficients. For the former, the regression function is tested against the null hypothesis that all regression coefficients equal 0 (or, that the independent variables of choice are no predictors of the independent variable). For the latter, the variables' coefficients are tested individually against the null hypothesis that the specific coefficient tested equals 0 (or, that the independent variable selected is no predictor of the independent variable, regardless if the other independent variables are or are not valid predictors). In both cases, it is common to calculate the P-value to "answer" the tests and, if P-values are lower than 5%, the null hypothesis can be rejected with sufficient certainty and the whole model (and/or the regression coefficient) can be seen as significant predictors of the independent variable.

Please note that the definitions in our statistics encyclopedia are simplified explanations of terms. Our goal is to make the definitions accessible for a broad audience; thus it is possible that some definitions do not adhere entirely to scientific standards.

Entries starting with R