Discriminant Analysis: By Vaibhav Khaparde

Discriminant Analysis (DA) undertakes the same task as multiple linear regressions by predicting an outcome. However, multiple linear regressions is limited to cases where the dependent variable on the Y axis is an interval variable so that the combination of predictors will, through the regression equation, produce estimated mean population numerical Y values for given values of weighted combinations of X values. But many interesting variables are categorical, such as political party voting intention, migrant/non-migrant status, making a profit or not, holding a particular credit card, owning, renting or paying a mortgage for a house, employed/unemployed, satisfied versus dissatisfied employees, which customers are likely to buy a product or not buy, what distinguishes Stellar Bean clients from

Gloria Beans clients whether a person is a credit risk or not, etc.

DA is used when:

The dependent is categorical with the predictor at interval level such as age, income, attitudes, perceptions, and years of education, although dummy variables can be used as predictors as in multiple regression. Logistic regression can be of any level of measurement. There are more than two DV categories, unlike logistic regression, which is limited to a dichotomous dependent variable.

The major underlying assumptions of DA are:

The observations are a random sample;

Each predictor variable is normally distributed;

Each of the allocations for the dependent categories in the initial classification are correctly classified;

There must be at least two groups or categories, with each case belonging to only one group so that the groups are mutually exclusive and collectively exhaustive (all cases can be placed in a group);

Each group or category must be well defined, clearly differentiated from any other group(s) and natural. Putting a median split on an attitude scale is not a natural way to form groups. Partitioning quantitative variables is only justifiable if there are easily identifiable gaps at the points of division;

For instance, three groups taking three available levels of amounts of housing loan; the groups or categories should be defined before collecting the data;

The attribute(s) used to separate the groups should discriminate quite clearly between the groups so that group or category overlap is clearly non-existent or minimal;

Group sizes of the dependent should not be grossly different and should be at least five times the number of independent variables.

There are several purposes of DA:

To investigate differences between groups on the basis of the attributes of the cases, indicating which attributes contribute most to group separation. The descriptive technique successively identifies the linear combination of attributes known as canonical discriminant functions (equations) which contribute maximally to group separation.

Predictive DA addresses the question of how to assign new cases to groups. The DA function uses a person’s scores on the predictor variables to predict the category to which the individual belongs.

To determine the most parsimonious way to distinguish between groups.

To classify cases into groups. Statistical significance tests using chi square enable you to see how well the function separates the groups.

To test theory whether cases are classified as predicted.

Methods implemented in this area are Multiple Discriminant Analysis, Fisher's Linear Discriminant Analysis, and K-Nearest Neighbors’ Discriminant Analysis.

Multiple Discriminant Analysis

(MDA) is also termed Discriminant Factor Analysis and Canonical Discriminant Analysis. It adopts a similar perspective to PCA: the rows of the data matrix to be examined constitute points in a multidimensional space, as also do the group mean vectors. Discriminating axes are determined in this space, in such a way that optimal separation of the predefined groups is attained. As with PCA, the problem becomes mathematically the eigen reduction of a real, symmetric matrix. The eigen values represent the discriminating power of the associated eigenvectors. The n_Ygroups lie in a space of dimension at most n_Y - 1. This will be the number of discriminant axes or factors obtainable in the most common practical case when n > m > n_Y(where n is the number of rows, and m the number of columns of the input data matrix).

Linear Discriminant Analysis

is the 2-group case of MDA. It optimally separates two groups, using the Mahalanobis metric or generalized distance. It also gives the same linear separating decision surface as Bayesian maximum likelihood discrimination in the case of equal class covariance matrices.

K-NNs Discriminant Analysis

: Non-parametric (distribution-free) methods dispense with the need for assumptions regarding the probability density function. They have become very popular especially in the image processing area. The K-NNs method assigns an object of unknown affiliation to the group to which the majority of its K nearest neighbors’ belongs.

Submitted By:

Vaibhav Khaparde

12054

Marketing