Calculation
Last updated
Last updated
can make this calculation more faster.
In normal dsitrubiton, the quadratic form means mahalanobis distance. This is the distance that(https://darkpgmr.tistory.com/41) how much each data is away from the mean over the standard deviation.
Using this expression, we can interpret it as assigningto the loc . This expression mahalanobis distance can be changed into euclidiean distance. In a transformed space, each point is assigned into the closest median point. It controls the effect of prior probability(). The transformed space is equal to Whitening transformation that makes the variance of X .
: The dimension of input matrix
: The number of centroid.
LDA(QDA) reduces dimension into . The number of centroids in dimension is , so the dimension is reduced into . Let be a subspace spanned by these centroids. The distance between this subspace and can be neglected. ( These centroids are already in the subspace, so this distance has same impact on these points. Project transformed onto , and compare distance between the projected points. If the variance of these projected centroids is big, this is an optimal situation. Finding the optimal subspace is same to finding PC space of centroids.
Class centroids , Common covariance matrix
using the eigen-decomposition of
, Columns of are
: Within-variance matrix, : Between-variance matrix
Find the linear combination such that the between-class variance is maximized relative to the within-class variance.
The problem can be changed into this one:
The solution is the biggest eigenvector of . We can represent our data in a reduced form using the axis of . is called as a canonical variate and this becomes a new axis.