> For the complete documentation index, see [llms.txt](https://statduck.gitbook.io/statduck/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://statduck.gitbook.io/statduck/recommender-system/collaborative-filtering/a-view-of-regression.md).

# A View of Regression

## <mark style="background-color:red;">A Regression Modeling View of NB methods.</mark>

$$
\hat{r}*{ut}=\dfrac{\Sigma*{j\in Q\_t(u)}AdjustedCosine(j,t)\cdot r\_{uj}}{\Sigma\_{j \in Q\_t(u)}|AdjustedCosine(j,t)|}
$$

### User-Based Nearest Neighbor Regression

&#x20; The predicted rating is a weighted linear combination of other ratings of the same item. If $$P\_u(j)$$contains all ratings of item j, this combination becomes similar to a linear regression. The difference is that the linear regression find coefficients by solving optimization problems, whereas the recommender system chooses coefficients in a heuristic way with the user-user similarities.&#x20;

$$
\hat{r}*{uj}=\mu\_u+\dfrac{\Sigma*{v\in P\_u(j)}Sim(u,v)\cdot s\_{vj}}{\Sigma\_{v \in P\_u(j)}|Sim(u,v)|}, ;; s\_{vj}=r\_{vj}-\mu\_v
$$

&#x20;The above expression is changed into the below expression.

$$
\hat{r}*{uj}=\mu\_u+\sum*{v \in P\_u(j)} w^{user}*{vu} \cdot (r*{vj}-\mu\_v)
$$

$$
minJ\_u=\sum\_{j\in I\_u}(r\_{uj}-\hat{r}*{uj})^2=\sum*{j\in I\_u}(r\_{uj}-\[\mu\_u+\sum\_{v \in P\_u(j)} w^{user}*{vu}\cdot (r*{vj}-\mu\_v)])^2
$$

$$
min\sum^m\_{u=1}J\_u=\sum^m\_{u=1}\sum\_{j\in I\_u}(r\_{uj}-\[\mu\_u+\sum\_{v \in P\_u(j)} w^{user}*{vu}\cdot (r*{vj}-\mu\_v)])^2
$$

&#x20;To reduce model complexity, the regularization term like $$\lambda \Sigma\_{j \in I\_u} \Sigma\_{v \in P\_u(j)} (w^{user}*{vu})^2$$ could be added as regression do. $$P\_u(j)$$ can be vastly different for the same user $$u$$ and varying item indices(denoted by $$j$$), because of the extraordinary level of sparsity inherent in rating matrices. Let me consider a scenario where one similar user rated movie $$Nero$$ whereas four similar user rated $$Gladiator$$ for target user $$u$$. The regression coefficient $$w^{user}*{vu}$$is heavily influenced by the rating for $$Gladiator$$ because it has more sample. It leads to overfitting problem, so scaling method needs to be applied in $$P\_u(j)$$.

$$
\hat{r}*{uj}\cdot \dfrac{|P\_u(j)|}{k}=\mu\_u+\sum*{v \in P\_u(j)} w^{user}*{vu} \cdot (r*{vj}-\mu\_v)
$$

This expression predicts a fraction $$\frac{|P\_u(j)|}{k}$$ of the rating of target user $$u$$ for item $$j$$.

$$
\hat{r}*{uj}=b^{user}*u+\dfrac{\Sigma*{v \in P\_u(j)} w^{user}*{vu} \cdot (r\_vj - b^{user}\_v)}{\sqrt{|P\_u(j)|}}
$$

$$\mu\_v$$ is replaced by a bias variable $$b\_u$$

$$
\hat{r}*{uj}=b^{user}*u+b^{item}*j+\dfrac{\Sigma*{v \in P\_u(j)} w^{user}*{vu} \cdot (r*{vj} - b^{user}\_v - b^{item} \_ j )}{\sqrt{|P\_u(j)|}}
$$

### Item-Based Nearest Neighbor Regression

$$
\hat{r}*{ut}=\sum*{j \in Q\_t(u)}w^{item}*{jt} \cdot r*{uj}
$$

$$
minJ\_t=\sum\_{u \in U\_t} (r\_{ut}-\hat{r}*{ut})^2=\sum*{u \in U\_t} (r\_{ut}-\sum\_{j \in Q\_t(u)} w^{item}*{jt} \cdot r*{uj})^2
$$

$$
min\sum\_{t=1}^n\sum\_{u \in U\_t}(r\_{ut} -\sum\_{j \in Q\_t(u)} w^{item}*{jt}\cdot r*{uj})^2
$$

$$
\hat{r}*{ut}=b^{user}*u+b^{item}*t+\dfrac{\Sigma*{j \in Q\_t(u)}w^{item}*{jt} \cdot (r*{uj}-b^{user}\_u-b^{item}\_j)}{\sqrt{|Q\_t(u)|}}
$$

### Combined Method

$$
\hat{r}*{uj}=b^{user}*{u}+b^{item}*j+\dfrac{\Sigma*{v \in P\_u(j)} w^{user}*{vu} \cdot (r*{vj}-B\_{vj})}{\sqrt{|P\_u(j)|}}+\dfrac{\Sigma\_{j\in Q\_t(u)} w^{item}*{jt} \cdot (r*{uj}-B\_{uj})}{\sqrt{|Q\_t(u)|}}
$$

$$
\
$$