Contents

# 1. 基本形式

一元线性回归的基本形式为：

$$f(x) = w x + b \tag{1}$$

$$J(w, b) = \frac{1}{m} \sum_{i=1}^m [y_i – f(x_i)]^2 = \frac{1}{m} \sum_{i=1}^m (y_i – w x_i – b_i)^2 \tag{2}$$

# 2. 参数求解

最佳的 $w$ 和 $b$ 会最小化均方误差，即

$$(w^*, b^*) = \underset{(w, b)}{\arg\min} \; J(w, b) \tag{3}$$

式 $(2)$ 分别为 $w$ 和 $b$ 求偏导，得

\begin{aligned} \frac{\partial J}{\partial w} &= \sum_{i=1}^m 2(y_i – w x_i + b)(-x) \\ &= 2 \bigg( w \sum_{i=1}^m x_i^2 – \sum_{i=1}^m (y_i – b) x_i \bigg) \end{aligned} \tag{4}

\begin{aligned} \frac{\partial J}{\partial b} &= \sum_{i=1}^m 2(y_i – w x_i + b)(-1) \\ &= 2 \bigg( mb – \sum_{i=1}^m (y_i – w x_i) \bigg) \end{aligned} \tag{5}

$$b = \frac{1}{m} \sum_{i=1}^m (y_i – w x_i) = \overline{y} – w \overline{x} \tag{6}$$

$$w \sum_{i=1}^m x_i^2 – \sum_{i=1}^m (y_i – b) x_i = 0$$

$$w \sum_{i=1}^m x_i^2 – \sum_{i=1}^m (y_i – \overline{y} + w \overline{x}) x_i = 0$$

$$w \sum_{i=1}^m x_i^2 – \sum_{i=1}^m y_i x_i + \overline{y} \sum_{i=1}^m x_i – w \overline{x} \sum_{i=1}^m x_i = 0$$

$$w \bigg( \sum_{i=1}^m x_i^2 – \overline{x} \sum_{i=1}^m x_i \bigg) = \sum_{i=1}^m y_i x_i – \overline{y} \sum_{i=1}^m x_i$$

$$w \bigg( \sum_{i=1}^m x_i^2 – \frac{1}{m} \sum_{i=1}^m x_i \sum_{i=1}^m x_i \bigg) = \sum_{i=1}^m y_i x_i – \sum_{i=1}^m y_i \overline{x}$$

$$w = \frac{\sum\limits_{i=1}^m y_i (x_i – \overline{x})}{\sum\limits_{i=1}^m x_i^2 – \frac{1}{m} \bigg( \sum\limits_{i=1}^m x_i \bigg)^2 } \tag{7}$$