Example : Finding weights and biases using Normal Equation

Here is how the Normal Equation finds the perfect line for your data without iterative guessing.

1. Define the Scenario

Imagine we want to predict a house's Price (in thousands of dollars) based on its Size (in hundreds of square feet). We have three data points:

• House 1: Size = 1, Price = 15
• House 2: Size = 2, Price = 20
• House 3: Size = 3, Price = 25

We want to fit a straight line equation:

Price = w × Size + b

2. Construct the Matrices

To use the Normal Equation, we must arrange our data into a feature matrix X and a target vector y.

To calculate the bias (b), we add a dummy feature column filled with 1s to our matrix X.

X =

1	1
1	2
1	3

,

y =

15

20

25

In matrix X, the first column represents the bias multiplier (1). The second column represents the house sizes.

3. Apply the Normal Equation

The Normal Equation solves for the parameter vector θ using:

θ = (X^TX)⁻¹ X^T y

First, we calculate the transpose X^T:

X^T =

1	1	1
1	2	3

Next, we multiply X^T by X:

X^TX =

1	1	1
1	2	3

×

1	1
1	2
1	3

=

3	6
6	14

Next, we find the inverse of (X^TX):

(X^TX)⁻¹ =

1

(3 × 14) − (6 × 6)

×

14	−6
−6	3

=

1

6

×

14	−6
−6	3

=

7 3	−1
−1	1 2

Next, we multiply X^T by y:

X^Ty =

1	1	1
1	2	3

×

15

20

25

=

60

130

Finally, we multiply (X^TX)⁻¹ by X^Ty to get θ:

θ =

7 3	−1
−1	1 2

×

60

130

=

(7/3 × 60) + (−1 × 130)

(−1 × 60) + (1/2 × 130)

=

140 − 130

−60 + 65

=

10

5

✅ Final Answer

The Normal Equation outputs θ =

10

5

meaning the optimal bias (b) is 10 and the weight (w) is 5, yielding the perfect prediction line:

Price = 5 × Size + 10

---

Why the Dummy Feature Column is Always Filled with 1s

Important Note

Note that the dummy feature column is always initialized to all 1s.

Why It Must Be All 1s

• Isolates the Bias: Any number multiplied by 1 remains itself.

b × 1 = b

• Maintains Consistency: It ensures the bias term is added equally to every single data point.
• Allows Matrix Math: It turns the separate addition step (wX + b) into a clean dot product multiplication.

What Happens With Other Numbers?

• If you used 0s: The bias term b would be multiplied by 0 and completely disappear from the equation.
• If you used 2s: The math would still solve, but the final output in your θ vector would be

b

2

instead of the actual bias.

Key Takeaway

By keeping it as a column of 1s, the Normal Equation can treat the bias exactly like any other weight weight, saving you from writing separate algebraic code for it.