Weights and the weighted sum

DanielFebruary 23, 2023March 8, 2023

We have seen in the post about perceptrons in Javascript that the total input of a neuron is calculated by using the weighted sum.

But, why do we need the weights for the inputs?

In the end, training a neural network to make predictions involves tweaking these weights of the neurons multiple times so that the rate of predictions success becomes better and better.

Imagine that you need to build a TensorflowJs model that will predict the price of a house based on the following inputs:

• the total size of the house
• the year when it was built
• the distance from the city center

Each of these inputs matters, but in a different proportion. Therefore we will assign a weight to each input.

totalInput = inputSize*weightSize + inputYear*weightYear + inputDistance*weightDistance

For example, the same applies when we build a model that recognizes animals in a photo. We can say that the inputs are the shape and size of the head, the body, and the tail. And in the case of a kangaroo, the tail input will have a much bigger weight than in the case of a bear.

By the way, small random values are the best initialization values for the weights of a neural network.

Calculating the weighted sum in TensorflowJs

A Javascript function that calculates the weighted sum can look like this:

const weightedSum = (inputs, weights)=> {
  let wSum = 0
  inputs.forEach((val, i) => {
    wSum = wSum + weights[i]*val
  })
  return wSum
}

weightedSum([1, 2, 3, 4], [0.2, 0.5, 0.1, 0])
// 1.5

Given that in TensorflowJs we have a built set of mathematical operations we can easily do:

const inputs = tf.tensor([1, 2, 3, 4])
const weights = tf.tensor([0.2, 0.5, 0.1, 0])
const weightedSum = tf.sum(inputs.mul(weights))
weightedSum.print(); // 1.5

Before you go, be sure to take a look also at the bias neuron, as it is often the case to add it to the total weighted sum.