One hot encoding refers to transforming the representation of categories from using numeric codes such as 1, 2, 3 to using arrays such as [1, 0, 0], [1, 0, 0], [1, 0, 0] where the index of the value 1 represents the actual category.
An example of one hot encoding
Let's say we have a set of photos with pets and we need to group these pictures based on the main subject of the picture. And we want to do this with a neuronal network.
The categories are as follows:
Remember that neuronal networks work much better with numbers instead of strings.
So, the initial temptation is to assign each category to a code represented by a numeric value:
cats 0 dogs 1 birds 2
Our neuronal network will take some training data with photo objects and the code labeling the main object from that image:
const traingData = tf.tensor([photoA, photoB, photoC .... ]) const traingLabels = tf.tensor([0, 2, 2 .... ]) // in photoA we have a cat // in photoB we have a bird // in photoC we also have a bird and so on
This means that we expect our network to have just one output neuron, that gives a value representing what it thinks the image contains.
Well, this is our human brain speaking.
From a neuronal network point of view, when it comes to categorizing items, is much more efficient to have something like 3 output neurons where each value represents a probability of that object being in a category.
So, a better output can be:
[0.21, 0.88, 0.11] // 21% this is a cat // 88% this is a dog // 0.11% this is a bird
The category will be represented with an array where if we have the value 1 at a specific index means that we have a 100% chance that the photo contains the animat type.
cats [1, 0 , 0] // 100% cat, 0% dog, 0% bird dogs [0, 1 , 0] // 0% cat, 100% dog, 0% bird birds [0, 0 , 1] // 0% cat, 0% dog, 100% bird
Therefore we will now represent the training data as follows:
const traingData = tf.tensor([photoA, photoB, photoC .... ]) const traingLabels = tf.tensor([ [ 1, 0, 0 ], [ 0, 0, 1 ] [ 0, 0, 1 ] //... ]) // in photoA we have 100% cat , 0% dog, 0% bird // in photoB we have 0% cat, 0% dog, 100% bird // in photoC we also have 0% cat, 0% dog, 100% bird
If we would need to add one extra category we will increase the length of the resulting arrays:
cats [1, 0 , 0, 0] dogs [0, 1 , 0, 0] birds [0, 0 , 1, 0] fishes [0, 0 , 0, 1]
Using the tf.oneHot() in TensorflowJs
The TensorflowJs library comes with the tf.oneHot built-in function for hot encoding.
So, if we want to one hot ecode a tensor we can do as follows:
const labels = tf.tensor1d([0,2,2], 'int32') const oneHotLabes = tf.oneHot(labels, 3) // we have 3 labels in total oneHotLabes.print() // Tensor // [[1, 0, 0], // [0, 0, 1], // [0, 0, 1]]
Basically, for each value from within the
labels tensor, the
oneHot() function will create a corresponding tensor that:
- has a length of 3
- is filled with 0 values
- will place the value 1 at the index equal to the value read from the
Note, that the
oneHot() function requires a tensor of type
int32, or we will get an error.
If we want to replace the 0 and 1 values from when the index matches or not the location we can use the
offValue optional parameters:
const labels = tf.tensor1d([0,2], 'int32', 5, -1) const oneHotLabes = tf.oneHot(labels, 3) oneHotLabes.print() // Tensor // [[5, -1, -1], // [-1, -1, 5]]
Before you go be sure to check also the TensorflowJs argmax() as it directly relates to the hot encoding function.