The one hot encoding function in TensorflowJs: tf.oneHot()

DanielMarch 18, 2023

One hot encoding refers to transforming the representation of categories from using numeric codes such as 1, 2, 3 to using arrays such as [1, 0, 0], [1, 0, 0], [1, 0, 0] where the index of the value 1 represents the actual category.

An example of one hot encoding

Let's say we have a set of photos with pets and we need to group these pictures based on the main subject of the picture. And we want to do this with a neuronal network.

The categories are as follows: cats, dogs, and birds.

Remember that neuronal networks work much better with numbers instead of strings.

So, the initial temptation is to assign each category to a code represented by a numeric value:

cats  0      
dogs  1      
birds 2      

Our neuronal network will take some training data with photo objects and the code labeling the main object from that image:

const traingData = tf.tensor([photoA, photoB, photoC .... ])
const traingLabels = tf.tensor([0, 2, 2 .... ])
// in photoA we have a cat
// in photoB we have a bird
// in photoC we also have a bird and so on

This means that we expect our network to have just one output neuron, that gives a value representing what it thinks the image contains.

Well, this is our human brain speaking.

From a neuronal network point of view, when it comes to categorizing items, is much more efficient to have something like 3 output neurons where each value represents a probability of that object being in a category.

So, a better output can be:

[0.21, 0.88, 0.11]
// 21% this is a cat 
// 88% this is a dog
// 0.11% this is a bird

The category will be represented with an array where if we have the value 1 at a specific index means that we have a 100% chance that the photo contains the animat type.

cats   [1, 0 , 0]  // 100% cat, 0% dog, 0% bird
dogs   [0, 1 , 0]  // 0% cat, 100% dog, 0% bird
birds  [0, 0 , 1]  // 0% cat, 0% dog, 100% bird    

Therefore we will now represent the training data as follows:

const traingData = tf.tensor([photoA, photoB, photoC .... ])
const traingLabels = tf.tensor([
    [ 1, 0, 0 ],
    [ 0, 0, 1 ]
    [ 0, 0, 1 ]
    //...
])
// in photoA we have 100% cat , 0% dog, 0% bird
// in photoB we have 0% cat, 0% dog, 100% bird
// in photoC we also have 0% cat, 0% dog, 100% bird

If we would need to add one extra category we will increase the length of the resulting arrays:

cats    [1, 0 , 0, 0]   
dogs    [0, 1 , 0, 0]
birds   [0, 0 , 1, 0]
fishes  [0, 0 , 0, 1]    

Using the tf.oneHot() in TensorflowJs

The TensorflowJs library comes with the tf.oneHot built-in function for hot encoding.

So, if we want to one hot ecode a tensor we can do as follows:

const labels = tf.tensor1d([0,2,2], 'int32')
const oneHotLabes = tf.oneHot(labels, 3) // we have 3 labels in total
oneHotLabes.print()
// Tensor
//   [[1, 0, 0],
//    [0, 0, 1],
//    [0, 0, 1]]

Basically, for each value from within the labels tensor, the oneHot() function will create a corresponding tensor that:

• has a length of 3
• is filled with 0 values
• will place the value 1 at the index equal to the value read from the labels tensor

Note, that the oneHot() function requires a tensor of type int32, or we will get an error.

If we want to replace the 0 and 1 values from when the index matches or not the location we can use the onValue and offValue optional parameters:

const labels = tf.tensor1d([0,2], 'int32', 5, -1)
const oneHotLabes = tf.oneHot(labels, 3) 
oneHotLabes.print()
// Tensor
//   [[5, -1, -1],
//    [-1, -1, 5]]

Before you go be sure to check also the TensorflowJs argmax() as it directly relates to the hot encoding function.