Contents

Automate Data Labeling using Temperature Differential

← Back to Projects


Data labeling for object detection/segmentation is expensive to acquire. In this post we show we can sped up this proesses of labeling data for certain objects using a thermal camera, and standard computer vision techniques. We can create mask or ROI annotations depending on annotations of interest.


Introduction

There are many popular dataset for computer vision, but for custom objects one might be required to collect and label data manually. For example, see below:



In this post I propose a new method that can be applied in two ways:

  • To speed up objects labeling process to train supervised computer vision models

  • For object localization in combination with a classification model, for a hybrid object detection approach. This can be useful in certain cases were you have a fixed camera, for example: refrigerator cameras, parking lot cameras, security cameras, etc.

We have some requirements for when this method can be used. Primarily, we require a specific setting where the objects of interest have a noticeable temperature difference. Such as:

  • Living things

  • Stove top cooking, ovens, or cooking in general

  • Items that can be cooled before collecting data

Here is the code used: Auto-label

You will need a Flir-camera specifically since it is crucial to be able to take an image with both RGB and Thermal sensors and calibrate overlay between the two images.


Method

The method consists of the following steps.

Step 1: Fix the Flir thermal/RGB camera

  • It is possible to skip this step and take handheld images however you will need to be careful with your calibration.

Step 2: Specify expected temperature of item of interest

  • For example: 10C < temp > 30C or temp > 30C and classes name

Step 3: Run

  • Apply thermal image postprocessing: Bucket fill -> K-means -> Find contour, to obtain localization of objects.

  • Extract RGB image using python

  • Store RGB image and labeled image in desired format: ROI or PNG Mask


Stove Top Examples

Here we have an example applying our proposed method on food being cooked. This setting provides us a very natural temperature differential since the pan is almost always hotter than the food. Of course there are exceptions but this method is not meant to sutable for all settings, otherwise we wouldnt need any neural nets!



Cooling Down Objects

Here we have an example applying our proposed method on items that were left in the fridge for a few min. This creates the necessary temperature differential:



Humans or Animals

Finally mammals usually have a body temperature higher than the surroundings. In these settings our method can be applied naturally



Conclusion

Altough these are just a few examples you can use the code to test and try new settings. Applying what we have shown here can save you many hours of labeling data, or help enchace certain computer vision systems!