🔮 Object-detection in the browser!

Using TensorFlow.js (see here for our blog posts on the topic) we can use the camera on your device to detect objects and draw boxes around them! In order to do this, we need to request to access your camera.

When it's working, you will see something like the below (depending on what it's pointing at!)

Where we have black boxes drawn around what it thinks it sees; with the name of the thing above it, and in brackets, (roughly) an estimate of "how confident" the network is that what it's it is seeing. Note that the network itself is not perfect, and it will get confused and certianly won't detect everything (for example, it misses the second cup in this photo, as well as the book). But it gives a feeling for the kinds of things that cna be done!

Demo

Loading ...

Detectable things
person bicycle car motorcycle airplane bus train truck boat traffic light fire hydrant stop sign parking meter bench bird cat dog horse sheep cow elephant bear zebra giraffe backpack umbrella handbag tie suitcase frisbee skis snowboard sports ball kite baseball bat baseball glove skateboard surfboard tennis racket bottle wine glass cup fork knife spoon bowl banana apple sandwich orange broccoli carrot hot dog pizza donut cake chair couch potted plant bed dining table toilet tv laptop mouse remote keyboard cell phone microwave oven toaster sink refrigerator book clock vase scissors teddy bear hair drier toothbrush
Details

This code is a light-weight wrapper around coco-ssd from the TensorFlow.js examples page.