Instance Segmentation in PyTorch | Mask RCNN

Please check the pinned comment for important information.
This video is about instance Segmentation. We will use Mask RCNN to segment images. This model was trained on the COCO dataset.
Notebook Link: colab.research.google.com/dri...
Here is the Coco classes
[ '__background__', 'person', 'bicycle', 'car', 'motorcycle', 'airplane', 'bus', 'train', 'truck', 'boat', 'traffic light', 'fire hydrant', 'N/A', 'stop sign', 'parking meter', 'bench', 'bird', 'cat', 'dog', 'horse', 'sheep', 'cow',
'elephant', 'bear', 'zebra', 'giraffe', 'N/A', 'backpack', 'umbrella', 'N/A', 'N/A',
'handbag', 'tie', 'suitcase', 'frisbee', 'skis', 'snowboard', 'sports ball', 'kite', 'baseball bat', 'baseball glove', 'skateboard', 'surfboard', 'tennis racket',
'bottle', 'N/A', 'wine glass', 'cup', 'fork', 'knife', 'spoon', 'bowl', 'banana', 'apple', 'sandwich', 'orange', 'broccoli', 'carrot', 'hot dog', 'pizza',
'donut', 'cake', 'chair', 'couch', 'potted plant', 'bed', 'N/A', 'dining table', 'N/A', 'N/A', 'toilet', 'N/A', 'tv', 'laptop', 'mouse', 'remote', 'keyboard', 'cell phone', 'microwave', 'oven', 'toaster', 'sink', 'refrigerator', 'N/A', 'book',
'clock', 'vase', 'scissors', 'teddy bear', 'hair drier', 'toothbrush']
COLORS list below
[[0, 255, 0],[0, 0, 255],[255, 0, 0],[0, 255, 255],[255, 255, 0],[255, 0, 255],[80, 70, 180], [250, 80, 190],[245, 145, 50],[70, 150, 250],[50, 190, 190]]
URL for image below
hips.hearstapps.com/hmg-prod....

[@programmingdatascienceandother]

It seems that the OpenCV API has changed slightly which causes the code shown in this video to break. Make sure to check the Colab notebook because I updated the code there.
colab.research.google.com/drive/1b3TwHdeWAgmZ7n1eOFbzVw1T0oZvC9U6?usp=sharing

[@antoinebuisson5595]

Thank you a lot for this tutorial, I understand how to use PyTorch much better now

[@programmingdatascienceandother]

Glad it helped!

[@user-co6pu8zv3v]

Thank you for great tutorial!

[@programmingdatascienceandother]

Glad it was helpful!

[@shahzebimtiaz1849]

This is the most excellent and comprehensive video about mask RCNN on the internet! Can you please guide how can we train our custom dataset using this notebook? I have my training and testing dataset ready with the annotations in a json file format. Any help/guide in this regard will be highly appreciated.

[@programmingdatascienceandother]

Great suggestion! I might make a video on that soon!

[@notimportant9018]

@@programmingdatascienceandother Hi Have you made a video on that? That would be excellent. Thanks a lot in advance. This was very helpful.

[@aivalo]

​@@programmingdatascienceandother Yes, It would be great. I´m waiting for that to use for my thesis.

[@programmingdatascienceandother]

Sorry for being very late, but I have a training video out now.

[@muhammadrezahaghiri]

@@programmingdatascienceandother May you leave a link?

[@abdelrahmanamr6752]

amazing video helped me alot but i actually have a problem when I try running it get tboxes however for masks all values are zeros

[@programmingdatascienceandother]

You get bounding boxes but no masks?

[@manuthvann5341]

thats impressive sir , but since m new to this m wondering if it could work with text line ? cause based on what i have seen in the coco classes you have written in the description box , there is no text line at all . Looking forward to hearing back from you . Thanks

[@programmingdatascienceandother]

Did you mean Optical Character Recognition? Right now, you should use a library like Tesseract.

[@manuthvann5341]

@@programmingdatascienceandother yeah sir

[@programmingdatascienceandother]

Only 5% of my viewers are subscribed, if you aren't subscribed, please like and subscribe so that I can continue to make content.
If you want to train a Object Detection Model, I now have a tutorial on that: czcams.com/video/Uc90rr5jbA4/video.html

[@AJCricketstar]

What editör is he using?

[@programmingdatascienceandother]

I am using Google Colab. It can be found here: colab.research.google.com

[@hitharthkadam4874]

Sir,may I know how we can see the accuracy of each object identified?

[@programmingdatascienceandother]

You might want to look into object detection metrics like Mean Average Precision (mAP).

[@hitharthkadam4874]

@@programmingdatascienceandother Thank you😊

[@beginnerssamayal8319]

Sir how to send batch of images in mark Rcnn not single

[@programmingdatascienceandother]

You would have to load all your images. Then resize them to the same size (e.g 1024 x 768). You can create a batch of images by just placing them in a torch tensor and pass that to the model directly. Note you will have change the functions that process the model output in order to get the segmented images.

[@applejuice5785]

Can I train a custom COCO dataset with this

[@aivalo]

No, it´s just pre-trained with the original COCO dataset

[@programmingdatascienceandother]

If you modified the model, you could train it on a different dataset

[@programmingdatascienceandother]

I now have a video on that. czcams.com/video/Uc90rr5jbA4/video.html