286 - Object detection using Mask RCNN: end-to-end from annotation to prediction
Vložit
- čas přidán 6. 09. 2022
- Code generated in the video can be downloaded from here:
github.com/bnsreenu/python_fo...
All other code:
github.com/bnsreenu/python_fo...
This video helps you with end-to-end Mask RCNN project, all the way from annotations to training to prediction. Handling of VGG and Coco style JSON annotations is demonstrated in the video. Code is also made available for both approaches.
For this video, I've used the annotation tool from www.makesense.ai/
You can try other annotation tools like:
www.makesense.ai/
labelstud.io/
github.com/Doodleverse/dash_d...
labelme.csail.mit.edu/Release3.0/
github.com/openvinotoolkit/cvat
www.robots.ox.ac.uk/~vgg/soft...
coco weights can be downloaded from: github.com/matterport/Mask_RC... - Věda a technologie
I have been watching your entire series on Mask RCNN this afternoon and learned so much from your video, Sreeni! Super grateful for your wonderful work here! I'll be following your code tomorrow step by step :)
Hey!
Your videos are always long enough, informative and beyond what anybody does. Very polite and understanding delivery. Kudos!
I appreciate that!
Keep on learning from your great lectures.
Your videos on Mask RCNN is great thank you!
Very useful presentation. Nice to see an end-to-end example!
Glad it was helpful!
Fantastic video! I'm going to run through the code in detail next. Many thanks.
Thank you for the presentation, makes the whole thing so much clearer.
Glad it was helpful!
highly informative presentation and explanation!!
Hello, I need to use a model to do semantic segmentation of human organs from CT scans (3D). I have an important question, do you think I should test both Mask RCNN and Unets? I am well documented on U nets since it seems to be used a lot for these medical tasks. I used Mask RCNN with Detectron2 in the past but have no clue how to code one from scratch. Do you think Mask RCNN is not well suited when you have only few data?
your presentation is nice and pretty good. would please show how to classify image(Remote sensing image such as sentinel 1,2 or land sat) by CNN in jupyter notebook
Hi,
Excellent explanation step by step to understand MaskRCNN. Pls, let us know how we can calculate the accuracy of generated mask instead of the mAP of object detection?
I'm sure the is more elegant ways of referencing paths than the crude form I employed here, but it fixed my problems. Running the code in full, no problem was detected. It just wasn't generating the weights. Going line by line or groups of lines by group of lines, following the video, I corrected some things that were wrong, printing things to see were the paths were pointing to. My changes are crude, so you will have to manually change the paths, as your path will not be like mine. All this on the coco style
In my file 'F:\joaopedro\downloads\python_for_microscopists-master\286-Object detection using mask RCNN - end to end\286-marbles_maskrcnn_coco_style_labels.py', I had to make the following changes to make it work:
line before change:
dataset_train.load_data('marble_dataset/train/labels/marbles_two_class_coco_json_format.json', 'marble_dataset/train')
line after change:
dataset_train.load_data('F:/joaopedro/downloads\python_for_microscopists-master/286-Object detection using mask RCNN - end to end/marble_dataset/train/labels/marbles_two_class_coco_json_format.json', 'F:/joaopedro/downloads\python_for_microscopists-master/286-Object detection using mask RCNN - end to end/marble_dataset/train')
similarly for the 'COCO_WEIGHTS_PATH = '... line, the line after the change became:
COCO_WEIGHTS_PATH = 'F:/joaopedro/downloads/python_for_microscopists-master/286-Object detection using mask RCNN - end to end/coco_weights/mask_rcnn_coco.h5'
Also added lines to save the weights (I'm not sure how Sreenivas was able to save it without those lines...), below the last line of the training section of the code, 'model.train(dataset_train, dataset_train, learning_rate=config.LEARNING_RATE, epochs=3, layers='heads')':
# Save the trained model weights
##print(os.path.join(DEFAULT_LOGS_DIR, "mask_rcnn_trained_weights.h5"))
model_path = os.path.join(DEFAULT_LOGS_DIR, "mask_rcnn_trained_weights.h5")
model.keras_model.save_weights(model_path)
print(f"Trained weights saved to {model_path}")
-==-=-
Now on the inference (which I put in a separate file), I've made these changes:
line before change:
model = MaskRCNN(mode='inference', model_dir='logs', config=cfg)
line after the change:
model = MaskRCNN(mode='inference', model_dir='F:/joaopedro/downloads/python_for_microscopists-master/286-Object detection using mask RCNN - end to end/logs', config=cfg)
line before change:
model.load_weights('logs/mask_rcnn_trained_weights.h5', by_name=True)
line after change:
model.load_weights('F:/joaopedro/downloads/python_for_microscopists-master/286-Object detection using mask RCNN - end to end/logs/mask_rcnn_trained_weights.h5', by_name=True)
-==-=-=-
Now on the section 'Show detected objects in color and all others in B&W' I changed 1 line (below '#line changed 1')and added another (below '#line added 2') to specify an output folder, otherwise it was outputting in my C: in my user. The changed code is below:
##############################################
#Show detected objects in color and all others in B&W
def color_splash(img, mask):
"""Apply color splash effect.
image: RGB image [height, width, 3]
mask: instance segmentation mask [height, width, instance count]
Returns result image.
"""
# Make a grayscale copy of the image. The grayscale copy still
# has 3 RGB channels, though.
gray = skimage.color.gray2rgb(skimage.color.rgb2gray(img)) * 255
# Copy color pixels from the original color image where mask is set
if mask.shape[-1] > 0:
# We're treating all instances as one, so collapse the mask into one layer
mask = (np.sum(mask, -1, keepdims=True) >= 1)
splash = np.where(mask, img, gray).astype(np.uint8)
else:
splash = gray.astype(np.uint8)
return splash
import skimage
def detect_and_color_splash(model, image_path=None, video_path=None):
assert image_path or video_path
# line added 2
output_dir = 'F:/joaopedro/downloads/python_for_microscopists-master/286-Object detection using mask RCNN - end to end/viz/'
# Image or video?
if image_path:
# Run model detection and generate the color splash effect
#print("Running on {}".format(img))
# Read image
img = skimage.io.imread(image_path)
# Detect objects
r = model.detect([img], verbose=1)[0]
# Color splash
splash = color_splash(img, r['masks'])
#line changed 1
file_name = output_dir + "splash_{:%Y%m%dT%H%M%S}.png".format(datetime.datetime.now())
skimage.io.imsave(file_name, splash)
elif video_path:
import cv2
# Video capture
vcapture = cv2.VideoCapture(video_path)
width = int(vcapture.get(cv2.CAP_PROP_FRAME_WIDTH))
height = int(vcapture.get(cv2.CAP_PROP_FRAME_HEIGHT))
fps = vcapture.get(cv2.CAP_PROP_FPS)
# Define codec and create video writer
file_name = "splash_{:%Y%m%dT%H%M%S}.avi".format(datetime.datetime.now())
vwriter = cv2.VideoWriter(file_name,
cv2.VideoWriter_fourcc(*'MJPG'),
fps, (width, height))
count = 0
success = True
while success:
print("frame: ", count)
# Read next image
success, img = vcapture.read()
if success:
# OpenCV returns images as BGR, convert to RGB
img = img[..., ::-1]
# Detect objects
r = model.detect([img], verbose=0)[0]
# Color splash
splash = color_splash(img, r['masks'])
# RGB -> BGR to save image to video
splash = splash[..., ::-1]
# Add image to video writer
vwriter.write(splash)
count += 1
vwriter.release()
print("Saved to ", file_name)
detect_and_color_splash(model, image_path="F:/joaopedro/downloads/python_for_microscopists-master/286-Object detection using mask RCNN - end to end/marble_dataset/val/test4.jpg")
######################################################
=-=-=-=-
thank you so much for this. I've been quite confused on how the weights were getting saved
Hi, thanks for the video. Can we get coco format output for prediction?
Thanks for video. I have trained MaskRNN model with my custom dataset. Could you tell me about how to measure large image where few objects span to another patch. In that case, objects metrics are not accurate.
I have learned a lot from your videos. Could you please upload any video on Cell Nuclei segmentation using Mask R-CNN?
Sir can you tell why are you loading the coco weights we have,nt trained the model yet and also coc o weights are not included in mask repo folder
I am getting the train mAP as 0.000. Could you please let me know what could be the reason behind this?
Hi Sreeni, Is the mask r-cnn model in the Tensorflow model zoo the same as Matterport?
Hi Sreeni i really like your videos and thanks a lot. But I have a scenario here if you want to detect whether a person is turning right or left or back what is the best approach you suggest?
Where can I find a description of the COCO file? And where do I find the requirements.txt information? Please also provide information on various auxiliary programs and versions that need to be installed.
nice tutorial, thank you.what's about the confusion matrix
Hi Sreeni sir, thank you for sharing mask rcnn workflow, I was using it, I am struggling to deploy it using redis as this model has custom objects and class, could you please help
amazing session, thnx a lot
can't find the code on Github?
Your video is totally informative, totally love it. However, I am having a small problem with the model= MaskRCNN line, it keeps telling me that type object "Config" has no attribute "image_shape". Can you please tell me what has gone wrong and what is the best way to fix it? Thanks a lot and I'm looking forward to your reply!!
Please make a video for object detection using Faster R-CNN from annotation to end.
I used this to train my custom dataset on Vehicle Damage Segmentation but it's results are very poor. I trained it for 100 epochs but still it cannot work correctly on training images. Please help me to solve this. Thank you in advance
sir please make a video about nnunet with costum dataset on colab.
What could be wrong if the training stops at the message "Epoch 1/30"? There is no loss information or anything loading like in your case. This occurs after model.train has been run.
The process where it gets stuck is:
get_next_batch() -> get() -> get() -> wait() -> wait() -> wait(), so I believe it doesn't get the required responses just yet or it's looping.
Hello sir... Wonderful lecture... I tried the same... Couldnt see where the trained weights are stored... So they are not loaded during the 'inference' and shows Train mAP value as 0.000. Please address the issue...
Great course! What is the best way for small object detection?
This works for small and large objects. Although, if your small objects are roundish, give StarDist a try.
Sir please give some content for point cloud data processing by deep learning methods
hey, just a question i have an issue with training maskrcnn on my custom dataset after reaching 100 epochs the training restart from the beginning any idea how to resolve this i followed the exact same tutorial. would really appreciate it im stuck any solution please
Hello bro I always get this error
load_image_gt() got an unexpected keyword argument 'use_mini_mask'
When I run the map score from inference part
Can you help me how I can solve this
I am constantly getting the error "You are using the default load_mask(), maybe you need to define your own one." But I already did custom it . Someone please help
How to train with binary mask?
Sir how we will calculate accuracy here..?
Hello, I've an error when I try to import the model. The error says that there is no module named 'keras.engine'. How can i fix it?
Can you make a video to do the whole process in cloud I mean google collab
Hi Sreeni, im following the tutorial and doing some modifications to adjust it to my needs, but im not able to use my system's GPU, do you have any recomendation? i have cuda 11.7 installed but i dont know if i need to do another extra step. Thanks for your content!
For that you have to upgrade the TF version but it has some dependencies on skimage so you cannot do it. Have you trained on CPU?
On Medium there is a good guide on how to install 'Install CUDA and CUDNN on Windows & Linux'. CUDNN had to be installed manually, as there is no installer for older versions.
You need to look up the compatibilities between the programs. My GPU is RTX 2060 Super.
cdnn used by me: cuDNN v7.6.5 (November 5th, 2019), for CUDA 10.1 (had to remove the developer link... but in the 'cuDNN Archive' you can find it)
cuda used: 10.1
python used: 3.7.11
tensorflow-2.2.0
Here you can find cuda and cdnn compatibility, search on google, as I cannot put links here: tensorflow tested_build_configurations
Another resource that gives a little bit of context, search on google: stack exchange Dlerror: cudnn64_7.dll not found
Does anyone knows why I cannot find the logs of the trained model? I created a logs folder but it is empty, what am I doing wrong? Please help
Similar problem here, on the coco style code. Did you find a solution? In my case not even the 'logs' folder is being created automatically, as I think is supposed to. Even puting an exact and direct path for the variable did not work.
Hi, Thanks for your video! I get a maP=0.0 while I was implementing this video. Do you know what may be the different? I would appreciate if you can share your mask_rcnn_marble_cfg_0003.h5 file to Github so that I can try to run and see if maP=0.0 to figure out my problem. Thanks!
Hello bro when I run the map I always get
load_image_gt() got an unexpected keyword argument 'use_mini_mask'
Can you tell me whether you also have got this type of error.And can you suggest me how to solve this
How much data is required for such kind of transfer learning? Is 40 - 50 images enough?
Usually 50 objects per class is a good starting point. The more the merrier!
ValueError: Error when checking input: expected input_image_meta to have shape (16,) but got array with shape (15,) Are anyguy have this ploblem ? And how can i fix it
It's late, but I'll post it, maybe it will help someone, perhaps the number of specified classes is different
NUM_CLASSES = 1 + count_your_classes
Dear Sir, I am using this code for my own image object detection(the image is .png format). However, I encountered a issue which confused me a lot. it is at detection stage, where "detected = model.detect([marbles_img])", and I got errors saying
"mold_image
return images.astype(np.float32) - config.MEAN_PIXEL
ValueError: operands could not be broadcast together with shapes (1024,1024,4) (3,)"
it seems that my image has 4 channels so I checked that png indeed has 4 channels. Therefore, I changed all my png files to jpg files and re ran the whole training process. However, when the code executes at the same location, it still reports the same errors.
Do you have any idea what caused this error? I appreciate your kind help. Thank you!
It is not saving the weights after training... no folder 'logs' is being created... Trying to find a solution, no apparent error message...
The coco style code.
I'm going to try running the VGG style to see.
I solved it. There is a comment explaining quickly what I did.
Does RCNN work like Yolo?
The actual way it works is different but if object segmentation is what you need then both Mask R-CNN and Yolo will do the job.
i am using google colab and got an error AttributeError: module 'tensorflow.python.framework.ops' has no attribute '_TensorLike'
please help me
I'm still waiting for your reply
I think that need use oldest version tensorflow.