OpenCVPythonTutorial

How to load Tensorflow models with OpenCV

Jean Vitor de Paulo
October 13, 2018
2 Mins Read
59 Comments

A very useful functionality was added to OpenCV’s DNN module: a Tensorflow net importer.




     To use the DNN, the opencv_contrib is needed, make sure to install it. This article is focused on the Python language, where the function has the following format:

1
cv2.dnn.readNetFromTensorflow('frozen_inference_graph.pb', 'graph.pbtxt')

As you might have seen, to use it, two files are needed:

  • frozen_inference_graph.pb
  • graph.pbtxt

 

About Tensorflow’s .pb and .pbtxt files

     Tensorflow models usually have a fairly high number of parameters. Freezing is the process to identify and save just the required ones (graph, weights, etc) into a single file that you can use later. So, in other words, it’s the TF way to “export” your model. The freezing process produces a Protobuf ( .pb) file.

     The good news is: There are a bunch of trained, optimized and widely used models on the Tensorflow’s detection model zoo repository that you can use freely. You won’t need to train one (if the available models, trained with well know datasets, fit your needs).

     Additionally, OpenCV requires an extra configuration file based on the .pb, the .pbtxt. It is possible to import your own models and generate your own .pbtxt files by using one of the following files from the OpenCV Github repository.

     If you want to use pre-trained models, the amazing OpenCV community already did the hard work for the following models.

 

ModelVersion
MobileNet-SSD v12017_11_17weightsconfig
MobileNet-SSD v1 PPN2018_07_03weightsconfig
MobileNet-SSD v22018_03_29weightsconfig
Inception-SSD v22017_11_17weightsconfig
Faster-RCNN Inception v22018_01_28weightsconfig
Faster-RCNN ResNet-502018_01_28weightsconfig
Mask-RCNN Inception v22018_01_28weightsconfig
Font: OpenCV’s Github wiki.

The Python code

The entire import/use procedure could be split into 5 steps:

  • Load your model using the downloaded files;
  • Load your images;
  • Use those images as network inputs;
  • Get the output with the detected objects.

 

Take a look at the Python code snippet for doing these steps:

Python
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
# How to load a Tensorflow model using OpenCV
# Jean Vitor de Paulo Blog - https://jeanvitor.com/tensorflow-object-detecion-opencv/
 
import cv2
 
# Load a model imported from Tensorflow
tensorflowNet = cv2.dnn.readNetFromTensorflow('frozen_inference_graph.pb', 'graph.pbtxt')
 
# Input image
img = cv2.imread('img.jpg')
rows, cols, channels = img.shape
 
# Use the given image as input, which needs to be blob(s).
tensorflowNet.setInput(cv2.dnn.blobFromImage(img, size=(300, 300), swapRB=True, crop=False))
 
# Runs a forward pass to compute the net output
networkOutput = tensorflowNet.forward()
 
# Loop on the outputs
for detection in networkOutput[0,0]:
    
    score = float(detection[2])
    if score > 0.2:
    
        left = detection[3] * cols
        top = detection[4] * rows
        right = detection[5] * cols
        bottom = detection[6] * rows
 
        #draw a red rectangle around detected objects
        cv2.rectangle(img, (int(left), int(top)), (int(right), int(bottom)), (0, 0, 255), thickness=2)
 
# Show the image with a rectagle surrounding the detected objects
cv2.imshow('Image', img)
cv2.waitKey()
cv2.destroyAllWindows()

 

Output examples

 

OpenCV+Tensorflow output OpenCV+Tensorflow output

 

 

 

Have fun with your projects! 
If this post helped you, please consider buying me a coffee 🙂

Tags:

Please share this article if you like it!

Share It!
Written By

Jean Vitor de Paulo

Other Articles

59 Comments

  3. hello, I’m getting the following error while trying to load a tensorflow model ssd mobilenet V2. Using the pretrained model in Model Zoo everything works fine, but after a fine tuning of the model I get error. Could it be the version of tensorflow used for finetuning? Currently I’m using Tf 1.15. Thanks

    Can’t create layer “FeatureExtractor/MobilenetV2/Conv/BatchNorm/FusedBatchNormV3” of type “FusedBatchNormV3” in function ‘getLayerInstance’

  4. hi all and thanks for help
    when i’m using my custom object detection

    tensorflowNet = cv2.dnn.readNetFromTensorflow(‘models/my_custom.pb’, ‘models/my_custom.pbtxt’)

    i get error :
    networkOutput = tensorflowNet.forward()
    cv2.error: OpenCV(4.1.1) C:\projects\opencv-python\opencv\modules\dnn\src\dnn.cp
    p:693: error: (-215:Assertion failed) inputs.size() == requiredOutputs in functi
    on ‘cv::dnn::dnn4_v20190621::DataLayer::getMemoryShapes’

  5. hey jean,i am using this code for my custom tensorflow graph of mobilenet ssd,but getting this error:

    error: OpenCV(4.2.0) C:\projects\opencv-python\opencv\modules\dnn\src\dnn.cpp:730: error: (-215:Assertion failed) inputs.size() == requiredOutputs in function ‘cv::dnn::dnn4_v20191202::DataLayer::getMemoryShapes’

    could you help me please.

      1. Hi Jean, may I know how to generate tensorflow .pb file to .pbtxt?
        I have yolo v3 model with cfg and weights file. I already convert them into .pb file (tensorflow), but now i need .pbtxt. How to i generate the .pbtxt file.?
        Thank you.
        Look for your reply soon~

  7. Hi Jean,

    Thank you for sharing!

    I am trying to use OpenCV to recognized a new object based on a trained mobilenet-ssd-v2 model, but always get an error. In Tensorflow it recognizes the new objects, but in OpenCV there is an error.

    I am unable to generate .pb and .pbtxt files to use in the cv.dnn.readNetFromTensorflow call.

    If you get the already made file, it works, but if you need to generate them, seems pretty impossible.

    Could you point me some resource to research how to solve this?

    Thank you,

  9. Traceback (most recent call last):
    File “c:/Users/RR/Desktop/project/project1.py”, line 7, in
    tensorflowNet = cv2.dnn.readNetFromTensorflow(‘frozen_inference_graph.pb’, ‘graph.pbtxt’)
    cv2.error: OpenCV(4.4.0) C:\Users\appveyor\AppData\Local\Temp\1\pip-req-build-h4wtvo23\opencv\modules\dnn\src\caffe\caffe_io.cpp:1133: error: (-2:Unspecified error) FAILED: fs.is_open(). Can’t open “frozen_inference_graph.pb” in function ‘cv::dnn::ReadProtoFromBinaryFile’

    what could be the reason for this error?

  10. Anyone has come across this issue:
    tensorflowNet = cv2.dnn.readNetFromTensorflow(
    cv2.error: OpenCV(4.4.0) /private/var/folders/nz/vv4_9tw56nv9k3tkvyszvwg80000gn/T/pip-req-build-wv7rsg8n/opencv/modules/dnn/src/tensorflow/tf_importer.cpp:672: error: (-215:Assertion failed) const_layers.insert(std::make_pair(name, li)).second in function ‘addConstNodes’
    ???
    Please any hints?
    Thanks in advance

Leave a Reply

You Might Also Like

This site uses cookies. By using this site, you agree to the Privacy Policy and Terms of Use.

Accept