Spurious error when running Jupyter notebook, code runs fine in terminal

Notebook
1

My code runs perfectly fine from the command-line, but it fails with a spurious error when it is run from Jupyter notebook. The error is:

—————————————————————————————————————
error                                     Traceback (most recent call last)
<ipython-input-32-efc5ccb15aed> in <module>
      1 im = cv2.imread(DATA_PATH + 'images/street.jpg')
----> 2 objects = detect_objects(net, im)
      3 display_objects(im, objects)

<ipython-input-31-a2dc02cd6a3d> in detect_objects(net, im)
     14 
     15     # Peform Prediction
---> 16     objects = net.forward()
     17     return objects

error: OpenCV(4.1.2) /Users/brisance/opencv/modules/dnn/src/layers/concat_layer.cpp:95: error: (-201:Incorrect size of input array) Inconsistent shape for ConcatLayer in function 'getMemoryShapes'

The dependencies are numpy (v1.17.3), OpenCV (v4.1.2) and matplotlib (v3.1.1). My set up is as follows:

ProductName:    Mac OS X
ProductVersion: 10.15.1
BuildVersion:   19B88

Python version 3.7.5 (default, Nov  1 2019, 02:16:23) 
[Clang 11.0.0 (clang-1100.0.33.8)]

General configuration for OpenCV 4.1.2 =====================================
  Version control:               4.1.2

  Extra modules:
    Location (extra):            /Users/brisance/opencv_contrib/modules
    Version control (extra):     4.1.2

  Platform:
    Timestamp:                   2019-10-29T07:32:45Z
    Host:                        Darwin 19.0.0 x86_64
    CMake:                       3.15.4
    CMake generator:             Unix Makefiles
    CMake build tool:            /usr/bin/make
    Configuration:               RELEASE

  CPU/HW features:
    Baseline:                    SSE SSE2 SSE3 SSSE3 SSE4_1
      requested:                 DETECT
    Dispatched code generation:  SSE4_2 FP16 AVX AVX2 AVX512_SKX
      requested:                 SSE4_1 SSE4_2 AVX FP16 AVX2 AVX512_SKX
      SSE4_2 (2 files):          + POPCNT SSE4_2
      FP16 (1 files):            + POPCNT SSE4_2 FP16 AVX
      AVX (5 files):             + POPCNT SSE4_2 AVX
      AVX2 (29 files):           + POPCNT SSE4_2 FP16 FMA3 AVX AVX2
      AVX512_SKX (6 files):      + POPCNT SSE4_2 FP16 FMA3 AVX AVX2 AVX_512F AVX512_COMMON AVX512_SKX

  C/C++:
    Built as dynamic libs?:      YES
    C++ Compiler:                /Library/Developer/CommandLineTools/usr/bin/c++  (ver 11.0.0.11000033)
    C++ flags (Release):         -fsigned-char -W -Wall -Werror=return-type -Werror=non-virtual-dtor -Werror=address -Werror=sequence-point -Wformat -Werror=format-security -Wmissing-declarations -Wmissing-prototypes -Wstrict-prototypes -Wundef -Winit-self -Wpointer-arith -Wshadow -Wsign-promo -Wuninitialized -Winit-self -Wno-delete-non-virtual-dtor -Wno-unnamed-type-template-args -Wno-comment -fdiagnostics-show-option -Wno-long-long -Qunused-arguments -Wno-semicolon-before-method-body -ffunction-sections -fdata-sections  -fvisibility=hidden -fvisibility-inlines-hidden -O3 -DNDEBUG  -DNDEBUG
    C++ flags (Debug):           -fsigned-char -W -Wall -Werror=return-type -Werror=non-virtual-dtor -Werror=address -Werror=sequence-point -Wformat -Werror=format-security -Wmissing-declarations -Wmissing-prototypes -Wstrict-prototypes -Wundef -Winit-self -Wpointer-arith -Wshadow -Wsign-promo -Wuninitialized -Winit-self -Wno-delete-non-virtual-dtor -Wno-unnamed-type-template-args -Wno-comment -fdiagnostics-show-option -Wno-long-long -Qunused-arguments -Wno-semicolon-before-method-body -ffunction-sections -fdata-sections  -fvisibility=hidden -fvisibility-inlines-hidden -g  -O0 -DDEBUG -D_DEBUG
    C Compiler:                  /Library/Developer/CommandLineTools/usr/bin/cc
    C flags (Release):           -fsigned-char -W -Wall -Werror=return-type -Werror=non-virtual-dtor -Werror=address -Werror=sequence-point -Wformat -Werror=format-security -Wmissing-declarations -Wmissing-prototypes -Wstrict-prototypes -Wundef -Winit-self -Wpointer-arith -Wshadow -Wsign-promo -Wuninitialized -Winit-self -Wno-delete-non-virtual-dtor -Wno-unnamed-type-template-args -Wno-comment -fdiagnostics-show-option -Wno-long-long -Qunused-arguments -Wno-semicolon-before-method-body -ffunction-sections -fdata-sections  -fvisibility=hidden -fvisibility-inlines-hidden -O3 -DNDEBUG  -DNDEBUG
    C flags (Debug):             -fsigned-char -W -Wall -Werror=return-type -Werror=non-virtual-dtor -Werror=address -Werror=sequence-point -Wformat -Werror=format-security -Wmissing-declarations -Wmissing-prototypes -Wstrict-prototypes -Wundef -Winit-self -Wpointer-arith -Wshadow -Wsign-promo -Wuninitialized -Winit-self -Wno-delete-non-virtual-dtor -Wno-unnamed-type-template-args -Wno-comment -fdiagnostics-show-option -Wno-long-long -Qunused-arguments -Wno-semicolon-before-method-body -ffunction-sections -fdata-sections  -fvisibility=hidden -fvisibility-inlines-hidden -g  -O0 -DDEBUG -D_DEBUG
    Linker flags (Release):      -Wl,-dead_strip  
    Linker flags (Debug):        -Wl,-dead_strip  
    ccache:                      NO
    Precompiled headers:         NO
    Extra dependencies:          /Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/System/Library/Frameworks/OpenGL.framework
    3rdparty dependencies:

  OpenCV modules:
    To be built:                 aruco bgsegm bioinspired calib3d ccalib core cvv datasets dnn dnn_objdetect dnn_superres dpm face features2d flann fuzzy gapi hfs highgui img_hash imgcodecs imgproc line_descriptor ml objdetect optflow phase_unwrapping photo plot python2 python3 quality reg rgbd saliency shape stereo stitching structured_light superres surface_matching text tracking ts video videoio videostab xfeatures2d ximgproc xobjdetect xphoto
    Disabled:                    world
    Disabled by dependency:      -
    Unavailable:                 cnn_3dobj cudaarithm cudabgsegm cudacodec cudafeatures2d cudafilters cudaimgproc cudalegacy cudaobjdetect cudaoptflow cudastereo cudawarping cudev freetype hdf java js matlab ovis sfm viz
    Applications:                tests perf_tests examples apps
    Documentation:               NO
    Non-free algorithms:         NO

  GUI: 
    QT:                          YES (ver 5.13.1)
      QT OpenGL support:         YES (Qt5::OpenGL 5.13.1)
    Cocoa:                       YES
    OpenGL support:              YES (/Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/System/Library/Frameworks/OpenGL.framework /Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/System/Library/Frameworks/OpenGL.framework)
    VTK support:                 NO

  Media I/O: 
    ZLib:                        build (ver 1.2.11)
    JPEG:                        build-libjpeg-turbo (ver 2.0.2-62)
    WEBP:                        build (ver encoder: 0x020e)
    PNG:                         build (ver 1.6.37)
    TIFF:                        build (ver 42 - 4.0.10)
    JPEG 2000:                   build (ver 1.900.1)
    OpenEXR:                     build (ver 2.3.0)
    HDR:                         YES
    SUNRASTER:                   YES
    PXM:                         YES
    PFM:                         YES

  Video I/O:
    DC1394:                      NO
    FFMPEG:                      NO
      avcodec:                   NO
      avformat:                  NO
      avutil:                    NO
      swscale:                   NO
      avresample:                NO
    GStreamer:                   NO
    AVFoundation:                YES
    v4l/v4l2:                    NO

  Parallel framework:            GCD

  Trace:                         YES (with Intel ITT)

  Other third-party libraries:
    Intel IPP:                   2019.0.0 Gold [2019.0.0]
           at:                   /Users/brisance/opencv/build/3rdparty/ippicv/ippicv_mac/icv
    Intel IPP IW:                sources (2019.0.0)
              at:                /Users/brisance/opencv/build/3rdparty/ippicv/ippicv_mac/iw
    Lapack:                      YES (/Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/System/Library/Frameworks/Accelerate.framework)
    Eigen:                       NO
    Custom HAL:                  NO
    Protobuf:                    build (3.5.1)

  OpenCL:                        YES (no extra features)
    Include path:                NO
    Link libraries:              -framework OpenCL

  Python 2:
    Interpreter:                 /usr/bin/python2.7 (ver 2.7.16)
    Libraries:                   /usr/lib/libpython2.7.dylib (ver 2.7.16)
    numpy:                       /System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/numpy/core/include (ver 1.8.0rc1)
    install path:                lib/python2.7/site-packages

  Python 3:
    Interpreter:                 /usr/local/bin/python3 (ver 3.7.4)
    Libraries:                   /usr/local/Frameworks/Python.framework/Versions/3.7/lib/libpython3.7m.dylib (ver 3.7.4)
    numpy:                       /usr/local/lib/python3.7/site-packages/numpy/core/include (ver 1.17.2)
    install path:                /Users/brisance/OpenCV-4.1.2-py3/lib/python3.7/site-packages/

  Python (for build):            /usr/bin/python2.7

  Java:                          
    ant:                         NO
    JNI:                         /Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/System/Library/Frameworks/JavaVM.framework/Headers /Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/System/Library/Frameworks/JavaVM.framework/Headers /Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/System/Library/Frameworks/JavaVM.framework/Headers
    Java wrappers:               NO
    Java tests:                  NO

  Install to:                    /Users/brisance/installation/OpenCV-4.1.2

-----------------------------------------------------------------

How do I go about tracking down the source of this bug?

2

It’s a opencv errror.
You can search the opencv error message to find the answer.

3

Hello… thank you for replying.

If it’s an OpenCV error, it would not run successfully from the terminal, no? All the dependencies are the same.

4

Sounds like a state issue. Have you tried restarting kernel & then executing everything again?

5

Hello, thanks for replying.

Yes, I’ve restarted the kernel. Even quit the process and rebooted the computer. Still experiencing the same error.

6

No obvious ideas; however, if you really need to track it down here are some directions to consider…

A comment to this blog post linked that error to memory running out, it seems. Is it possible the extra overhead of running the notebook happens to trigger it while you skate by on the command line? Is it possible to try both places with less a demanding version of your task?

If it isn’t what you are doing already, does your code work if you run %run <your_script_name>.py or !python <your_script_name>.py from inside the notebook?

What you posted for your set up doesn’t show your notebook situation as far as I can tell. There are a lot of ways to run a notebook. If you run !pip list inside the notebook do you see the same versions for the packages as you posted in your question? There are a lot of other suggestions where this comment is found for looking into things. In particular, such as running !which python, !jupyter troubleshoot, and import sys; sys.path run in the notebook might help you understand if both the Python running when you use the command line and that in the notebook kernel are seeing the same thing, as you seem to expect.

Test if it is your notebook installation or not by running your code from sessions launched via the launch binder badge here where I have the OpenCV version able to run without a monitor attached installed. This might require you use a less of your data in order to upload it to the running session. If anything else you need doesn’t seem installed run %pip install <package> or %conda install <package> to install the package to the session and try again. You may need to restart the kernel after the install.

1 Like
7

Hello, thank you for replying and the helpful suggestions. I learned how to troubleshoot Jupyter because of your reply.

A comment to this blog post linked that error to memory running out, it seems. Is it possible the extra overhead of running the notebook happens to trigger it while you skate by on the command line? Is it possible to try both places with less a demanding version of your task?

I ran the exact same code on my MBP with 8 GB of RAM, and my iMac with 32 GB of RAM. Both gave the same error.

If it isn’t what you are doing already, does your code work if you run %run <your_script_name>.py or !python <your_script_name>.py from inside the notebook?

No, it does not work if I do that. Gives the same error.

What you posted for your set up doesn’t show your notebook situation as far as I can tell. There are a lot of ways to run a notebook. If you run !pip list inside the notebook do you see the same versions for the packages as you posted in your question? There are a lot of other suggestions where this comment is found for looking into things. In particular, such as running !which python , !jupyter troubleshoot , and import sys; sys.path run in the notebook might help you understand if both the Python running when you use the command line and that in the notebook kernel are seeing the same thing, as you seem to expect.

I compared the packages, they are identical. Here is the output when I run !jupyter troubleshoot

$PATH:
/Users/brisance/.virtualenvs/OpenCV-4.1.2-py3/bin
/Applications/Racket v7.4/bin
/usr/local/bin
/usr/bin
/bin
/usr/sbin
/sbin

sys.path:
/Users/brisance/.virtualenvs/OpenCV-4.1.2-py3/bin
/usr/local/Cellar/python/3.7.4_1/Frameworks/Python.framework/Versions/3.7/lib/python37.zip
/usr/local/Cellar/python/3.7.4_1/Frameworks/Python.framework/Versions/3.7/lib/python3.7
/usr/local/Cellar/python/3.7.4_1/Frameworks/Python.framework/Versions/3.7/lib/python3.7/lib-dynload
/Users/brisance/.virtualenvs/OpenCV-4.1.2-py3/lib/python3.7/site-packages

sys.executable:
/Users/brisance/.virtualenvs/OpenCV-4.1.2-py3/bin/python3

sys.version:
3.7.4 (default, Sep  7 2019, 18:27:02) 
[Clang 10.0.1 (clang-1001.0.46.4)]

platform.platform():
Darwin-18.7.0-x86_64-i386-64bit

which -a jupyter:
/Users/brisance/.virtualenvs/OpenCV-4.1.2-py3/bin/jupyter

pip list:
Package            Version
------------------ -------
appnope            0.1.0  
attrs              19.3.0 
backcall           0.1.0  
bleach             3.1.0  
cycler             0.10.0 
decorator          4.4.1  
defusedxml         0.6.0  
entrypoints        0.3    
imageio            2.6.1  
importlib-metadata 0.23   
ipykernel          5.1.3  
ipython            7.9.0  
ipython-genutils   0.2.0  
ipywidgets         7.5.1  
jedi               0.15.1 
Jinja2             2.10.3 
joblib             0.14.0 
jsonschema         3.1.1  
jupyter            1.0.0  
jupyter-client     5.3.4  
jupyter-console    6.0.0  
jupyter-core       4.6.1  
kiwisolver         1.1.0  
MarkupSafe         1.1.1  
matplotlib         3.1.1  
mistune            0.8.4  
more-itertools     7.2.0  
nbconvert          5.6.1  
nbformat           4.4.0  
networkx           2.4    
notebook           6.0.2  
numpy              1.17.3 
pandocfilters      1.4.2  
parso              0.5.1  
pexpect            4.7.0  
pickleshare        0.7.5  
Pillow             6.2.1  
pip                19.3.1 
prometheus-client  0.7.1  
prompt-toolkit     2.0.10 
ptyprocess         0.6.0  
Pygments           2.4.2  
pyparsing          2.4.4  
pyrsistent         0.15.5 
python-dateutil    2.8.1  
PyWavelets         1.1.1  
pyzmq              18.1.0 
qtconsole          4.5.5  
scikit-image       0.16.2 
scikit-learn       0.21.3 
scipy              1.3.1  
Send2Trash         1.5.0  
setuptools         40.8.0 
six                1.13.0 
terminado          0.8.2  
testpath           0.4.4  
tornado            6.0.3  
traitlets          4.3.3  
wcwidth            0.1.7  
webencodings       0.5.1  
wheel              0.33.6 
widgetsnbextension 3.5.1  
zipp               0.6.0

The MBP is running Catalina, while the iMac is running Mojave.