I received an Intel NCS 2 from the Hackster Deep Learning Giveaway. I've used the original NCS on the PYNQ-Z2 and the Ultra96v2 using the NCSDK toolkit and I've just tried the NCS and NCS 2 on a RPi4 using the OpenVINO toolkit.
I ran a quick performance check on my RPi4 using the "benchmark_app" that is provided with the toolkit. The app just loads an image or images and runs inference continuously for a minute and measures the equivalent frame rate.
Here are the results for the NCS and NCS 2:
NCS on RPi4 (2GB)
NCS 2 on RPi4 (2GB)
So, a little better than 2.5x performance improvement.
I'd like to use the NCS 2 on the PYNQ-Z2 for my Embedded Vision project. The NCS 2 does not use the NCSDK but requires OpenVINO, so the first thing that I need to do is install the OpenVINO toolkit. There isn't a pre-built configuration for the PYNQ-Z2 like the Raspbian configuration for the RPi. There is a Linux configuration but I want to use OpenVINO with PYNQ which uses a stripped down Ubuntu 18.04 rootfs. I thought the best alternative was to build a configuration from scratch using this set of instructions: Arm7* Single Board Computers and the Intel® Neural Compute Stick 2 (Intel® NCS 2) https://software.intel.com/content/www/us/en/develop/articles/arm-sbc-and-ncs2.html .
It turns out that building the OpenVINO toolkit is quite a time consuming process - not for the faint at heart.
"Some of the toolkit’s dependencies do not have prebuilt ARMv7 binaries and need to be built from source – this can increase the build time significantly compared to other platforms. Preparing to build the toolkit requires the following steps:"
- Installing build tools
- Installing CMake from source
- Installing OpenCV from source
- Cloning the toolkit and installing the Inference Engine
I spent the better part of a couple of days going through the install/build process. Each major component (CMake, OpenCV, Inference Engine (dldt)) seemed to require 6 - 8 hours to build. It doesn't help that the Z7020 only has dual A9 processors and there is only 512MB of memory. It also didn't help that the installation instructions were slightly out of date and also had some typos. I worked through most of the issues myself but got stuck trying to install the Inference Engine (dldt). Luckily, one of the Intel OpenVINO guys helped me interpret the CMake error file and it again was due to an error in the instructions. I guess I'm not familiar enough with using git or I would have recognized the error. The other problem that I had was that the build process would always seem to hang near the end (80-90%). I had that problem on the Ultra96v2 running multiple threads due to running out of memory but I didn't expect it with dual processors - maybe a memory leak? I wasn't able to successfully install the OpenVINO toolkit on the Ultra96v2 PYNQ but if the PYNQ-Z2 installation works - I may try it again. Slightly different configuration as it's using 64 bit aarch64 vs 32 bit ARMv7. I'd like to be able to compare the performance of the NCS 2 in the different board configurations.
Okay, I started this post 7 days ago and I was still struggling with the install this morning. A significant problem is that the long build time makes figuring out issues very time consuming. I discovered that the only clean way to install CMake and OpenCV is to fully uninstall the prior versions in the PYNQ image before you build and install the new versions. I got to the point where I had successful builds of CMake, OpenCV, and the Inference Engine - but was still having some problems compiling some of the sample programs. It will take some figuring out what to fix in the CMake configuration file. I've temporarily decided to punt to save my sanity - I just want to see OpenVINO work on the PYNQ-Z2! I decided to try to use the pre-built Raspbian configuration to see if it would work. I got that from here: https://download.01.org/opencv/2020/openvinotoolkit/2020.1/l_openvino_toolkit_runtime_raspbian_p_2020.1.023.tgz.
The pre-built configuration includes OpenVINO and OpenCV 4.2.0. It seems that CMake 3.10 works with the sample applications - building OpenVINO requires a later version (I was building CMake 3.14.4). Things required some tweaking but I'm expecting that this will only be a temporary solution until I get the build from source working.
Long story but I got to the point where I could run the benchmark_app.
NCS on PYNQ-Z2
NCS 2 on PYNQ-Z2
The NCS performance is pretty much the same as on the RPi4 and the NCS 2 performance is about 10% worse. Probably somewhat to be expected.
I want to see if I can get this cobbled together configuration working with iPython in the Jupyter Notebooks. If I have any issues I probably won't use the NCS 2 with my project.