Skip navigation
> RoadTest Reviews

RoadTest the Raspberry Pi 4 Model B (2GB) - Review

Scoring

Product Performed to Expectations: 10
Specifications were sufficient to design with: 10
Demo Software was of good quality: 9
Product was easy to use: 9
Support materials were available: 10
The price to performance ratio was good: 10
TotalScore: 58 / 60
  • RoadTest: RoadTest the Raspberry Pi 4 Model B (2GB)
  • Buy Now
  • Evaluation Type: Development Boards & Tools
  • Was everything in the box required?: Yes - All parts described by the manufacturer for the package were present
  • Comparable Products/Other parts you considered: A direct comparison is made in the RoadTest to the Raspberry Pi 3B+. Other products from the Raspberry Pi Foundation and other small single-board computers that run Linux also exist.
  • What were the biggest problems encountered?: The Raspberry Pi 4 Model B only runs on Buster, the latest version of Raspbian Debian Linux. Software planned for use in the RoadTest that worked on previous Raspberry Pi computers using Stretch could not be installed until revisions were made.

  • Detailed Review:

    Thanks to element14 and the Raspberry Pi Foundation for selecting me to do this RoadTest.  It is a head to head comparison of the Raspberry Pi 4 Model B to the Pi Model 3B+ while performing face detection with OpenCV.  Benchmarks released with the Raspberry Pi 4B indicate increases in speed of 2 to 4 times in many instances.  Will improvements of this magnitude be realized in real world applications like face detection using OpenCV?  Is a 4 GB Pi 4B required for face detection using OpenCV with a small dataset or is 1 GB enough?  What is the sweet spot for training photo size?  In this RoadTest these questions will be addressed and more…

     

    Introduction

     

    UPDATE 30 August 2019:  Plots of CPU and memory utilization added

    UPDATE 31 August 2019: Clarified potential contribution of DDR4 RAM in Pi 4B

     

    There have been a number of benchmarks published, including this one at element 14 and this one in the Raspberry Pi Foundation published MagPi.  However, the Pi 4 is relatively new and information on the improvement gains from upgrading existing projects is limited.  PiCasso Adapting Art to Viewers is a project running OpenCV running on a Rasbperry Pi Model 3B+ that I recently published on element14.

    PiCasso Art Deluxe Project using Raspberry Pi Model 3B+

     

    It features two Raspberry Pi computers – a 3B+ to do facial recognition and a 3B to control the video output.  Two computers are necessary because of the computing demands of facial recognition. At the conclusion of the project it was noted that facial recognition frame rate was slow and would benefit from being sped up. In addition, there were a number of false identifications which would be interesting to explore further.

     

    Below are benchmarks plucked from the Magpi that illustrate the improvements a Pi 4B might be expected to deliver relative to a Pi 3B+.

    Raspberry Pi 3B+ and 4 Benchmarks

     

    This RoadTest will be a performance comparison using a small dataset facial recognition application and OpenCV.  Doubling the frame rate would be considered a significant improvement and from the table above it appears that this might be possible.  The RoadTest includes:

     

    • Unboxing and getting started
    • Identifying software incompatibilities between Rasbian “Buster” and the previous Debian version “Stretch” and any Python incompatibilities
    • Comparing frame rate (frames / second aka FPS) of facial recognition between Pi 4 and Pi 3B+
    • Examination of important parameters in facial recognition such as:
      • Number of faces in dataset
      • Number of training photos
      • Training photo size
      • Recognition frame resolution
      • Memory and CPU utilization
    • Summary of results

     

    Unboxing and Getting Started

     

    The Raspberry Pi arrived undamaged with the retail box well packaged in a larger element14 box.

    Raspberry Pi 4 Computer with 2 GB Ram

    As is widely known by now the Raspberry Pi 4 has a number of new and improved hardware features:

     

    • More and faster memory
    • Broadcom 2711, Quad-core Cortex-A72 64-bit SoC @ 1.5 GHz
    • VideoCore VI Graphics Processor
    • Gigabit Ethernet
    • Dual micro-HDMI outputs
    • USB-C power source
    • 2x USB 2.0 “Hi-Speed” ports
    • 2x USB 3.0 “Super-Speed” ports
    • Bluetooth Low Energy (BLE) 5.0
    • Dual Band Wireless LAN (2.4Ghz and 5Ghz) with IEEE 802.11 b/g/n/ac support with new on board antenna

     

    The Raspberry Pi is not ready to use out of the box.  Even those who are already using a Raspberry Pi may need to purchase the following:

     

    • Micro-HDMI adapter or cable
    • USB-C power cable
    • Enclosure (it helps to have a fan)
    • MicroSD card
    • 15 Watt Power Supply

     

    In addition, for this RoadTest a keyboard, mouse, and the official Raspberry Pi Camera Module V2 were used.

     

    Although there are known issues with the USB-C power, I did not experience them.  The Raspberry Pi 4 can run hot as can previous versions.  This was experienced and the solution used during the RoadTest is described below.

     

    Of the hardware changes, the one that I found most troublesome was the dual micro-HDMI outputs. While some may find dual HDMI useful I expect most users won’t.  My issue is that I use the same monitor for development on several devices and switch them frequently.  The HDMI cable is difficult to unplug from the monitor due to position and I elected to use an adapter for HDMI to micro-HDMI at the Raspberry Pi.  Micro-HDMI is inherently a weaker connection and after a while the adapter failed due to stresses put on it during plugging and unplugging.  In addition, it is just another uncommon cable / connection to have to buy.  The replacement adapter I purchased after the first failed has a short length of cable between the female HDMI and male micro-HDMI parts of the adapter and hopefully will last longer.

     

    All in all though, the upgrades are greatly welcomed.

     

    Operating System and Software

     

    The Raspberry Pi 4B requires the new Rasbian image Buster, which is based on the latest version of Debian Linux.  Python 3.7 is the version of Python 3.x which comes with Buster.  One of my concerns was whether OpenCV would run without too much drama on the Raspberry Pi 4 with these changes.

     

    Getting started instructions for the Raspberry Pi 4 Model B can be found here and Buster installed easily and without a problem. In addition to the normal changes in setup for keyboard and environment, be sure to enable the camera for face recognition.

     

    Unfortunately, installing OpenCV using the method described in my previous project broke during the Build and Install.  The problem took several weeks to resolve and is described here.    Users of the Pi 4 should be aware that not all software and methods that ran on Debian Stretch will run without issue on Buster.  And the Pi 4 will not run on Stretch.

     

    A recipe for building OpenCV on Rasbian Buster and the Raspberry Pi 4B is given here (my thanks and gratitude to the author). Be prepared to spend several hours downloading, compiling, linking, and installing.

     

    There are a couple of changes in the installation for the RoadTest from the one used in the PiCasso Art Deluxe Project:

     

    • Raspbian Buster rather than Stretch
    • Upgrades OpenCV to 4.1.0 from 3.2.0
    • Upgrades Python 3.7 from 3.5
    • Creates a virtual Python environment

     

    Therefore it is not a direct comparison to that project but the contrasts presented here between the Raspberry Pi 4 and the Raspberry Pi 3B+ are representative of what might be expected in similar OpenCV face detection projects with small datasets.

     

    Temperature Throttling

     

    The Pi was heavily loaded during the OpenCV build process and started to show the dreaded thermometer in the upper right hand corner indicating it was throttling back due to overheating.  In this post I describe the case and fan that was designed to overcome the problem.  The case is suitable for both the Raspberry Pi 4B and the Raspberry Pi 3B+. No temperature throttling was observed during the RoadTest.

    Raspberry Pi Enclosure with Openings for Fand and Camera

    Raspberry Pi 4 with Fan

     

    Test Design

     

    NOTE: The primary purpose of this RoadTest is to determine the relative increases in speed that might be obtained when doing face detection with small datasets on a Raspberry Pi 4.  The datasets have not been optimized, nor have the settings for OpenCV.  In addition this is not a tutorial on face detection although there should be enough information to replicate the tests.

     

    Both the Raspberry Pi 4B and the Raspberry Pi 3B+ ran the tests using the same microSD card and installed in the same case with the fan running to assure maximum speed.

     

    There is a treatise on facial recognition here.  The author does a great job of explaining and most of the tutorials and such for the Raspberry Pi on face recognition seem to be based on (or copies of) his work. Since there is no way I could improve on it please follow the link if interested in knowing more.

     

    The parameters examined during the test are:

     

    • Size of training photos
    • Training Dataset Size (Number of training photos)
    • Number of persons in frame being captured
    • Frame width evaluated from capture

     

    Data recorded  includes time to train, approximate frame rate, encoded dataset size, Raspberry Pi memory utilization and CPU usage.

     

    The primary training set has a total of 64 photos of seven different faces (me, the cast of Seinfeld, the Prime Minister of Denmark and Supergirl) with a resolution of 400 x 600 pixels.  The number of photos of each face varies from 22 to 4.  Training photos for each person are stored in their own individual folders where the encoder can find them.  The training set is relatively small and evaluation with a larger dataset would be interesting but was not done.

    Training Dataset Photos

     

    The influence of training photo size was examined at 50x75, 100x150, 200x300, 400x600, 800x1200, and 1200x1800 pixels on a subset of photos of me, 22 in all.  The full training set containing all the photos is sized 400x600.

    Training Photo Sizes

     

     

     

    During my previous experiments the size of the training photos was not examined and it was assumed that larger photos gave better results.  However, larger photos take longer to encode and the value of larger photos if any had not been examined.  A cursory review of information available on the internet revealed that there is a minimum size sometimes correlated to interpupillary distance in pixels. In the training photos used here the interpupillary distance is approximately one third of the width of the photo or a bit less.  So it is varying from something around 15 pixels (one third of 50 = 17) to as much as 400 pixels (one third of 1200 = 400).

     

    Interpupillary Distance

     

    The images captured during face recognition are resized before evaluation in order to keep frame rate up with most of the tests done at 500 pixels in width.  The impact of decreasing and increasing this value was evaluated by testing at 400, 500, 800, and 1200 pixels in width.  Decreasing width increases frame rate but presumably might reduce recognition accuracy for subjects further from the camera. Increasing width has the opposite influence.  The following screen shots show the resized frames used face recognition with training photos of 400x600 pixels.

    Frame Width = 400 pixels

    Frame Width = 500 pixels

    Frame Width = 800 pixels

    Frame Width = 1200 pixels

     

    The Task Manager was used to get a rough idea of CPU utilization and memory being used. These numbers are approximate in that they fluctuated some during the tests.  As well, the OS may have had tasks running at some points that would contribute to CPU utilization and memory usage and warp the results somewhat.  Python code was used to determine the time required to encode all of the training photos.   Results are shown in a terminal window as the test progresses and frames per second are calculated when the test is exited.

    Screen Capture with Task Manager

     

    Since the frame rate is dependent upon how many objects are identified as faces it was necessary for me to remain in the frame and relatively still at a fixed distance during testing to get consistent results.  Where testing was done on faces other than mine an Apple iPad with the photo was held in front of the camera at a fixed distance.  The test was stopped and the results were recorded after 30 seconds. 

     

    Raw Results

     

    The video below gives an idea of how the data was captured and how it looked on the display.

     

     

    The raw data for the two versions of the Raspberry Pi is given below.

    Raw Data from Tests

     

     

    Interpretation

     

    Training Photo Size

     

    The training photo size was varied from 50x75 to 1200x1800 pixels.  All photos were medium resolution JPEG.  There was some increase in CPU memory usage with the very largest photos that I cannot explain as shown in the plot below.

    Frame Rate

    It may be that there is a cache during encoding that is not released during face detection.  The primary impact of training photo size is on the encoding rate.  This is illustrated in the graph below showing photos / second encoded as a function of training photo size for the Raspberry Pi 4 (blue) and the Raspberry Pi 3B+ (orange).  A higher encoding rate is better.

     

    Encoding Rate

     

    The Raspberry Pi 4 is about twice as fast at encoding data as the Raspberry Pi 3B+.  Encoding speed is highly dependent on the photo size but size of the resulting encoded data used for face detection is relatively independent of the size of the training photo.  An increase in false detections seemed to be present with the very smallest training photos (50x75).  Further experiment is necessary but it appears that a sweet spot in terms of encoding time and accurate face detection may occur around 100x150 to 200x300 pixels. This is in keeping with some of the information found while researching photo size.  Larger photos in the range of 800x1200 and above appear to be overkill.

     

    Training Dataset Size

     

    The training dataset size was varied from 4 to 64 photos and evaluated.  In the chart below frame recognition rate is on the X axis with the blue bars representing the Pi 4 and orange bars representing the Pi 3B+.  All data is for a single face with the exception of a group detection of the Seinfeld cast containing 30 faces in the training dataset and marked on the chart.  A higher frame rate is better.

     

    Face Recognition Rate

    For a single face varying the number of photos in the training set had little (if any) impact on frames per second during face recognition.  The Pi 4 evaluated around 2.5 FPS with a 3.6 times increase in frame rate over the Pi 3B+ overall.

     

    Of interest is the influence of multiple faces in the recognition frame.  For 4 faces the frame rate dropped to 0.73 for the Raspberry Pi 4 while the Pi 3B+ dropped to 0.22 FPS.  The Pi 4 was around 3.3 times faster in this case.  This implies that the detection of a face and evaluating it takes more time than comparing it to the already encoded faces in the dataset (at least for the small datasets in use here).

     

    The average encoded size of the datasets (Y axis) is largely a function of the number of photos (X axis) as shown in the nearly straight line relationship plotted below.

    Encoded Dataset Size

    The encoded size averages 1.05 kB per photo with the OpenCV parameters used in this test, regardless of the size of the training photo itself.

     

    Frame Evaluation Width

     

    During face recognition the user can set the width of the frame being evaluated in pixels.  Setting the width smaller speeds up evaluation but a larger width captures more data, which may be important for faces at a distance.  The following chart shows the influence of frame size on frame rate.  The Pi 4 is the blue line and the Pi 3B+ is orange.  Higher frame rate is better.

     

    Frame Rate

     

    At a frame width of 400 pixels the frame rate is 3 FPS for the Raspberry Pi 4 and 0.8 for the Raspberry Pi 3B+.  It drops to 1.6 FPS for the Raspberry Pi 4 when frame width is 1200 pixels and 0.4 for the Raspberry Pi 3B+.  Subjectively the increased frame size also contributed to increased false recognitions with a busy background.  Accordingly frame width should be minimized where frame rate is important and busy backgrounds avoided.

     

    Other Observations

     

    Throughout the tests memory usage hovered around 300 MB during face recognition for both the Raspberry Pi 4 and the Pi 3B+.  One exception is when the frame evaluation width is increased as shown in the plot below.

    Memory Usage

    This would imply that much larger dataset could be handled even with a 1 GB Raspberry Pi.  As noted by Shabaz in the comments below the faster DDR4 and extra RAM is contributing to the speed increases observed earlier but the contribution was not quantified.

     

    Also interesting is that CPU percent utilization for the two computers was essentially the same and did not vary much from 39% on average when evaluating a single face.  Again, an exception was observed with frame evaluation width.

    CPU Utilization

    In the plot above using single faces an increasing frame width results in increasing CPU utilization.  When multiple faces were present the CPU utilization dropped - perhaps because frame rate dropped and the video screen was not being updated as frequently.

     

    Summary

     

    For the small dataset evaluated in this RoadTest:

     

    • The Raspberry Pi Model 4 encodes face recognition datasets roughly twice as fast as a Raspberry Pi Model 3B+
    • The Raspberry Pi Model 4 does face recognition 3.3 to 4 times faster than a Raspberry Pi Model 3B+ depending on circumstances

     

    The following applies to both Raspberry Pi models tested:

     

    • There appears to be a sweet spot for training photos at around 100x150 to 200x300 pixels in terms of encoding time and accuracy.  This should be examined further in a more controlled fashion.
    • For small datasets such as the one examined and smaller frame recognition size, a 1 GB Raspberry Pi has sufficient memory for the OpenCV configuration used.  Larger datasets appear possible even with the 1 GB model.  This is not to say that the larger Pi models, such as the 2 GB model used here do not contribute to some of the speed increase observed through for example caching but this was not quantified.  The DDR4 RAM present will also contribute to higher speed from the Raspberry Pi 4B.
    • Facial recognition slows significantly when multiple faces are present.
    • Evaluating large frames during face recognition significantly slows frames per second and should be done only when needed such as detection of faces at a distance.
    • Busy backgrounds are to be avoided in both the training dataset and the recognition environment when possible.
    • The algorithm is designed to work on faces pointed more or less directly at the camera as opposed to turned to the side or up and down.

     

    Problems were initially encountered getting the software to run and that was not entirely unexpected while the differences between the Raspbian Buster image and Stretch are resolved.  Users should be aware of this however, especially for code that is outside that supplied by the Raspberry Pi Foundations or isn’t widely used and updated by the community.

     

    The Raspberry Pi Model 4B delivered as promised and is a bargain.  Even with the software glitches experienced and minor hardware issues it is bound to be very popular and the glitches soon resolved.

     

    Useful Links

     

    Using OpenCV for Face Recognition on the Raspberry Pi

    OpenCV Resources on the Raspberry Pi

    Building OpenCV on Raspian Buster and the Raspberry Pi 4

    Raspberry Pi MagPi Benchmarks for the Raspberry Pi 4

    Element14 Benchmarks for the Raspberry Pi 4

    Element14 Picasso Design Challenge: PiCasso Art Deluxe implementing OpenCV

    Getting started with the Raspberry Pi 4


Comments

Also Enrolling

Enrollment Closes: Aug 21 
Enroll
Enrollment Closes: Aug 28 
Enroll
Enrollment Closes: Aug 4 
Enroll
Enrollment Closes: Aug 25 
Enroll
Enrollment Closes: Aug 18 
Enroll
Enrollment Closes: Aug 18 
Enroll
Enrollment Closes: Aug 17 
Enroll
Enrollment Closes: Aug 7 
Enroll