<<<<Previous Blog     Next Blog>>>>

 

Posture detection using OpenCV and Raspberry Pi HQ Camera Kit

 

Overview of the project

 

This pandemic has made most of us work from home. While we may not have an ergonomic chair to sit for long enough at home, we may need to sit in a straight posture (without like slouching) such that it does not affect our body's posture or our health in the long term.

 

So, in this project, we'll use Raspberry Pi HQ Kit and Computer Vision (CV) to monitor the posture of the person sitting on a chair and alert when the person's posture is bad and not as per standards (refer image below).

 

Sitting Posture

 

Source: Sitting positions: Posture and back health (medicalnewstoday.com)

 

This way we can prevent some of the below detrimental effects on our body.

  • Back, neck and shoulder pain
  • Poor circulation
  • Impaired lung function
  • Poor digestion
  • Constricted Nerves
  • Misaligned Spine
  • A curvature of the spine

 

Source: 7 Detrimental Effects of Bad Posture | BackFit Health + Spine

 

 

Unboxing of the kit

 

Thanks to tariq.ahmad and Element14 community for selecting my proposal and providing me with a Raspberry Pi HQ Camera kit. I was pleased to see my proposal selected for the additional kits than it was originally planned for the 'Photography' Project14 contest with an extended deadline (July 28, 2021).

 

On July 16, 2021, I got a call from UPS delivery person, that the kit parcel has arrived at my office address (which I used before for getting Element14 parcels), unfortunately, I was in a different location (updated in my profile) by then due to the pandemic situation here. So, there was a week's delay in receiving it and I got it on July 24, 2021. Hence couldn't make it to the contest deadline July 28, 2021, as planned. However, as proposed, I would like to complete and submit this project.

 

Let's now see what we got in the Raspberry Pi HQ Camera kit in the video below,

 

 

 

Getting started

 

The contents of the Raspberry Pi HQ Camera kit are,

 

ItemLink
Raspberry Pi 4 Model B (8 GB)

Buy a Raspberry Pi 4 Model B – Raspberry Pi

Element14 Link - https://www.element14.com/community/view-product.jspa?fsku=3051887&nsku=02AH3164&COM=noscript

Raspberry Pi HQ Camera (12.3 MP)

Buy a Raspberry Pi High Quality Camera – Raspberry Pi

Element14 Link - https://www.element14.com/community/view-product.jspa?fsku=3381605&nsku=67AH5587&COM=noscript

Telephoto Lens (10 MP, 16mm C-Mount)Element14 Link - https://www.element14.com/community/view-product.jspa?fsku=3381606&nsku=67AH5588&COM=noscript

 

1. Raspberry Pi 4

 

Let's first get started with Raspberry Pi 4. For this you'll need,

 

ItemUseLink
15.3W USB-C Power Supply (at least 3A recommended)To power up the Pi 4Buy a Raspberry Pi 15.3W USB-C Power Supply – Raspberry Pi
SD card with a recommended size of at least 8 GB spaceTo store files and for the Raspberry Pi OS

Micro HDMI to Standard HDMI Cable *

(x2 if you wish, since RPi 4 has two HDMI ports,

you can now connect to two separate monitors)

To connect Raspberry Pi to a screenBuy a Micro HDMI to Standard HDMI (A/M) Cable – Raspberry Pi
A Keyboard and a mouse *To provide input to Raspberry Pi

Buy a Raspberry Pi Keyboard and Hub – Raspberry Pi

Buy a Raspberry Pi Mouse – Raspberry Pi

A monitor or TV which supports HDMI input *To view Raspberry Pi Desktop Environment

 

* These are optional if you wish to control your Raspberry Pi through your personal computer or laptop via VNC (Virtual Network Computing) - Raspberry Pi Documentation

 

Step 1: Flash the SD card with OS (Refer Setting up your Raspberry Pi - Set up your SD card | Raspberry Pi Projects)

Step 2: Mount the SD card in its slot and connect the power supply.

Step 3: Turn on the power supply and that's it! you have your Pi up and running.

 

People at Raspberry Pi have done a wonderful job with the getting started documentation which you can find here - Setting up your Raspberry Pi - Introduction | Raspberry Pi Projects

 

2. Raspberry Pi HQ Camera

 

Now let's jump onto the Raspberry Pi HQ Camera and Telephoto Lens. Again, people at Raspberry Pi have documented clearly the steps required to get started,

 

Step 1: Remove the dust caps from both HQ Camera and lens.

Step 2: Turn off your Pi and connect the HQ camera to Raspberry Pi using the ribbon connector.

Step 3: Fit the 16mm C-mount lens onto the C-CS adapter that comes with the HQ Camera (Refer HQ Cam Lens Diagrams (raspberrypi.org))

Step 4: Turn on your Pi and enable Camera. For this go to Raspbian Menu -> Preferences -> Interfaces. Select the radio button below enable and next to camera and OK. Reboot your Pi.

Step 5: Enter the below line in the Pi's terminal. A preview will open, and you should be able to view an image of object/ scene the camera and lens is pointed to. After 5 seconds, preview will close, and an image will be saved to your Home folder with name 'test.jpg'.

 

raspistill -o test.jpg

 

 

Voila! you now have setup Raspberry Pi & HQ Camera. Let me know in the comments section if you face any trouble following the steps or while getting started.

 

Source/ Reference:

     1. Official Guide - hq-camera-getting-started.pdf (raspberrypi.org)

     2. Get started with the High Quality Camera — The MagPi magazine (raspberrypi.org)

 

3. Raspberry Pi 4 and accessories

 

Here are some photos of Raspberry Pi 4 and accessories - Heat sink, SD Card, Micro HDMI to standard HDMI cable and Power supply.

 

{gallery} Raspberry Pi 4 and accessories

Raspberry Pi 4

Raspberry Pi 4

Raspberry Pi 4 with Heat Sink

Raspberry Pi 4 with Heat Sink

32GB SD Card

32 GB SD Card

Micro HDMI to HDMI Cable

Micro HDMI to Standard HDMI Cable

15.3W USB-C Power Supply

15.3W USB-C Power Supply

 

 

4. Raspberry Pi 4 Case

 

Here are some photos of the case for Raspberry Pi 4 with fan which I bought and an animation of fitting Pi into the case.

Fitting Raspberry Pi into the case

 

{gallery} Raspberry Pi 4 Case

Bottom of case

Bottom piece of the case

Middle part of the case

Middle piece of the case - Top View

Middle part of the case - Bottom View

Middle piece of the case - Bottom View

Top part of the case - Top View

Top piece of the case - Top View

Top part of the case - Bottom View

Top piece of the case - Bottom View

Fan fitted onto the case's top part

Fan fitted onto the case's top piece

SD Card inserted through the case

SD Card inserted through the case

Cables plugged through the case

Cables plugged through the case

 

 

OpenCV - Posture detection

 

Initially I was waiting for the Raspberry Pi HQ Camera kit to arrive. So, without wasting any time, I started exploring OpenCV in my laptop (with Linux OS) and this is how it went...

 

OpenCV has been in my to-do list for a long time. I am happy that finally I got a nice excuse to start learning and making projects using OpenCV. From OpenCV's Website we can understand that (About - OpenCV),

OpenCV (Open Source Computer Vision Library) is an open source computer vision and machine learning software library.

 

In this project, we are going to use OpenCV as the brains to analyze the photos/ videos that Raspberry Pi HQ Camera kit would capture and estimate the posture of the person who is sitting. Then we'll alert the person if required to correct his posture to prevent any long-term detrimental effects on his/ her body.

 

There is a project called OpenPose which will help to detect key points of body. Then based on the position of key points we can identify whether ideal sitting posture is followed or not. In this blog, we'll cover up to key point detection and the remaining will be covered in upcoming blog posts.

 

1. Installation of OpenCV in Linux - I had followed instructions in the following link to install OpenCV in Linux OpenCV: Installation in Linux

 

I faced an issue while importing OpenCV library (command given below) even after following the required steps in the web page mentioned above,

import cv2 as cv

 

This was resolved by using the below command in terminal. Thanks to the following post - python - Cannot find module cv2 when using OpenCV - Stack Overflow

pip install opencv-python

 

After installing OpenCV in Linux I ran a basic project given in OpenCV website - OpenCV: Getting Started with Images. It ran as expected, so now let's move on to the next step.

 

2. Selecting a Model - After searching for a while, I landed on this beautifully written article on human pose estimation - Deep Learning based Human Pose Estimation using OpenCV (learnopencv.com). I could see two data sets that can be used for pose estimation or key point detection. One is MPII and another is COCO model. But based on the key points generated I can see COCO model covers more facial details like ear and eye whereas MPII detects only as head (and other body key points as well). So, I picked up and proceeded with the COCO model - COCO - Common Objects in Context (cocodataset.org). Since it can help to better estimate the position of head like titled or straight.

 

3. Model Weights - We'll need to now get model weights which will be used in the OpenPose project. To do the same, first you need the project from here - openpose/models at master · CMU-Perceptual-Computing-Lab/openpose · GitHub. Then you can run the below commands in terminal.

 

sudo chmod a+x getModels.sh 
./getModels.sh

 

Or it can be downloaded from here as well - http://posefs1.perception.cs.cmu.edu/OpenPose/models/pose/coco/pose_iter_440000.caffemodel

 

4. OpenPose Project - This is the last step, for this you need project from here - learnopencv/OpenPose at master · spmallick/learnopencv · GitHub. Place the weights file generated or downloaded from previous step inside 'pose/coco' folder. You can then place the image/ video that needs to be processed and key points identified into the project home directory. I used Visual Studio Code to run the 'OpenPoseImage.py' or 'OpenPoseVideo.py' files. A screenshot of the same is given below. On successful run, output skeleton and output keypoints detection files will be placed in the same project home directory.

 

VS Code Open Pose Project

 

First, I was using images and videos available online or captured using my mobile phone to test the OpenPose project. And then the Raspberry Pi HQ Camera kit arrived which was then used to capture images and video to provide input to the OpenPose project.

 

Below are the images and video of the camera kit in action. I made a case out of foam board to temporarily support the camera module and to hold the lens.

 

 

{gallery} Camera in Action

Case for Camera

Case for camera using foam board

case for camera using foam board

Case for camera using foam board

Camera full setup

Camera full setup with support for telephoto lens using foam board

camera full setup

Camera full setup with support for telephoto lens using foam board - top view

camera full setup - side view

Camera full setup with support for telephoto lens using foam board - side view

 

 

Output Skeleton

 

 

{gallery} Output Skeleton

Sitting straight

Sitting straight

Sitting straight

Sitting straight

Bending forward

Slight bending forward

Extreme bending forward

Extreme bending forward

 

Output KeyPoints

 

{gallery} Output KeyPoints

Sitting straight

Sitting straight

Sitting straight

Sitting straight

Slight bending forward

Slight bending forward

Extreme bending forward

Extreme bending forward

 

Video processed using OpenPose project - Output Skeleton

 

 

 

 

Source/ Reference:

     1. OpenCV: OpenCV Tutorials

     2. Deep Learning based Human Pose Estimation using OpenCV (learnopencv.com)

     3. learnopencv/OpenPose at master · spmallick/learnopencv · GitHub

 

Next Steps

 

Next is to use the output from OpenPose project to estimate the posture of the person sitting and alert him/ her to correct his/ her posture to avoid any long-term effects on his/ her body.

 

 

 

 

<<<<Previous Blog     Next Blog>>>>