Camera mount

As I already mentioned last week I manufactured a camera mount, where the Raspberry Pi camera and the Panasonic Grid-Eye sensor are mounted close together. Here is again the foto of the installation:

 

camera mount with raspberry pi camera and grid-eye sensor

 

The idea was to overlay the image of the camera and the temperature sensor to 1. get an image with temperature information which is very nice to view and to 2. get information how good the sensors are aligned.

 

Matching the images together

The grid-eye sensor has an viewing angle of 60 degrees in both directions and the Raspberry Pi Camera Module v2 has a field of view of horizontal 62.2 degrees and vertical 48.8 degrees. So horizontal it should match quite well (with little tolerance) but vertical the grid-eye sensor can see more than the camera. The grid-eye sensor has a aspect ration of 1:1 and the camera has one of 4:3. This is no big deal: I will add some black fields at the top and bottom of the camera and the data matches.

Here is an image of the combined output:

 

combined output of camera and grid-eye sensor

Making a video

The next step is to make a video with camera and temperature data to see how the data aligns in the whole sensing area. Making a video with OpenCV is quite simple: you just have to add each image/frame and OpenCV does the rest. First here is my source code (the image above was also made with the same source code):

 

#include <stdio.h>
#include <stdlib.h>
#include <linux/i2c-dev.h>
#include <fcntl.h>
#include <unistd.h>
#include <opencv2/opencv.hpp>
#include <opencv2/imgproc/imgproc.hpp>
#include <opencv2/contrib/contrib.hpp>

int main(void)
{
    int file;
    int addr=0x68;        // adress of AMG88xx
    int x,y;     // variables to got through the array
    char register_address=0;    // register address of each sensor pixel
    signed short int internal_temp;
    signed short int pixel_temp;
    
    int end=1;  // variable to end program
    printf("Pi Chef Stove Assistant Demo with AMG88xx\n");  // print start message
    if((file=open("/dev/i2c-1",O_RDWR))<0)    // open i2c-bus
    {
        perror("cannot open i2c-1");
        exit(1);
    }
    if(ioctl(file, I2C_SLAVE, addr)<0)    // open slave
    {
        perror("cannot open slave address");
        exit(1);
    }

    internal_temp=(signed short)(i2c_smbus_read_word_data(file, 0x0e)<<4);    // read internal temperature from sensor
    internal_temp=internal_temp/16;        // recalculate correct value
    printf("Internal Temp: %f C (0x%04X = %i)\n",(float)internal_temp*0.0625,internal_temp,internal_temp);    // print internal temperature

    cv::Mat cameraImage;  // create opencv mat for camera
    cv::Mat cameraImageGray; // create opencv mat for grayscale camera image
    cv::Mat cameraImageBig(320,320,CV_8UC3);  // create opencv mat for camera with final resolution
    cv::VideoCapture cap;    // create camera input
    cap.open(0);        // open camera
    cap.set(CV_CAP_PROP_FRAME_WIDTH, 320);    // change camera width to 320 - we do not need more
    cap.set(CV_CAP_PROP_FRAME_HEIGHT, 240);    // change camera height to 240
    cv::VideoWriter outputVideo;    // create video output
    outputVideo.open("output.avi", CV_FOURCC('M','J','P','G'), 10, cv::Size(320,320), true);    // set video output to 10 fps and MJPG
    if (!outputVideo.isOpened())    // check if generation of video output was successful
    {
        perror("Could not open the output video for write\n");
        exit(1);
    }

    cv::Mat outSmall(8,8,CV_8UC1);        // create opencv mat for sensor data
    cv::Mat outSmallnorm(8,8,CV_8UC1);    // create opencv mat for normalized data
    cv::Mat outColor;    // create opencv mat for color output
    cv::Mat combined;    // create opencv mat for combined output

    while(end==1)  // check end variable
    {
        for(x=0;x<8;x++)
        {
            for(y=0;y<8;y++)
            {
                register_address=0x80+2*((x*8)+y);
                pixel_temp=(signed short)(i2c_smbus_read_word_data(file, register_address)<<4);    // read pixel_temp and cast to signed short
                pixel_temp=pixel_temp/16;    // set pixel_temp to original value
                outSmall.at(7-x,7-y)=pixel_temp;    // save data to opencv mat and rotate it
            }
        }

        cv::normalize(outSmall,outSmallnorm,255,0,cv::NORM_MINMAX);    // normalize Mat to values between 0 and 255
        cv::resize(outSmallnorm,outSmallnorm,cv::Size(320,320));    // resize Mat to 320 x 320 pixel
        cv::applyColorMap(outSmallnorm,outColor,cv::COLORMAP_JET);  // generate colored output with colormap
        cap >> cameraImage;    // copy camera input to opencv mat
        cv::cvtColor(cameraImage,cameraImageGray,CV_RGB2GRAY);    // convert camera image to grayscale
        cv::cvtColor(cameraImageGray,cameraImageGray,CV_GRAY2RGB);    // make cameraImage 3 channels again
        cameraImageGray.copyTo(cameraImageBig(cv::Rect(0,40,320,240)));// copy camera ingae to mat with same resolution as temperature mat
        cv::addWeighted(cameraImageBig,0.5,outColor,0.5,0.0,combined);    // combine camera mat and temperature mat into one single image
        cv::imshow("combined",combined);  // display mat on screen
         outputVideo << combined;    // add frame to video

        char key = cv::waitKey(1);  // check keys for input
        if(key=='e') end=0;  // end if e was pressed
    }
    printf("ended regularly!\n");  // print end message
    close(file);
    return 0;
}

 

This code led to the following video:

 

Unfortunatelly OpenCV and the video input buffer always add a little delay to the video stream. So there is a little delay between the temperature data and the video. One solution would be to handle the video input in a second thread. But this is more complex so I have to live with this little delay for now.

 

What's next?

In the next days I will mount the sensor combination atop of my cook top and do some test measurements.