Space Vegetables ICON

Hi all !

Hope everyone is safe.

 

This is the first post about the software side of Space Vegetables. Here everything will be explained and how to install and configure.

 

Everything installed and used in this post is installed in the Raspberry PI 4B 2GB and the Enviro HAT .

 

Inside the user pi home directory, at /home/pi, create a new directory called SpaceVegetablesWebServices. All the files will be created there (but not the database).

 

Hardware

  • Raspberry PI 4B 2GB
  • Enviro HAT

 

Software

The Operating System is Raspberry Pi OS.

 

First, because the Raspberry PI will be used solely for the Space Vegetables Project, I'm not going to use Python Virtual Environments. Everything will be installed system wide by using either apt or pip with sudo.

Second, I'm going to assume that Python3 is the default version. You can change it easily by following this instructions.

 

Python libraries

The libraries needed are not many - Flask, os and sqlite3 . sqlite3 and os are part of Python3. The one needed to install is just Flask

 

Web Services

For serving the web services, I'm going to use Flask and Nginx .

 

Flask

Flask is a micro  and lightweight web application framework. It was designed to make getting started really quick and easy, with the ability to scale up to very complex applications.

 

Nginx

Nginx is a free and open-source high performance HTTP server and lightweight compared with others available.

 

Let's start by install them all

 

sudo apt-get install python3-flask nginx uwsgi-plugin-python3

 

Let's start programming our web services.

 

Database

The database is SQLITE3 and is populated by the SpaceVegetables Client. Here's the schema:

CREATE TABLE sqlite_sequence(name,seq);
CREATE TABLE IF NOT EXISTS "Vegetables" (
        "idVegetables"  INTEGER,
        "PH"    REAL DEFAULT 0.0,
        "TDS"   NUMERIC DEFAULT 0,
        "waterPumpActive"       NUMERIC DEFAULT 0,
        "lightsActive"  NUMERIC DEFAULT 0,
        "airPumpActive" NUMERIC DEFAULT 0,
        "temperature"   REAL DEFAULT 0.0,
        "humidity"      REAL DEFAULT 0.0,
        "pressure"      REAL DEFAULT 0.0,
        "lightSensor"   REAL DEFAULT 0.0,
        "temperatureInside"     REAL DEFAULT 0.0,
        "humidityInside"        REAL DEFAULT 0.0,
        "dateTime"      TEXT DEFAULT '0000-00-00 00:00:00',
        PRIMARY KEY("idVegetables" AUTOINCREMENT)
);

 

Code

And here's the code for SpaceVegetablesWebServices.py

 

from flask import Flask, request, jsonify
import os
import sqlite3

app = Flask(__name__)

app.debug = True


# Always get latest entries
# Empty variable is used when
# we want the latest entry that HAS VALUES
# or not - empty = 0 we don't want an empty value
# it's this way because are less calls 
def getDataFromDB(data,empty = 0):
    # connection to DB
    conn = sqlite3.connect ("/home/pi/SpaceVegetables/spaceVegetables.db")
    cur = conn.cursor()
    if not empty:
        # data mas not be 0
        cur.execute("select " + data + " from Vegetables where " + data + " != 0.0 order by idVegetables desc limit 1")
    else:
        cur.execute("select " + data + " from Vegetables order by idVegetables desc limit 1")

    temp = cur.fetchone()
    if temp is None:
        return 0
    return temp[0]

# Router per definition
@app.route("/", methods=['GET'])
def home():
    return '<h1>Web services for Space Vegetables</h1>\n'

# Get temperature
@app.route('/api/v1/resources/temperature', methods=['GET'])
def getTemperature():
    return jsonify({'temperature':getDataFromDB('temperature')})

# Get temperature Inside
@app.route('/api/v1/resources/temperatureInside', methods=['GET'])
def getTemperatureInside():
    return jsonify({'temperature Inside':getDataFromDB('temperatureInside')})

# Get PH
@app.route('/api/v1/resources/ph', methods=['GET'])
def getPH():
    return jsonify({'ph':getDataFromDB('PH')})

# Get TDS
@app.route('/api/v1/resources/tds', methods=['GET'])
def getTDS():
    return jsonify({'tds':getDataFromDB('TDS')})

# Get humidity
@app.route('/api/v1/resources/humidity', methods=['GET'])
def getHumidity():
    return jsonify({'humidity':getDataFromDB('humidity')})

# Get humidity Inside
@app.route('/api/v1/resources/humidityInside', methods=['GET'])
def getHumidityInside():
    return jsonify({'humidity Inside':getDataFromDB('humidityInside')})

# Get pressure
@app.route('/api/v1/resources/pressure', methods=['GET'])
def getPressure():
    return jsonify({'pressure':getDataFromDB('pressure')})

# Get luminosity
@app.route('/api/v1/resources/luminosity', methods=['GET'])
def getLuminosity():
    return jsonify({'luminosity':getDataFromDB('lightSensor',1)})

# Get ligths on or off
@app.route('/api/v1/resources/lights', methods=['GET'])
def getLightStatus():
    return jsonify({'lights':getDataFromDB('lightsActive',1)})

# Get air pump on or off
@app.route('/api/v1/resources/airpump', methods=['GET'])
def getAirPumpStatus():
    return jsonify({'air pump active':getDataFromDB('airPumpActive',1)})

# Get water pump on or off
@app.route('/api/v1/resources/waterpump', methods=['GET'])
def getWaterPumpStatus():
    return jsonify({'water pump active':getDataFromDB('waterPumpActive',1)})

# Get all data
@app.route('/api/v1/resources/all', methods=['GET'])
def getAllData():
    temperature = getDataFromDB('temperature')
    temperatureInside = getDataFromDB('temperatureInside')
    humidity = getDataFromDB('humidity')
    humidityInside = getDataFromDB('humidityInside')
    pressure = getDataFromDB('pressure')
    luminosity = getDataFromDB('lightSensor',1)
    airPumpActive = getDataFromDB('airPumpActive',1)
    waterPumpActive = getDataFromDB('waterPumpActive',1)
    ph = getDataFromDB('PH')
    tds = getDataFromDB('TDS')
    # create response
    environment = [ 
            {
                'temperature': temperature,
                'temperatureInside': temperatureInside,
                'humidity': humidity,
                'humidityInside': humidityInside,
                'pressure': pressure,
                'air Pump Active': airPumpActive,
                'water Pump Active': waterPumpActive,
                'luminosity': luminosity,
                'tds': tds,
                'ph': ph
            }
            ]
    return jsonify(environment[0])

# main
if __name__ == "__main__":
    app.secret_key = os.urandom(24)
    app.run(host='0.0.0.0')




 

Now that we already have the Web Services Python app ready, let's configure uWSGI and Nginx to serve the Python script.

 

NOTE: There are other ways of serving the Flask app - gunicorn is an example of another way. I'm just going to use uWSGI .

 

Now, let's create the uWSGI configuration file. I'm going to create it in the SpaceVegetablesWebServices directory - that's were the Flask Python script is.

 

vi SpaceVegetablesUWSGI.py

 

and enter the following lines:

from SpaceVegetablesWebServices import app

if __name__ == "__main__":
    app.run()

 

NOTE: My file is named SpaceVegetablesWebServices.py, replace with the name (without the extension .py) of yours

 

Now, let's see if uWSGI can serve our application:

 

uwsgi_python3 --socket 0.0.0.0:5000 --protocol=http -w SpaceVegetablesUWSGI:app

 

NOTE: I called the file for UWSGI SpaceVegetablesUWSGI.py - that is what I'm going to put after the -w parameter above. If you call it other name, change it after.

 

and you should see something like:

[uwsgi] implicit plugin requested python3
*** Starting uWSGI 2.0.18-debian (32bit) on [Fri Nov 13 14:38:41 2020] ***
compiled with version: 8.2.0 on 10 February 2019 02:42:46
os: Linux-5.4.72-v7l+ #1356 SMP Thu Oct 22 13:57:51 BST 2020
nodename: enviroPI
machine: armv7l
clock source: unix
pcre jit disabled
detected number of CPU cores: 4
current working directory: /home/pi/SpaceVegetablesWebServices
detected binary path: /usr/bin/uwsgi-core
*** WARNING: you are running uWSGI without its master process manager ***
your processes number limit is 12873
your memory page size is 4096 bytes
detected max file descriptor number: 1024
lock engine: pthread robust mutexes
thunder lock: disabled (you can enable it with --thunder-lock)
uwsgi socket 0 bound to TCP address 0.0.0.0:5000 fd 3
Python version: 3.7.3 (default, Jul 25 2020, 13:03:44)  [GCC 8.3.0]
*** Python threads support is disabled. You can enable it with --enable-threads ***
Python main interpreter initialized at 0x1283590
your server socket listen backlog is limited to 100 connections
your mercy for graceful operations on workers is 60 seconds
mapped 64400 bytes (62 KB) for 1 cores
*** Operational MODE: single process ***
WSGI app 0 (mountpoint='') ready in 0 seconds on interpreter 0x1283590 pid: 3237 (default app)
*** uWSGI is running in multiple interpreter mode ***
spawned uWSGI worker 1 (and the only) (pid: 3237, cores: 1)

 

And, just point your browser to it and you should see the following (according to my SpaceVegetablesWebServices.py ):

In the window were the uwsgi test is being made (the same window with all the text above):

 

[pid: 3237|app: 0|req: 1/1] 127.0.0.1 () {24 vars in 250 bytes} [Fri Nov 13 14:39:45 2020] GET / => generated 42 bytes in 1 msecs (HTTP/1.1 200) 2 headers in 79 bytes (1 switches on core 0)

 

And your browser

 

I get the information above. It's working.

 

Because we want something more long term, let's create a configuration file with the options needed.

 

vi SpaceVegetablesUWSGI.ini

 

[uwsgi]
module = SpaceVegetablesUWSGI:app

master = true
processes = 1

socket = SpaceVegetablesUWSGI.sock
chmod-socket = 660
vaccum = true

die-on-term = true

 

vaccum will clean up the socket when the process stop and the die-on-term will ensure that the init system and uWSGI will have the same assumptions about what each process signal means.

 

Let's now create a systemd unit file to start our project when the system boots. Systemd is a suit of tools that provides a init model for managing system services.

 

Create a file that has the .service extension inside the directory /etc/systemd/system

 

sudo vi /etc/systemd/system/SpaceVegetables.service

 

[Unit]
Description = uWSGI instance for Space Vegetables
After = network-online.target

[Service]
User = pi
Group = www-data
WorkingDirectory = /home/pi/SpaceVegetablesWebServices
ExecStart = /usr/bin/uwsgi_python3 --ini /home/pi/SpaceVegetablesWebServices/SpaceVegetablesUWSGI.ini

[Install]
WantedBy = multi-user.target

 

I'm not going to explain the lines in the service file for systemd. You can see more details in the references.

 

Let's start the service and see if it runs without problems.

 

sudo systemctl start SpaceVegetables

 

and see if it's running:

 

pi@enviroPI:~/SpaceVegetablesWebServices $ sudo systemctl status SpaceVegetables
● SpaceVegetables.service - uWSGI instance for Space Vegetables
   Loaded: loaded (/etc/systemd/system/SpaceVegetables.service; disabled; vendor preset: enabled)
   Active: active (running) since Fri 2020-11-13 15:22:13 GMT; 3s ago
 Main PID: 3535 (uwsgi_python3)
    Tasks: 5 (limit: 3861)
   CGroup: /system.slice/SpaceVegetables.service
           ├─3535 /usr/bin/uwsgi_python3 --ini /home/pi/SpaceVegetablesWebServices/SpaceVegetablesUWSGI.ini
           ├─3536 /usr/bin/uwsgi_python3 --ini /home/pi/SpaceVegetablesWebServices/SpaceVegetablesUWSGI.ini

Nov 13 15:22:13 enviroPI uwsgi_python3[3535]: your mercy for graceful operations on workers is 60 seconds
Nov 13 15:22:13 enviroPI uwsgi_python3[3535]: mapped 322000 bytes (314 KB) for 4 cores
Nov 13 15:22:13 enviroPI uwsgi_python3[3535]: *** Operational MODE: preforking ***
Nov 13 15:22:13 enviroPI uwsgi_python3[3535]: WSGI app 0 (mountpoint='') ready in 0 seconds on interpreter 0x51d1b8 pid:
Nov 13 15:22:13 enviroPI uwsgi_python3[3535]: *** uWSGI is running in multiple interpreter mode ***
Nov 13 15:22:13 enviroPI uwsgi_python3[3535]: spawned uWSGI master process (pid: 3535)
Nov 13 15:22:13 enviroPI uwsgi_python3[3535]: spawned uWSGI worker 1 (pid: 3536, cores: 1)

 

If you have any errors, please correct them now. Let's move on into configuring Nginx to serve requests.

 

Setting up Nginx

 

Let's setup a configuration file for Nginx to our project

 

vi /etc/nginx/sites-available/SpaceVegetables

 

server {
        listen 80;
        server_name localhost;

        location / {
                include uwsgi_params;
                uwsgi_pass unix:/home/pi/SpaceVegetablesWebServices/SpaceVegetablesUWSGI.sock;
        }
}

 

In the server_name I've placed localhost because I don't have DNS configured. Replace by yours .

 

Now, let's tell Nginx to enable this site. Create a link to sites-available

 

sudo ln -s /etc/nginx/sites-available/SpaceVegetables /etc/nginx/sites-enabled/

 

Because this is the only site to serve, we need to remove the default file in sites-enable or Nginx won't serve ours.

 

rm -f /etc/nginx/sites-enabled/default

 

Now, let's start nginx

 

sudo systemctl start nginx

 

And now, try to access your site.

and calling a webservice:

 

Since all is working, let's tell the system to start nginx and our service to start when the Raspberry PI boots

sudo systemctl enable nginx
Synchronizing state of nginx.service with SysV service script with /lib/systemd/systemd-sysv-install.
Executing: /lib/systemd/systemd-sysv-install enable nginx

humidity

 

sudo systemctl enable SpaceVegetables
Created symlink /etc/systemd/system/multi-user.target.wants/SpaceVegetables.service → /etc/systemd/system/SpaceVegetables.service.

 

reboot and after that, try to see if you can access the site.

Explaining of the code

Let's start by explaning what the code does.

 

The first lines import the flask libraries and others required to work well.

 

function getDataFromDB (data, empty=0)

This function will retrieve the data from the database.

The data variable will have the field to be queried in the database, like temperature or humidity...

The empty variable has a 0 value by default. This means that, when calling the function, I may call it with or without the second variable - like you see in the code - because it has a default value assigned .

if I'm calling the function without the variable (meaning 0), the data returned is the latest available - be it 0 or not .

If I'm calling the function with 1 in the empty variable, the value return it will be the latest available with data in it, not 0.

Why ?

Because, like you can see from the database schema, it has many fields. The SpaceVegetables Client will not populate all the values every time.

ie: when activating the water pump, it will insert in the database a value of 1 for the waterPumpActive field. When inserting this value, it will not fill all the others and they will be inserted with their default value, alongside the 1 in the waterPumpActive field. Because of this, if I'm going to query, by example, the temperature, using the latest data available (the idVegetables will be the last value - the bigger one) it may be 0 because the latest operation was when the airPump was activated. That is not what I wanted, so I came up with this solution.

 

So, if empty = 0, the SQL query executed is:

cur.execute("select " + data + " from Vegetables order by idVegetables desc limit 1")

It will query the data variable (temperature, ph, tds, etc..) latest available. It will be the latest because i'm ordering by idVegetables from latest to first and with a limit of 1 - just returning one result.

If empty = 1, the SQL query executed is:

cur.execute("select " + data + " from Vegetables where " + data + " != 0.0 order by idVegetables desc limit 1")

It's almost the same, but the field must not be empty.

 

 

Moving on, we get to line 31

@app.route("/", methods=['GET'])

 

Flask works by defining routes (decorators) for the URLs requested, either by post or get.

A decorator is used to register a view function for a given URL rule.

So, if someone will request http://<IP_for_RPI4_B>, this is where it will land. We now define a function that will be called.

def home():
    return '<h1>Web services for Space Vegetables</h1>\n'

And will return a web page with a title saying this is Web Services for Space Vegetables, like the following image:

Moving to line 36, we get our first real web service definiton - request the temperatue:

@app.route('/api/v1/resources/temperature', methods=['GET'])

This is register a decorator for the URL http://<IP_for_RPI4_B>/api/v1/resources/temperature .

Remember: We define the URL . It could also be called @app.route('/get/me/some/temperature').  You would put that in the browser and it would just work . 

 

If this URL is called, the following function is executed:

def getTemperature():
    return jsonify({'temperature':getDataFromDB('temperature')})

 

It's going to call our first function, that will query the database for the latest temperature. Since I'm calling the function without the empty variable, it will be the latest available that is not 0.

The jsonify function will transform the value returned (30.5) into json format. The result will be:

{
  "temperature": 30.5
}

 

in json format.

 

NOTE: I have two fields for temperature. The temperature is the temperature given by the Enviro HAT sensor, whereas the temperatureInside is given by a sensor inside the greenhouse - a more accurate value. The same applies for humidity.

All the other functions are of the same principle, but requesting other values instead of the temperature.

 

In the line 91, we have the final function:

@app.route('/api/v1/resources/all', methods=['GET'])

 

This function will be used by the Raspberry PI W Zero (insert post URL here) when getting the data to post to twitter.  The main diferences from the others is that it calls all the other functions and creates a list with a dictionary with all the values.

It thens converts it into json and returns it.

 

def getAllData():
    temperature = getDataFromDB('temperature')
    temperatureInside = getDataFromDB('temperatureInside')
    humidity = getDataFromDB('humidity')
    humidityInside = getDataFromDB('humidityInside')
    pressure = getDataFromDB('pressure')
    luminosity = getDataFromDB('lightSensor',1)
    airPumpActive = getDataFromDB('airPumpActive',1)
    waterPumpActive = getDataFromDB('waterPumpActive',1)
    ph = getDataFromDB('PH')
    tds = getDataFromDB('TDS',1)
    # create response
    environment = [ 
            {
                'temperature': temperature,
                'temperatureInside': temperatureInside,
                'humidity': humidity,
                'humidityInside': humidityInside,
                'pressure': pressure,
                'air Pump Active': airPumpActive,
                'water Pump Active': waterPumpActive,
                'luminosity': luminosity,
                'tds': tds,
                'ph': ph
            }
            ]
    return jsonify(environment[0])

 

Here it is. Hope it helps someone.

 

References

 

Nginx and uWSGI with Systemd - excellent tutorial from Digital Ocean

https://github.com/torfsen/python-systemd-tutorial