Hi! Welcome to our 11th blog post. In this post, we will talk about the Arduino MKR1000, its primary functions and the GET/POST requesting done by it.

 

First, let's talk about what all the Arduino has to do, It has to:

  1. Capture an image using the Camera
  2. Store the image in the SD Card
  3. Upload the image to the server
  4. Wait for the processing to end
  5. Retrieve the processed .wav file and store it in the SD card
  6. Play the .wav file using the speaker

 

All this logic is handled by 4 primary functions in the Arduino's sketch:

 

  • cam_2_SD(): This function will capture an image using the ArduCAM Mini 2MP camera and store it in the SD Card.
  • img_POST(): This function will send a POST request to the server, containing the image just snapped.
  • wav_GET(): This function will retrieve the processed .wav file from the server, and store it in the SD Card
  • play_WAV(): This function will supply the .wav file to the speaker, which will then play it.

 

We already talked about getting the camera images into the SD card in Blog #7. We have also talked about SD card audio playback in Blog #9 So, we'll talk about the middle two functions.

 

In order to send a image stored in the SD card to the server, we need to send a POST request to the server containing that image file.

 

This is a sample POST request:

POST / HTTP/1.1
HOST: flask_hostname
Content-Type: multipart/form-data; boundary = AaB0x
Content-Length: (length of content)
--AaB03x
Content-Type: image/jpeg
Content-Disposition: form-data; name="file"; filename=image_name
Content-Encoding: binary
Binary data of image here
--AaB03X--

 

In the Arduino sketch, it is written like so:

void img_POST() {
  Serial.begin(115200);
  while (!Serial) {
      ; // wait for serial port to connect. Needed for native USB port only
   }
  //Load Normal Picture
  while (SD.begin(4));
  String normalFilename = String(n)+".JPG";
  File normPicture = SD.open(normalFilename);


  // Get the size of the image (frame) taken
  while (!normPicture.available());
  long jpglen = normPicture.size();
  Serial.println("Sending "+String(n)+" image of");
  Serial.print(jpglen, DEC);
  Serial.println(" bytes");


  // Prepare request
  String start_request = "";
  String end_request = "";
  start_request = start_request + "\n" + "--AaB03x" + "\n" + "Content-Type: image/jpeg" + "\n" + "Content-Disposition: form-data; name=\"file\"; filename=\"" + normalFilename + "\"" + "\n" + "Content-Transfer-Encoding: binary" + "\n" + "\n";
  end_request = end_request + "\n" + "--AaB03x--" + "\n";
  long extra_length;
  extra_length = start_request.length() + end_request.length();
  //Serial.println("Extra length:");
  //Serial.println(extra_length);
  long len = jpglen + extra_length;
  
  //Serial.println(start_request);
  Serial.println("Starting connection to server...");
  char flask_hostname[]="192.168.1.109";
  // Connect to the server, please change your IP address !
  if (client.connect(flask_hostname, 5000)) {
    Serial.println("Connected");
    client.print("POST ");
    client.print("/");
    client.println(" HTTP/1.1");
    client.println("Host: " + String(flask_hostname));
    client.println("Content-Type: multipart/form-data; boundary=AaB03x");
    client.print("Content-Length: ");
    client.println(len);
    client.print(start_request);
    //Serial.println("Connected1");


    if (normPicture) {
      //Serial.println("Connected2");
      int count = 0;
      byte clientBuf[128];
      int clientCount = 0;


      while (normPicture.available()) {
        clientBuf[clientCount] = normPicture.read();
        clientCount++;
        count++;


        if (clientCount > 127) {
          //Serial.println(".");
          client.write(clientBuf, 128);
          clientCount = 0;
        }
      }
      if (clientCount > 0) client.write(clientBuf, clientCount);
      Serial.println(String(count));
      normPicture.close();
    }
    //Serial.println("Connected3");
    client.print(end_request);
    client.println();


    Serial.println("Transmission over");
  }
  else {
    Serial.println("Connection failed");
  }
  char sentence[100];
  Serial.println("HEARING STARTS");
  Serial.println("Sentence "+String(n)+" is ");
  while (client.connected()) {
    int flag=0;
    int i=0;
    while (client.available()) {
      // Read answer
      char c = client.read();
      if(c=='$')
      {
        flag=flag+1;
        continue;
      }
      if(flag==1)
      {
        Serial.print(c);
      }
     }
  }
  Serial.println("HEARING ENDS");
  delay(100);
  client.stop();
 }
      }
      //Serial.print(c);
    }
  }
  //Serial.println("Sentence "+String(n)+" is ");
  //Serial.println(sentence);
  Serial.println("HEARING ENDS");
  delay(100);
  client.stop();
  //while(true);
}
//Insert POST code here

 

This request will be sent to the server over WiFi as the MKR1000 is WiFi-enabled.

 

When the server receives this POST request, it will:

  1. Retrieve the image file and save it on its storage
  2. Generate a caption for the image using the model
  3. Convert the caption (string) to an audio file using TTS
  4. Host the audio file, available for download

 

In the previous blog, we saw how .wav files were generated on the server side. We need those .wav files on our Arduino's SD card so that the speaker could access them. Since the MKR1000 is Wifi-enabled, connecting and retrieving the audio files is basically sending a GET request for the audio file to the server.

 

Sample GET request:

GET /output.wav HTTP/1.1
User-Agent: Mozilla/4.0 (compatible; MSIE5.01; Windows NT)
Host: flask_hostname
Accept-Language: en-us
Accept-Encoding: gzip, deflate
Connection: Keep-Alive

 

In the Arduino sketch, it is written like so:

 

void wav_GET()
{
  Serial.println("Starting connection to server...");
  char flask_hostname[]="192.168.1.109";
  // Connect to the server, please change your IP address !
  if (client.connect(flask_hostname, 5000)) {
    Serial.println("hello");
    client.println("GET /output"+String(n)+".wav HTTP/1.1");  // change resource to get here
    client.println("Host: 192.168.1.109");                 // change resource host here
    client.println("Connection: close");
    client.println();}
  else{
    Serial.println("Connection failed");
  }
  Serial.println("Creating file.");
  theFile = SD.open("output"+String(n)+".wav", FILE_WRITE);  // change file name to write to here
  if (!theFile) {
    Serial.println("Could not create file");
    while (1);
  }
  Serial.println("Created");
  int x=1;
  Serial.println("Loop starts");
  while(x!=0)
  {
    if (client.available()) {
    char c = client.read();
    //Serial.println("Available");
    if (c == '\n' && currentLineIsBlank) {
      // end of HTTP header, now save requested file
      while (client.connected()) {
        // stay in this loop until the file has been received
        if (client.available()) {
          c = client.read();  // get file byte
          theFile.print(c);   // save file byte
          //Serial.println("reading");
        }
      }
    }
    // detect the end of the incoming HTTP header
    if (c == '\n') {
      // starting a new line
      currentLineIsBlank = true;
    }
    else if (c != '\r') {
      // got a character on the current line
      currentLineIsBlank = false;
    }
  }


  // if the server's disconnected, stop the client:
  if (!client.connected()) {
    Serial.println();
    Serial.println("disconnecting.");
    client.stop();
    theFile.close();
    Serial.println("Finished writing to file");
    
    // do nothing forevermore:
    //while (true);
    x=0;
  }
  }
  Serial.println("loop ends");
}

 

That's it for this blog. The next blog would be the final one, which will bring all this together working in synergy.

 

Thanks for having us. The next one will be coming soon!