Skip navigation

Arduino

3 Posts authored by: cloudformdesign

Everything that happens on a micro-controller has to be known. This includes dealing with memory. Many people (myself included) have developed a wonderful set of code that can dynamically use and free memory through the standard malloc and free functions. Only later do we discover that our available memory is getting reduced as time goes by -- followed by undetermined and random failures.

 

This is because no microcontroller (the ArduinoArduino included) has the ability to garbage collect.

 

For instance, let's say you malloc A and then malloc B. You then free A. There is now a memory hole (the size of A) that will never be closed again. Essentially a non-existing A is now taking up memory. If you now malloc A again, it will not be stored in that hole -- new memory will be taken up. You can see how this can become a problem.

 

To solve this problem, I developed the library ReMem (Reusable Memory), located in my usertools library, which among other things contains a full featured User Interface.

 

Documentation of ReMem can be found here. Basically ReMem creates an object (with a specified size) that has two functions: rmalloc and free. These work exactly the same way as the conventional malloc and free except when you free data, it will be used again if the same size data is requested. (The disadvantage is that it can be slow, and takes an extra byte of data per malloc).

 

 

Upcomming

The code is currently stable, but I'm going to be adding the ability to reclaim data that has been freed at the end, as well as cannibalize large allocations. Stay tuned.


Update

I have a new, more tested library for this purpose called tinymem. Check it out here: https://github.com/cloudformdesign/tinymem

User interfaces can take up a lot of memory. That is why, while developing the Arduino User Interface I focused on developing a way to store things in Program Memory instead. It turns out that storing -- and then using -- program memory objects can be slightly confusing. So I thought I would write a quick post about it.

 

Most people are familiar with the simple use of program memory for strings. Instead of writing

 

Serial.println("This is taking up a lot of mem");

 

You can write

Serial.println(F("This is taking up no mem"));

The F() stores the string in Flash (or Program Memory) instead of creating a character array in RAM and then passing it to println. This is very good, considering that the Arduino UnoArduino Uno has only 2k of SRAM and more than 32k of Program Memory!

 

Storing in Program Memory

But what if you want to do something more? The threading module requires an array of pointers to functions. At first glance, this also looks fairly easy.

typedef int (* funptr)(int index)  // declare the funptr type

 

int return_0(){

  return 0;

}

 

int return_1(){

  return 1;

}

 

PROGMEM const funptr fptrs[] = {&return_0, &return_1, NULL} // fun_x are declared functions

The problem comes about when you have to pass the pointers variable to other functions -- there is no way to declare that it is coming from program memory!

 

Before I explain, perhaps it is a good idea if we remember what a pointer is: it is a number which represents a certain location in memory. So:

 

int x = 8;          // x is storing the integer 8

int *p_x = &x;  // p_x is storing the location of x

 

So right now our fptrs are just an array of data that is stored in Program Memory, each of them pointing to functions.

 

Lets say we have a function like the one below:

 

int call_func(int index, const TH_funptr *ptrs){

     return ptrs[index](); // THIS DOESN'T WORK

}

 

The reason the above code doesn't work is because fptrs are in program memory! To access it we have to read out of program memory. Fortunately, there is a function to read 2 bytes of data (a word) from program memory: pgm_read_word

 

To solve this problem, I developed the get_pointer routine.

 

void *get_pointer(PGM_P s, uint8_t index, uint8_t size){

  return (void *)pgm_read_word(s + index * size);

}

 

So now our function would look like

 

// not very useful, but at least it works

int call_func(int index, const TH_funptr *ptrs){

     return get_pointer((PGM_P)fptrs, index, 2)(); // This works. Note that you have to cast fptrs as type PGM_P

}

 

 

More Complex Storage in Program Memory

 

The Arduino page has an excellent tutorial on how to store strings in program memory, and that is what is fundamentally used in the usertools library. Basically, you must construct each pointer, and then store an array of pointers (tons of fun).

 

This also applies to more complex data types. I would have THOUGHT that you could store basic types in an array, but it turns out you can't. For instance, variables are composed of a pointer to the variable and an 8 bit unsigned integer representing it's size. To store a variable, you have to store a pointer to each element in program memory, and then store the array of those pointers into program memory as well (using way more program memory than I would like, but no RAM).

 

That is the reason to expose variables you have to do:

UI_V(v1, myvar1);              // Variables have to be declared specially. Declare variable names first with UI_V

UI_V(v2, myvar2);              // Variables have to be declared specially. Declare variable names first with UI_V

expose_variables(UI_VA(v1), UI_VA(v2));     // Then wrap the variable names in UI_VA. Alot of things have to be done to take up zero RAM!

 

Now you know what goes into taking up zero RAM.

One of my first big micro-controller projects was called the Neonatal Closed Loop Oxygen System. When babies are born prematurely, they have to be fed very specific amounts of oxygen. Too little oxygen and they suffocate, too much oxygen and they can die from oxygen toxicity. Current methods are not advanced, essentially involving a nurse adjusting the oxygen levels manually. The goal of the project was to develop an automatic system.

 

We had to prevent this little guy from getting oxygen poisoning

450px-Premature_infant_with_ventilator.jpg

The project had the following design requirements:

  • Check oxygen levels from a UART input.
  • Process oxygen levels and respond appropriately (long calculation time).
  • Adjust valves electronically to correct oxygen levels.

 

The project also had to do the following:

  • implement a simple user interface with a few buttons and an LCD screen
  • Be able to sound an alarm for a set amount of time if there were issues.
  • Export periodic debugging data to improve the product.


All of these things had to happen simultaneously, and it was critical that nothing prevented the microcontroller from adjusting the oxygen. It would be no good if the Alarm sounding caused all other functions to cease!


I was in charge a few aspects of the project, but one of the main ones was developing a user interface and threading structure. Essentially my problem was: how do you do all of these things at once? I'm not going to go over the solution I used (it was nasty and difficult to understand), but about a year later I discovered the protothreading library written by Adam Dunkels. This made threading significantly easier, but it was still a pain -- you had to declare each thread object individually, it seemed almost impossible for functions to use multiple threads (as variables have to be static), and it still felt way too much like micro-controller programming (and not enough like Arduino programming!)

 

Why do you need a user interface?

Maybe you aren't trying to save a baby's life, maybe you are trying to figure out how to control a robot like the line follower below. You have to take user input, output debug data, take line readings, and control each motor separately. That's a lot of things for a little Arduino to do!


Line follower robot using an the ArduinoArduino and the  Motor Control ShieldMotor Control Shield

800px-Line_follower.jpg

 

So, a few months ago I started writing a threading and user interfacing library called usertools. Usertools contains the following files:

  • threading.h -- basic and lightweight threading. threading_led_example2 is only 6.8 KB of program memory and 240bytes of RAM!
  • ui.h -- lightweight and full featured user interface. ui_led_example3 and ui_led_example4 take up only about 11.5KB of program space and 320 bytes of RAM to implement a full featured user interface. This is the focus of this post.
  • errorhandling.h -- An error handling and debugging library. This will be the subject of a later post.

To demonstrate the usefulness of the library, this post is going to focus on ui_led_example4 -- as this example demonstrates most of the benefits of the user interface library. Before I go into that, lets overview what has to happen to create a low-memory threading and user interface

 

  • You have to store everything you can into Program Memory (Flash). This is done through the setup functions. For more information on the internals, see my blog post.
  • You want to be able to dynamically create and destroy data. This is done through the pthread class. Also, dynamically allocating and freeing memory is something you are not supposed to do on microcontrollers, so I had to create a class to do it. See my blog post about it.
  • You have to expose the threads and then "handle" them. In other words you have to have a way to know if they are active, deactivated, or should be killed and do the appropriate thing to them. This is done by expose_threads(...), schedule_thread(thread), and kill_thread(thread). Threads are handled from ui_loop() or thread_loop()
  • You have to monitor the serial data and behave appropriately, allowing the user to schedule threads, kill threads, call functions, and set/get variables. This is all done in ui_loop()

 

Upload the code to your Arduino

 

Get the latest release of usertools library from github. Move the "usertools" folder to your Arduino libraries ([Arduino Folder]/libraries). Start Arduino and go to:

file->Examples->usertools->ui_led_example4


Then upload the code to your Arduino. Open your Serial Terminal (Cntrl + Shift + M) and make sure to set the drop-down boxes in the bottom left to "Both NL & CR" and "57600 Baud".

 

Once you have done that, type the following (each followed by enter):

  • “?” to get info on commands you can type.
  • "v sub_time 500" -- you will see the led's blink much faster (500ms is being taken off both periods)
  • "v sub_time" -- get the value of sub_time in hex
  • "v sub_time 0" -- set it back to 0 (slow blinking)
  • "k led1" -- Kills thread led1. This will make it so that only 1 thread is blinking (blinks at period of 900ms)
  • “k led2” -- everything will be killed (no blinking)
  • “reinit” -- calls the reinit function, everything will be set to default values!

 

Explanation

 

Note: There is full featured documentation for all modules located in the Documentation folder (just open the .pdf files). If anything is confusing you, please reference these documents as they should explain most questions.

 

Declaring our Thread function

 

In this part of the code, we include our usertools library <ui.h> and write our main thread function "blinky_thread". As you can imagine, this thread will cause the LED to blink.

 

At first blinky_thread might look a little odd -- that is an awful lot of code to just blink an LED! The key is in the PT_WAIT_MS call -- this call is what makes the whole thing tick. If you run this program, you will notice that your LED is blinking... weird. Sometimes it's a long period, sometimes it is really short. This is because we have two threads running that are toggling the same LED. Well get to that soon!

 

Keeping track of Threads

 

in this part, we label the index names of our threads, functions and variables (we will give them string names later). This is just so that it is easy to get them and schedule them later in the code.

 

Creating the reinit function (accessible through the terminal)

 

This function will be used in our setup -- but it is also declared such that it can be called from the command line. Take note of the function type: uint8_t thread_function(pthread * pt). All functions or threads must be of this type.

 

Let's go through this function step by step:

  thread *th;

We will use this pointer to access our thread's data later in the function.

 

  TRY(set_thread_innactive(get_thread(LED1)));

  clrerr();

  TRY(set_thread_innactive(get_thread(LED2)));

  clrerr();

Remember that this function will be called by both the setup AND by the user. If it is called by the setup, then we wouldn't need to set these threads innactive. However, if it is called by the user we will need to set them inactive before we can re initialize them.

 

The TRY macro is part of the errorhandling library I wrote to make debugging and error checking easier. For more information, open Documentation/errorhandling.html in your web browser.

 

th = get_thread(LED1);

th->pt.put_input(1000);

th = get_thread(LED2);

th->pt.put_input(900);

These lines retrieve the thread objects and then put a value into their input buffers. This sets one thread to toggle the LED at a 1000ms period, and the other to toggle at a 900 ms period.

 

Note that every protothread implements a dynamically allocated linked list. This means that you can add input and output data to the pthread object and then delete it (with clear_data() or del_input(index)). In addition, when a thread is killed or set innactive, all of it's data is destroyed and reclaimed for future use.

 

  schedule_thread(LED1);

  schedule_thread(LED2);

The threads are finally scheduled, and will be called by the ui_loop at the bottom of the file.

 

Setting everything up

 

During setup is probably one of the most confusing parts of this module. It is very important that you do things in the right order and in the right way. Let's take a look!

 

expose_threads(TH_T(blinky_thread), TH_T(blinky_thread));  // wrap threads in TH_T

This line declares the threads that you are exposing. Once exposed, you can schedule or kill threads through the user interface or through code (as you saw above).

This is the only place that you can expose threads. Every "thread" is really a function of the type uint8_t (*TH_funptr)(pthread *pt) (for the newbee, this is a function pointer object. Your function must take one input pthread* and return uint8_t)

 

Note that all of this is done for you by wrapping each function name with TH_T.

 

expose_threads simultaneously allows you to schedule the thread, as well as kill (or schedule) the thread from the command line. For example, because of the names we are creating below, LED1 can be killed with "k led1" and then rescheduled with "t led1 PERIOD", where period is the blinking period you want (an integer).

 

expose_functions(UI_F(reinit));    // Wrap functions in UI_F

The same concept as expose_threads (except you use UI_F). This allows you to access the function reinit from the command line. Because of our naming scheme below, this function can be called with simply "reinit"

 

Note: Functions that are called from the command line only run once, whereas threads will be called every loop (until they return PT_EXITED or are killed)

 

UI_V(v1, sub_time);              // Variables have to be declared specially. Declare variable names first with UI_V

//UI_V(v2, othervar) -- if you had more variables, continue in this way

expose_variables(UI_VA(v1));     // Then wrap the variable names in UI_VA. Alot of things have to be done to take up

                                 // zero RAM!

 

Exposing variables is a little more involved, and requires you to pre-set a name to each variable, and then expose the names by wrapping them in UI_VA. If you are wondering why this has to be the process, then please see my blog post about it.

 

Because of our naming scheme, we will be able to access this variable from the terminal with the command "v sub" or set it with "v sub VALUE"

 

// The names have to be done similarily to variables

UI_STR(tn1, "led1");

UI_STR(tn2, "led2");

expose_thread_names(tn1, tn2);  // no wrapping required here.

 

UI_STR(fn1, "reinit");

expose_function_names(fn1);

 

UI_STR(vn1, "sub");

expose_variable_names(vn1);

Exposing names is done similarly to variables.

 

Note: if you are not using functions or variables, you have to explicitly declare this with no_functions() and no_variables(). See this in ui_led_example3

 

void setup() {

  pinMode(LEDPIN, OUTPUT); // LED init

  Serial.begin(57600);

 

  Serial.println("\n\n#########################  Start Setup");

  Serial.println(freeMemory());

  // This makes the names actually accessible to the threading module.

  set_thread_names();

  set_function_names();

  set_variable_names();

 

  // this must be called before any threading (except setting names)

  setup_ui(200);  // 200 == the amount of dynamic memory our threads can use.

                  // I recommend at least 100, more if you use alot of strings

 

  // This has two purposes: it initilizes the values during setup, and it can be called

  // by the user to reinitilize the values.

  reinit(NULL); // Note: NULL is only used because the function doesn't actually use it's pthread input.

 

}

 

In our setup function we call the set_NAME_names() to expose the names to the system, and then call setup_ui(200).

 

setup_ui(mem) must be called before anything is done with threading or the user interface (except setting the names). The "200" is the amount of memory we have set asside for dynamic memory. This memory is used for all of our pthread put_input and put_output data. If you are using strings, I recommend using at least 200 bytes. For more information about dynamic reusable memory see my blog post about my ReMem module.

 

 

Call in the loop

Now the only thing left is to call one function during the loop:

void loop() {

  ui_loop();

}

ui_loop takes care of everything for you -- reading the user input, putting into the threads pthread objects so you can get the inputs, and killing threads.

 

Summary

 

As you can imagine, this library took a significant amount of work to complete. It is currently in Beta because it does not have enough testers and is not currently used in any project. I ask the Element14 community to help me with the following:

  • Are you working on an open source project that could use threading and user interface functionality? If you are, please let me know if you would like help implementing this library in your project. I would love to be part of the team!
  • This library needs additional testers, and developers would be much appreciated as well! There are a few features I am planning to implement in the next couple of months, like Signal blocking and a better thread-spawning system. My hope is that this will eventually become a standard Arduino library, and to make that happen I could use other developers!

If you are interested in either of these, please contact me at Garrett@CloudformDesign.com. Together we can make the usertools library fantastic!

Filter Blog

By date: By tag: