Skip navigation

Raspberry Pi Projects

6 Posts authored by: screamingtiger Top Member

Previous posts for this project here:

http://www.element14.com/community/community/raspberry-pi/raspberrypi_projects/blog/tags#/?tags=pieintheface

 

Here is a demonstration of my final product.  I will be making another post after Halloween to go over the final code.

 

Its outside ready to annoy people!

IMG_0511.JPG

 

Here is the video demonstration:

Previous posts for this project here:

http://www.element14.com/community/community/raspberry-pi/raspberrypi_projects/blog/tags#/?tags=pieintheface

 

I have the following functionality:

  • Load up images in a specific structure
  • Load up sounds in a specific structure
  • Animate eyes, with a realistic yet random blinking method
  • Animate the mouth to make it appear to be talking
  • The eye can be poked (demonstrated on last post) which results in a specific action.

 

To Do:

Play sounds

Finish Physical Pumpkin.

Make "scare" technique using PIR sensor.

 

 

 

 

 

The eye Poke

 

 

squint.png

 

The poking Sequence for the left eye, the right eye is the same with variables swapped:

def getPoking(self):
if(self.leftPoking):
if(self.pokeStart == 0):
self.leftEyeBlit = display.blit(self.leftEyeBlink[-1],(self.leftEyeX,self.leftEyeY))
self.lastPoke = time.time();
self.pokeStart = time.time();
else:
if(time.time() - self.pokeStart > 10):
self.leftPoking = 0
self.pokeStart = 0
self.leftEyeBlit = display.blit(self.leftEye,(self.leftEyeX,self.leftEyeY))







 

 

In this code, the eye poke is the same image as when the eye is nearly fully closed in a blink.  I will change this to blit the squint image that is given in the file structure.

 

There are a couple variables here that come into play.

pokeStart = the time the poke sequence started, if it has started, else it is 0

lastPoke = the time the last poke occurred

 

You can see on line 8 above:

if(time.time() - self.pokeStart > 10):

 

We can only start the poke sequence every 10 seconds.  I plan to put this into a configurable variable.

 

Detecting the Eye Poke

Whenever an images is blitted to the screen, it returns a rectangle object that gives the coordinates of the rectangle the image is on.

In my main game loop I have this code:

for event in pygame.event.get():

if event.type == QUIT:

pygame.quit()

sys.exit()

elif event.type == KEYDOWN:

if event.key == K_ESCAPE:

pygame.quit()

sys.exit()

elif event.type == pygame.MOUSEBUTTONDOWN:

pos = pygame.mouse.get_pos()

if(faces[0].eyes.leftEyeBlit.collidepoint(pos)):

faces[0].eyes.leftPoking = 1

elif(faces[0].eyes.rightEyeBlit.collidepoint(pos)):

faces[0].eyes.rightPoking = 1;





 

The event MOUTBUTTONDOWN gives us the x,y coordinates of the button press, or in this case a screen touch.  Given a rect object, there is a function called collidepoint, which given a x,y point will return true if that coordinate is within the rectangle.  If it is, we run the poke sequence for the eye that is poked.

 

 

 

Playing Sounds an the Talking Mouth

Playing sounds in pygame is fairly trivial.  I will be using the Music object to play sounds.  This is a multi-threaded approach as the sound played in the background of your code, and releases your code pointer after it is started.

 

The way this will work is I will have several sound bytes, some of which play together in a specific order the same way the images are grouped together.  Once a sound file is loaded,it can be played then continuously polled to see if it is done.

 

The sound will begin playing, and I will tell the mouth to start talking.  In the main game loop, the sound will be polled and if it is completed, the mouth will be told to stop talking.  The next sound in the sequence will then be played with a small pause between.  If the sequence is completed, a new sequence will be picked at random.

 

There are 2 special sequences.  One of which is the eye poke.  The eye can only be poked every 10 seconds, and the sound and animation sequence will play.  This will override the currently playing sequence.

 

The other special sequence is the scare sequence.  This is chosen at random in between randomly played sequences.

 

 

The Talking Mouth

 

def talk(self):
if(self.talking == 0): #determine when to talk again
self.mouthBlit = display.blit(self.mouth,(self.mouthX,self.mouthY))
if (time.time() - self.lastTalk >= self.talkDelay):
print("talk!" + "(" + str(self.talkDelay) + ")")
self.talking = 1
self.talkIndex = 0
self.talkDirection = 1
lastTalkSwap = time.time()
self.lastTalk = time.time()
else: #animate talking
if(time.time() - self.lastTalkSwap >= 0):
self.talkIndex = self.talkIndex + self.talkDirection
if(self.talkIndex >= len(self.mouthTalk)):
self.talkIndex = len(self.mouthTalk)-1
self.talkDirection = -1
if(self.talkIndex == 0 and self.talkDirection == -1):
## print("mouth reset")
self.talking = 0
self.talkDirection = 1
self.lasttalk = time.time()
## print("blink index:" + str(self.blinkIndex))
self.talkBlit = display.blit(self.mouthTalk[self.talkIndex],(self.mouthX,self.mouthY))
self.lastTalkSwap = time.time()





 

 

This is run the same way as the eyes.  An array of images is sequenced in order with a small delay between the blitting.  The array is played forwards, then backwards as one complete animation sequence.  Unlike the eye however, it does not stop there.  It continues to play the array back and forth until told to stop otherwise.

 

The variables in play here are:

talkDelay = The delay between talk blits, this is left small but can be configured.  I Emerpically determined a value that looked good for my images.

talkIndex = The current index of the image being blitted.  This cycles forwards and backwards through the mouth talking image array.

talkDirection = This is direction the talkIndex is moving.  it is 1 when moving forward, and -1 when moving backwards.  You can see on line 16 how this is set.

talkSwap - This last time the images were swapped, the time between swaps must be greater than or equal to talkDelay

 

 

 

The complete code thus far:

 

import os

import pygame,sys, time, random

from pygame.locals import *

 

#a generic function for loading a set of images from a given directory

def LoadImages(imagesPath):

imageList = list()

for dirname, dirnames, filenames in os.walk(imagesPath):

for filename in sorted(filenames):

try:

imageList.append( pygame.image.load(os.path.join(dirname, filename)))

except:

pass

return imageList



#a generic function for loading a set of sounds from a given directory

def LoadSounds(imagesPath):

soundList = list()

for dirname, dirnames, filenames in os.walk(imagesPath):

for filename in sorted(filenames):

try:

soundList.append( pygame.mixer.Sound(os.path.join(dirname, filename)))

except:

pass

return soundList

 

#define the face and sub classes, which is just used to keep all the images together that go together

 

 

 

 

class Mouth:

def __init__(self,path):

self.mouthTalk = LoadImages(os.path.join(path, 'mouth/talk/'))

print("Mouth images:" + str(len(self.mouthTalk)))

self.mouth = pygame.image.load(os.path.join(path, 'mouth/mouth.png'))

self.mouthX = 0

self.marginX = 20

self.marginY = 0;

self.mouthW = 480/2

self.mouthH = 350

self.mouthY = (480-350) / 2

self.lastTalk = time.time()

self.lastTalkSwap = time.time()

self.talkMin = 1

self.talkMax = 6

self.talkDelay =.03

self.talking = 0

self.talkIndex = 0

self.talkDirection = 1

self.leftPoking = 0

self.rightPoking = 0

self.lastPoke = 0

self.pokeStart = 0

self.mouthBlit = pygame.Rect(0,0,1,1)



def talk(self):

if(self.talking == 0): #determine when to talk again

self.mouthBlit = display.blit(self.mouth,(self.mouthX,self.mouthY))

if (time.time() - self.lastTalk >= self.talkDelay):

print("talk!" + "(" + str(self.talkDelay) + ")")

self.talking = 1

self.talkIndex = 0

self.talkDirection = 1

lastTalkSwap = time.time()

self.lastTalk = time.time()

else: #animate talking

if(time.time() - self.lastTalkSwap >= 0):

self.talkIndex = self.talkIndex + self.talkDirection

if(self.talkIndex >= len(self.mouthTalk)):

self.talkIndex = len(self.mouthTalk)-1

self.talkDirection = -1

if(self.talkIndex == 0 and self.talkDirection == -1):

## print("mouth reset")

self.talking = 0

self.talkDirection = 1

self.lasttalk = time.time()

## print("blink index:" + str(self.blinkIndex))

self.talkBlit = display.blit(self.mouthTalk[self.talkIndex],(self.mouthX,self.mouthY))

self.lastTalkSwap = time.time()

 

 

 

 

 

 

class Eyes:

def __init__(self,path):

self.leftEyeSquint = LoadImages(os.path.join(path, 'eye/left/squint/'))

self.leftEye = pygame.image.load(os.path.join(path, 'eye/left/eye.png'))

self.leftEyeBlink = LoadImages(os.path.join(path , 'eye/left/blink/'))

## self.rightEyeSquint = LoadImages(os.path.join(path , 'eye/right/squint/'))

## self.rightEyeBlink = LoadImages(os.path.join(path , 'eye/right/blink/'))

self.rightEyeSquint = list()

self.rightEyeBlink = list()

self.leftEyeX = 800/2-480/4;

self.leftEyeY = 0;

self.marginX = 20

self.marginY = 0;

self.leftEyeW = 480/2

self.leftEyeH = 480 / 4

self.rightEyeX = 800/2-480/4

self.rightEyeY = 480/2

self.rightEyeW = 20

self.rightEyeH = 20

self.lastBlink = time.time()

self.lastBlinkSwap = time.time()

self.blinkMin = 1

self.blinkMax = 6

self.blinkDelay = random.randint(self.blinkMin,self.blinkMax)

self.blinking = 0

self.blinkIndex = 0

self.blinkDirection = 1

self.leftPoking = 0

self.rightPoking = 0

self.lastPoke = time.time()

self.pokeStart = 0

self.leftEyeBlit = pygame.Rect(0,0,1,1)

self.rightEyeBlit = pygame.Rect(0,0,1,1)



def getBlink(self):

if(self.blinking == 0): #determine when to blink again

if(not self.leftPoking):

self.leftEyeBlit = display.blit(self.leftEye,(self.leftEyeX,self.leftEyeY))

if(not self.rightPoking):

self.rightEyeBlit = display.blit(self.rightEye,(self.rightEyeX,self.rightEyeY+self.marginX))

if (time.time() - self.lastBlink >= self.blinkDelay):

print("Blink!" + "(" + str(self.blinkDelay) + ")")

self.blinking = 1

self.blinkIndex = 0

self.blinkDirection = 1

lastBlinkSwap = time.time()

if(self.blinkDelay <= 2):

self.blinkMin = 4

else:

self.blinkMin = 1

if(self.blinkDelay >= 4):

self.blinkMax = 3

else:

self.blinkMax = 6

self.blinkDelay = random.randint(self.blinkMin,self.blinkMax)

self.lastBlink = time.time()

else: #animate blinking

if(time.time() - self.lastBlinkSwap >= .05):

self.blinkIndex = self.blinkIndex + self.blinkDirection

if(self.blinkIndex >= len(self.leftEyeBlink)):

self.blinkIndex = len(self.leftEyeBlink)-1

self.blinkDirection = -1

if(self.blinkIndex == 0 and self.blinkDirection == -1):

## print("reset")

self.blinking = 0

self.blinkDirection = 1

self.lastBlink = time.time()

## print("blink index:" + str(self.blinkIndex))

if(not self.leftPoking):

self.leftEyeBlit = display.blit(self.leftEyeBlink[self.blinkIndex],(self.leftEyeX,self.leftEyeY))

if(not self.rightPoking):

self.rightEyeBlit= display.blit(self.rightEyeBlink[self.blinkIndex],(self.rightEyeX,self.rightEyeY+self.marginX))

self.lastBlinkSwap = time.time()







def getPoking(self):

if(self.leftPoking):

if(self.pokeStart == 0):

self.leftEyeBlit = display.blit(self.leftEyeBlink[-1],(self.leftEyeX,self.leftEyeY))

self.lastPoke = time.time();

self.pokeStart = time.time();

else:

if(time.time() - self.pokeStart > 10):

self.leftPoking = 0

self.pokeStart = 0

self.leftEyeBlit = display.blit(self.leftEye,(self.leftEyeX,self.leftEyeY))



# elif(self.rightPoking):

# if(not pokeStart)

# self.rightEyeBlit = display.blit(self.rightEyeBlink[-1],(self.rightEyeX,self.rightEyeY))

# self.lastPoke = time.time();

# self.pokeStart = time.time();



class Face:

def __init__(self,path):

#load each component of the eyes using the generic image loading function



self.talkSounds = LoadSounds(os.path.join(path , 'sounds/talk'))

self.singSounds = LoadSounds(os.path.join(path , 'sounds/sing'))

self.scareSounds = LoadSounds(os.path.join(path , 'sounds/scare'))



#create the eyes class

self.eyes = Eyes(path)

self.mouth = Mouth(path)

#define vars for face



#emperically determined coordinates



self.mouthX = 20

self.mouthY = 150

self.mouthW = 200

self.mouthH = 200



def PrintInfo(self):

print str(len(self.eyes.leftEyeSquint)) + ' left squint images loaded'

print str(len(self.eyes.leftEyeBlink )) + ' left blink images loaded'

print str(len(self.eyes.rightEyeSquint )) + ' right squint images loaded'

print str(len(self.eyes.rightEyeBlink )) + ' right blink images loaded'

print str(len(self.mouthTalk )) + ' talk images loaded'

print str(len(self.talkSounds )) + ' talk sounds loaded'

print str(len(self.singSounds )) + ' sing sounds loaded'

print str(len(self.scareSounds )) + ' scare sounds loaded'

 

 





#main code here

pygame.init()

display = pygame.display.set_mode((800, 480),pygame.FULLSCREEN | pygame.HWSURFACE | pygame.DOUBLEBUF )

pygame.display.set_caption('Funny Pumpkin')

 

 

#Create a list of faces classes, this example only has 1 face, but multiple faces can be used

faces = list()

#load the default face

faces.append(Face('./faces/default/'))

 

#test the class

##faces[0].PrintInfo()

for i in range(len(faces[0].eyes.leftEyeBlink)):

faces[0].eyes.leftEyeBlink[i] = pygame.transform.scale(faces[0].eyes.leftEyeBlink[i], (faces[0].eyes.leftEyeW-faces[0].eyes.marginX,faces[0].eyes.leftEyeH-faces[0].eyes.marginY))

faces[0].eyes.leftEyeBlink[i] = pygame.transform.rotate(faces[0].eyes.leftEyeBlink[i],270)

faces[0].eyes.rightEyeBlink.append(pygame.transform.flip(faces[0].eyes.leftEyeBlink[i],False,True))

faces[0].eyes.leftEye = pygame.transform.scale(faces[0].eyes.leftEye, (faces[0].eyes.leftEyeW-faces[0].eyes.marginX,faces[0].eyes.leftEyeH-faces[0].eyes.marginY))

faces[0].eyes.leftEye = pygame.transform.rotate(faces[0].eyes.leftEye,270)

faces[0].eyes.rightEye = pygame.transform.flip(faces[0].eyes.leftEye,False,True)

 

for i in range(len(faces[0].mouth.mouthTalk)):

faces[0].mouth.mouthTalk[i] = pygame.transform.rotate(faces[0].mouth.mouthTalk[i],270)

faces[0].mouth.mouthTalk[i] = pygame.transform.scale(faces[0].mouth.mouthTalk[i], (faces[0].mouth.mouthW-faces[0].mouth.marginX,faces[0].mouth.mouthH-faces[0].mouth.marginY))

faces[0].mouth.mouth = pygame.transform.rotate(faces[0].mouth.mouth,270)

faces[0].mouth.mouth = pygame.transform.scale(faces[0].mouth.mouth, (faces[0].mouth.mouthW-faces[0].mouth.marginX,faces[0].mouth.mouthH-faces[0].mouth.marginY))

 

 

 

 

 

 

 

 

#global vars

FPS = 30

 

 

r = list()

 

 

#main game loop

i = 0

faces[0].eyes.getBlink()

while(1):

faces[0].eyes.getBlink()

faces[0].eyes.getPoking()

faces[0].mouth.talk()

for event in pygame.event.get():

if event.type == QUIT:

pygame.quit()

sys.exit()

elif event.type == KEYDOWN:

if event.key == K_ESCAPE:

pygame.quit()

sys.exit()

elif event.type == pygame.MOUSEBUTTONDOWN:

pos = pygame.mouse.get_pos()

if(faces[0].eyes.leftEyeBlit.collidepoint(pos)):

faces[0].eyes.leftPoking = 1

elif(faces[0].eyes.rightEyeBlit.collidepoint(pos)):

faces[0].eyes.rightPoking = 1;







pygame.display.update()







Previous posts for this project here:

http://www.element14.com/community/community/raspberry-pi/raspberrypi_projects/blog/tags#/?tags=pieintheface

 

My current progress:

 

 

 

Eyes are blinking, mouth is talking, and the eyes can be "poked".   This post is more of a construction post, I will show my updated code and comments in my next post tomorrow.

 

One thing I am finding challenging is getting a pumpkin that is the right size for the face.  Because the screen is tall but narrow, the face is out of proportion if I attempt to use the full screen.  So I have opted to use half the screen.  I am creating a pumpkin from scratch for this purpose, and the top portion of the screen will be covered by a hat.  I am considering having a window in the hat to see the top of the screen.  perhaps I can show a video here.

 

 

 

 

 

A few pics.

 

I started with half of a foam pumpkin, but the face is too round for the screen.

IMG_0486.jpg

So I used a heat gun to flatten it out.

IMG_0487.jpg

I traced the face onto a piece of paper.

IMG_0489.jpg

 

IMG_0490.jpg

Then I cutout the face.

IMG_0494.jpg

This gave me a template on where to cut the face out on the pumkin

IMG_0495.jpg

My version of a "hot knife" for cutting foam.

IMG_0497.jpg

Cutting out the face.

IMG_0498.jpg

Trial fitting to the Raspberry Pi Screen.

IMG_0499.jpg

Previous posts for this project here:

http://www.element14.com/community/community/raspberry-pi/raspberrypi_projects/blog/tags#/?tags=pieintheface

 

Got my kit, and assembled the Pi screen.  I did a few things that will help with development and deployment.  Follow the steps on this page:

http://opentechguides.com/how-to/article/raspberry-pi/5/raspberry-pi-auto-start.html

 

This will allow auto login, as well as automatic starting of the GUI on boot.  This is better for a project that will use the GUI and wont have a keyboard and/or mouse attached.

 

Edit your /boot/boot.config file to have these lines.  The lines probably already exist so just edit them.

framebuffer_width=800
framebuffer_height=480

 

I found this to be a perfect resolution for the Raspberry pi Screen as well as the command line interface.

 

I have been working on the code to animate the eyes.  I have one eye working that blinks at random.  I have it enlarged for development and here is a video demonstration:

 

 

 

 

 

 

The blinking is a simple algorithm that ensures it does not blink too fast or blink too slow.  The actual blink animation is a fixed time, but the time between blinks is based on the interval between the last blink.

if(self.blinking == 0):  #determine when to blink again
   display.blit(self.leftEye,(0,0))
   if (time.time() - self.lastBlink >= self.blinkDelay):
    print("Blink!" + "(" + str(self.blinkDelay) + ")")
    self.blinking = 1
    self.blinkIndex = 0
    self.blinkDirection = 1
    lastBlinkSwap = time.time()
    if(self.blinkDelay <= 2):
     self.blinkMin = 4
    else:
     self.blinkMin = 1
    if(self.blinkDelay >= 4):
     self.blinkMax = 3
    else:
     self.blinkMax = 6
    self.blinkDelay = random.randint(self.blinkMin,self.blinkMax)
    self.lastBlink = time.time()



 

 

 

I have added a Eye subclass to the Face class.  Each part of the face will be a class that encapsulates and draws its own images on the screen.  The completed code thus far:

 

import os
import pygame,sys, time, random
from pygame.locals import *

#a generic function for loading a set of images from a given directory
def LoadImages(imagesPath):
 imageList = list()
 for dirname, dirnames, filenames in os.walk(imagesPath):
  for filename in sorted(filenames):
   try:
    imageList.append( pygame.image.load(os.path.join(dirname, filename)))
   except:
    pass
 return imageList
 
#a generic function for loading a set of sounds from a given directory
def LoadSounds(imagesPath):
 soundList = list()
 for dirname, dirnames, filenames in os.walk(imagesPath):
  for filename in sorted(filenames):
   try:
    soundList.append( pygame.mixer.Sound(os.path.join(dirname, filename)))
   except:
    pass
 return soundList

#define the face and sub classes, which is just used to keep all the images together that go together
class Eyes:
 def __init__(self,path):
  self.leftEyeSquint = LoadImages(os.path.join(path, 'eye/left/squint/'))
  self.leftEye = pygame.image.load(os.path.join(path, 'eye/left/eye.png'))
  self.leftEyeBlink  = LoadImages(os.path.join(path , 'eye/left/blink/'))
  self.rightEyeSquint  = LoadImages(os.path.join(path , 'eye/right/squint/'))
  self.rightEyeBlink  = LoadImages(os.path.join(path , 'eye/right/blink/'))
  self.leftEyeX = 20
  self.leftEyeY = 20
  self.leftEyeW = 20
  self.leftEyeH = 20
  self.rightEyeX = 70
  self.rightEyeY = 70
  self.rightEyeW = 20
  self.rightEyeH = 20
  self.lastBlink = time.time()
  self.lastBlinkSwap = time.time()
  self.blinkMin = 1
  self.blinkMax = 6
  self.blinkDelay = random.randint(self.blinkMin,self.blinkMax)
  self.blinking = 0 
  self.blinkIndex = 0
  self.blinkDirection = 1
 
 def getBlink(self):
  if(self.blinking == 0):  #determine when to blink again
   display.blit(self.leftEye,(0,0))
   if (time.time() - self.lastBlink >= self.blinkDelay):
    print("Blink!" + "(" + str(self.blinkDelay) + ")")
    self.blinking = 1
    self.blinkIndex = 0
    self.blinkDirection = 1
    lastBlinkSwap = time.time()
    if(self.blinkDelay <= 2):
     self.blinkMin = 4
    else:
     self.blinkMin = 1
    if(self.blinkDelay >= 4):
     self.blinkMax = 3
    else:
     self.blinkMax = 6
    self.blinkDelay = random.randint(self.blinkMin,self.blinkMax)
    self.lastBlink = time.time()
  else:  #animate blinking
   if(time.time() - self.lastBlinkSwap >= .3):
    self.blinkIndex = self.blinkIndex + self.blinkDirection
    if(self.blinkIndex >= len(self.leftEyeBlink)):
     self.blinkIndex = len(self.leftEyeBlink)-1
     self.blinkDirection = -1
    if(self.blinkIndex == 0 and self.blinkDirection == -1):
##     print("reset")
     self.blinking = 0
     self.blinkDirection = 1
     self.lastBlink = time.time()
##    print("blink index:" + str(self.blinkIndex))
    display.blit(self.leftEyeBlink[self.blinkIndex],(0,0))
     
    
  
 
class Face:
 def __init__(self,path):
  #load each component of the eyes using the generic image loading function
  
  self.mouthTalk = LoadImages(os.path.join(path , 'mouth/talk/'))
  self.talkSounds = LoadSounds(os.path.join(path , 'sounds/talk'))
  self.singSounds = LoadSounds(os.path.join(path , 'sounds/sing'))
  self.scareSounds = LoadSounds(os.path.join(path , 'sounds/scare'))
  
  #create the eyes class
  self.eyes = Eyes(path)
  #define vars for face
  
  #emperically  determined coordinates
  
  self.mouthX = 20
  self.mouthY = 150
  self.mouthW = 200
  self.mouthH = 200
  
 def PrintInfo(self):
  print str(len(self.eyes.leftEyeSquint)) + ' left squint images loaded'
  print str(len(self.eyes.leftEyeBlink )) + ' left blink images loaded'
  print str(len(self.eyes.rightEyeSquint  )) + ' right squint images loaded'
  print str(len(self.eyes.rightEyeBlink  )) + ' right blink images loaded'
  print str(len(self.mouthTalk )) + ' talk images loaded'
  print str(len(self.talkSounds )) + ' talk sounds loaded'
  print str(len(self.singSounds )) + ' sing sounds loaded'
  print str(len(self.scareSounds )) + ' scare sounds loaded'


  
#main code here
pygame.init()
display = pygame.display.set_mode((800, 320))
pygame.display.set_caption('Funny Pumpkin')

#Create a list of faces classes, this example only has 1 face, but multiple faces can be used
faces = list()
#load the default face 
faces.append(Face('./faces/default/'))

#test the class
faces[0].PrintInfo()





#global vars
FPS = 30



#main game loop
i = 0

while(1):
 faces[0].eyes.getBlink()
 for event in pygame.event.get(): 
  if event.type == QUIT:
   pygame.quit()
   sys.exit()

 pygame.display.update()








Previous posts for this project can be found here:

http://www.element14.com/community/community/raspberry-pi/raspberrypi_projects/blog/tags#/?tags=pumpkinpi2015

 

I want to blog a little about the pumpkin mainly becuase I need to get the design nailed down while I wait for my hardware to arrive.

 

The pumpkin will play at random, pre-recorded sounds.  When a sound plays, the eyes and mouth move around.  When a

sound ends, the eyes will still move but the mouth wont.

 

Lets start with the eyes.

 

The eyes will blink randomly and if a user "presses" the eye it will squint.  This means I need a set of images

that can be "flipped" to imitate the motions.

 

As an example, a blink is a series of images that are played in order.  This means I need a method of loadig up images and storing them, as well as keeping them in order.

So what I will do is create a file structure that keeps everything nice and tidy.  For now I will only have two

actions for the eye which will be blink and squint.

 

I propose the following directories:

 

faces/

faces/<name>/eye/
faces/<name>/left/
faces/<name>/right/
faces/<name>/mouth/
faces/<name>/sounds/

The <name> will be the name of the face, as I want to be able to load multiple faces.  The default face must exists

and be called 'default'.

Within each eye folder, a set of files is needed.  I propose the following:

 

faces/<name>/eye/left/eye.jpg
faces/<name>/eye/left/blink1.jpg
faces/<name>/eye/left/blink2.jpg
...etc

faces/<name>/eye/left/squint1.jpg
faces/<name>/eye/left/squint2.jpg
...etc

 

The same structure exists for the right eye and the other structures.  When animating, the images are flipped in

order when the action is needed.

 

Having these common directories allows searching and loading of faces to be easier.  For now I will use only the

default face for my development.

 

With the directory in place, now we need to load all the face components which includes the eyes, mouth and sounds.

 

This will require the use of a class for each face to keep everything together.

 

Lets start with an easy example on how to load up all the squint images for the default face.

 

 

import os
import pygame,sys
leftEyeSquint = list()
for dirname, dirnames, filenames in os.walk('./eye/default/left/squint/'):
     for filename in sorted(filenames):
  try:
   leftEyeSquint.append( pygame.image.load(os.path.join(dirname, filename)))
  except:
   print(filename + ": invalid file")
print str(len(leftEyeSquint)) + " Squint images loaded\n"






This gives us a list with an ordered set of squint images.  It should be known that any fiels in the squint

directory will be attempted to be loaded as images, this is why an exception clause exists.

 

This function is a generic function for loading images for the left eye squint.  It does not need to be hard coded for the left eye as it can be used for all the face images.

So I will make it part of a Face Class.  This class will have this generic method of loading images given a path.

 

Here is the complete code for loading up all the components for the eyes and storing the images in a list that is part of a class.

 

import os
import pygame,sys
#define the face class, which is just used to keep all the images together that go together
class Face:
 def __init__(self,path):
  #load each component of the eyes using the generic image loading function
  self.leftEyeSquint = self.LoadImages(os.path.join(path, 'eye/left/squint/'))
  self.leftEyeBlink  = self.LoadImages(os.path.join(path , 'eye/left/blink/'))
  self.rightEyeSquint  = self.LoadImages(os.path.join(path , 'eye/right/squint/'))
  self.rightEyeBlink  = self.LoadImages(os.path.join(path , 'eye/right/blink/'))
  
 
 #a generic function for loading a set of images from a given directory
 def LoadImages(self,imagesPath):
  imageList = list()
  for dirname, dirnames, filenames in os.walk(imagesPath):
       for filename in sorted(filenames):
    try:
     imageList.append( pygame.image.load(os.path.join(dirname, filename)))
    except:
     pass
  return imageList

#Create a list of faces classes, this example only has 1 face, but multiple faces can be used
faces = list()
#load the default face 
faces.append(Face('./faces/default/'))

#test the class
print str(len(faces[0].leftEyeSquint)) + " Left squint images in class\n"
print str(len(faces[0].rightEyeSquint)) + " Right squint images in class\n"




 

 

The same pattern will be used for the mouth and sounds.  The sound function will call a different method of the pygame but the logic remains the same.  The class will be fully filled out and the a list of classes will be used in the main animation loop.

What I will build if I am selected(the more detail, the better!):

 

Synpopsis
At carnivals they usually have a dunk tank where a person, which I usually see dressed as a clown, heckles the crowd to lure them into paying money and attempt to dunk heckler.  The Heckler makes faces and yells things at people as they walk by.  I want to build a pumpkin that does the same thing but heckles trick or treaters or attempts to startle them by waking up suddenly with a loud "BOO!"

 

 

High level description

A group of trick or treaters walk up to the door and notice a foam pumpkin sitting there, no lights inside.  Suddenly eyes and a mouth appear and it begins speaking.  It yells out BOO! then begins heckling the kids using various pre-recorded skits.

 

The eyes are moving, the mouth is moving, all animated.  One of the kids touches the eye, and the pumpkin reacts with pain and says some choice words.  Another pokes the pumpkin in the mouth and it acts disgusted and pretends to spit out the taste.

 

After the kids leave the pumpkin  begins yelling out random things such as "Hey, You there!".  It sings some Halloween songs and later goes to sleep, waiting for its next victims. It may choose a funny face or an evil face next.

 

The Hardware

Inside the pumpkin will be a Raspberry Pi, a PIR motion sensor, an amplified speaker, and the Raspberry Pi Screen.

A large foam or plastic pumpkin will be used.

 

IMG_20150914_195004305.jpg

 

Only 1 screen is needed so a pumpkin the correct size will need to be found to fit the screen and have the face be the right size.  In the below picture the holes are cut and the screen is seen through the holes.  When the pumpkin turns on, animations will be displayed to have moving eyes and a moving mouth.  They eyes and mouth can be touched and will provoke a reaction.

 

The eyes and mouth will be sprite animations of various forms.  In order to move the project along, I will be using free clip art and sound bytes, though I will have to record some of my own sounds.

 

The PIR sensor will be hidden a disguised as a mole of I choose to get the pumpkin a hat, it can be hidden in there.  When the pumpkin is in sleep mode, it will be awaken by either a timer or an event from the PIR sensor.

 

animated.jpg

 

Here is a concept picture of what one face could look like:

 

animated.jpg

 

 


The Software
As much as I love C and SDL, I am going to use Python and the PyGame library.  If that will not perform well enough, I may switch to Java. 
The PyGame library is easy to use and has little overhead to get up and running.  In only a few lines of code, I can have a sprite moving around the screen.  I found that Python is more popular amongst many Pi users and I want this to be a recreatable project that can be more easily modified by Makers.

 

I plan to use as much pre-written software, clip art, and pre-recorded sounds bytes as I can.  I found trying to recreate everything from scratch causes me to lose focus on getting the project completed.

 

 

There exists animated GIF already that mimic mouth movements and the cells from the GIF can be animated with the Pygame library.

 

The bulk of the work will be to find images I want to use, and record skits I want to play back.

 

Animations will be on one thread, sounds on another.  An additional thread will monitor for "mouse clicks" or screen pokes.  All threads will have a synchronizing method.  The RPI 2 is great for multithreading as it as multiple cores and I feel has enough power to handle this application with a high level scripting language.

 

 

Summary

Plastic or foam pumpkin

An Animiated face

Pre-recorded sounds

A interact able face via the touch screen

PIR sensor for motion detection

Raspberry Pi

Raspberry Pi Screen

Powered speaker

 

 

 

About Me

Many years ago I decided to learn DirectX programming under windows.  While this is a specific framework, the generic principles of game programming apply here.  Such concepts as double buffering, bitmapping, the blitter, and the video memory are similar on any platform for general use.

My kids and I build Halloween decorations from felt like the kits that are sold to build little houses.  This will go inline with that activity and will give us a good prop to add to our current set of props we have built!

 

I recently completed a project in the Sci Fi Your Pi Challenge called the QuadCOP in which I had to get into the intricates of the Pi and its multi-core capabilities.  Threading on the cores will be an important aspect of this project in order to play sounds and animations at the same time, which keeping them in sync.  I did a similar concept with the QuadCOP where the GPS NMEA parsing is done on its own thread and it sets flags in order to let the main process know what is going on.  Python allows threading and I will be taking advantage of this.

 

I have learned from my previous project to not get bogged down in so many technical details but use more high level languages in order to move the macroscopic portion of the project along.

Filter Blog

By date: By tag: