Author Archives: TerryJField

Simple Protocol for One-way IR Arduino Communications

In a previous post I had posted code that  transmitted a message through an infrared link between two Arduinos. The main issue with this code is that if the infrared beam is intermittently blocked during the transmission, e.g. by waving your fingers in front of the IR LEDs, bits will be dropped and the message received won’t match the message sent. In this post I describe a simple protocol that provides error detection of the received message. In my weather station application, the communications is only one-way and so there is no mechanism for the receiver to re-request a garbled message; the message is sent periodically with updated temperature and barometric pressure data.

The message is put in a simple “envelope”. The format of the packet sent is as follows:

<STX><message buffer><ETX><chksum>

where:

  • <STX> is the “start of transmission” marker and is the character 0x02.
  • <message buffer> is the text message we want to send.
  • <ETX> is the “end of transmission” marker and is the character 0x03.
  • <chksum> is a single byte check-sum of all of the characters in the <message buffer>. You can see in the code that this check-sum is calculated by looping through the message buffer and adding the values of the individual characters.

A check-sum is not going to completely guarantee error detection but it is simpler to implement than a Cyclic Redundancy Check (CRC) check, and is sufficient for my application.

Below is a sample program that transmits “Hello” over the IR link. The function sendMessagePacket constructs the packet:

/*
  IR_Transmitter
  
  Author: Terry Field
  Date  : 20 June 2014
  
  Board: Arduino Uno
  
  Purpose: This demonstration program uses PWM to 
  output a 38 kHz carrier on pin 11 and serial data on
  pin 2. The 38 kHz output is used to modulate the 
  serial output from pin 2 so that it can be detected
  by an IR receiver. 
*/

#include <SoftwareSerial.h>

const int rxPin = 3;  
const int txPin = 2; 

const int STX = 2;
const int ETX = 3;

const long IR_CLOCK_RATE = 38000L;

#define pwmPin 11   // IR Carrier

SoftwareSerial displayPort(rxPin, txPin); // Rx, Tx

void setup()  {
  // set the data rate for the Serial port
  displayPort.begin(2400);
  
  // toggle on compare, clk/1
  TCCR2A = _BV(WGM21) | _BV(COM2A0);
  TCCR2B = _BV(CS20);
  // 38kHz carrier/timer
  OCR2A = (F_CPU/(IR_CLOCK_RATE*2L)-1);
  pinMode(pwmPin, OUTPUT);
}

/*
  sendMessagePacket: this function sends a text message over a serial link.
  The message is in the format <STX><message><ETX><chksum>
*/
void sendMessagePacket(char message[])
{
  int index = 0;
  byte checksum = 0;
  
  displayPort.print((char)STX);
  while (message[index] != NULL)
  {
    displayPort.print(message[index]);
    checksum = checksum + (byte)message[index];
    index++;
  }
  displayPort.print((char)ETX);
  displayPort.print((char)checksum);
}

void loop ()
{
   char msg[] = "Hello";
   sendMessagePacket(msg); 
   delay(1000);
}

The code to read from the serial port and implement error checking is in the function readMessage in the program SerialReaderProtocol shown below:

/*
  Program: SerialReaderProtocol
 
 Author: Terry Field
 Date  : 20 June 2014
 
 Board: Arduino Uno
 
 Purpose: This program reads from a serial port and 
 outputs it to another. This program is expecting 
 messages to be in the format
 <STX><message><ETX><chksum>
 
*/
 
#include <SoftwareSerial.h>

const int STX = 2;
const int ETX = 3;
const int INTERCHAR_TIMEOUT = 50;

const int rxPin = 2;
const int txPin = 255;

// set up a new serial port
SoftwareSerial displayPort =  SoftwareSerial(rxPin, txPin, true);

/*
  Function name: readChar
  
  Input:
    timeout: time in milliseconds to wait for a character on the
    serial port
  
  Output:
    inChar: character read from the serial port
  
  Returns:
    true if a character was read, otherwise false
*/
boolean readChar(unsigned int timeout, byte *inChar)
{
  unsigned long currentMillis  = millis();
  unsigned long previousMillis = currentMillis;

  while (currentMillis - previousMillis < timeout)
  {    
    if (displayPort.available())
    {
      (*inChar) = (byte)displayPort.read();
      return true;
    }
    currentMillis = millis();
  } // while  
  return false;
}

/*
  Function name: readMessage
  
  Input:
    timeout: time in milliseconds to wait for a message
    message: pointer to message buffer
    maxMessageSize: size of message buffer
  
  Output:
    inChar: character read from the serial port
  
  Returns:
    true if a message was received, otherwise false
*/
boolean readMessage(char *message, int maxMessageSize, unsigned int timeout)
{
  byte inChar;
  int  index = 0;
  boolean completeMsg = false;
  byte checksum = 0;

  unsigned long currentMillis  = millis();
  unsigned long previousMillis = currentMillis;

  while (currentMillis - previousMillis < timeout)
  {
    if (displayPort.available())
    {       
      inChar = (byte)displayPort.read();
  
      if (inChar == STX) // start of message?
      {
        while (readChar(INTERCHAR_TIMEOUT, &inChar))
        {
          if (inChar != ETX)
          {
            message[index] = inChar;
            index++;
            if (index == maxMessageSize) //buffer full
              break;
            checksum = checksum + inChar;
          }
          else // got ETX, next character is the checksum
          {
            if (readChar(INTERCHAR_TIMEOUT, &inChar))
            {
              if (checksum == inChar)
              {
                message[index] = NULL; // good checksum, null terminate message
                completeMsg = true;
                break;      
              }
              else // bad checksum
              {
                completeMsg = false;
                break;
              }
            } // read checksum
          } // next character is the checksum
        } // while read until ETX or timeout
      } // inChar == STX
    } // displayPort.available()
    if (completeMsg)
      break;
    currentMillis = millis();
  } // while
  return completeMsg;
}

void setup()  {
  // set the data rate for the SoftwareSerial port
  displayPort.begin(2400);
  Serial.begin(9600);
  Serial.println("\nStarting to listen...\n");
  delay(200);
}


void loop() 
{
  const int MSG_BUF_SIZE = 255;
  char msg[MSG_BUF_SIZE];
  
  if (readMessage(msg, MSG_BUF_SIZE, 5000))
  {
    Serial.println(msg);
  }
  else
    Serial.println("No message");
}



The function receiveMessage reads from the serial port, discarding all characters until it receives a STX. Once the STX is received it reads characters, adding them to the message buffer and calculating the check-sum. If an ETX is received, it assumes the next character is the check-sum. Once the check-sum is received, it compares it with the calculated check-sum; if the two check-sums do not match, the message is discarded.

I am using these functions to send text data only and so I am not expecting the bytes that represent STX (0x02) and ETX (0x03) to be in the message buffer. Note that once the message is received, it is null-terminated.

Note on compiling the code above: WordPress won’t render the null character in the HTML code listing above and so you need to replace NULL in the code listing with the <single quote><backslash>0<single quote>. I’ll see if I can fix the rendering issue.

Improved Infrared Communications Link using the Vishay TSOP382 (Arduino to Arduino communications)

This is a first in a series of posts where I describe how I built an Arduino-based weather station, positioned outside my house, that transmits temperature and barometric pressure data through a window using an IR link to an Arduino-based receiver…

In a previous posting, I had described my experiments with IR LEDs and an IR photo-transistor.  I finally purchased a miniaturized receiver for infrared remote control systems, the Vishay TSOP38238. This device contains a photo detector and a preamplifier in one miniaturized package, all for less than 2 dollars. It handles a supply voltage of 2.5 V to 5.0 V and therefore can be powered off the Arduino’s 5.0 V power output. The data sheet for this device can be found here: tsop382

I purchased these devices from SparkFun Electronics: https://www.sparkfun.com/products/10266

The main advantage of this IR receiver, besides the preamplifier, is that it ignores IR light unless it is modulated (pulsed) at a specific frequency. The last two digits of the part number indicate this carrier frequency in kilohertz and so the TSOP38238 works with a 38 kHz carrier frequency.

My original circuit used a transistor connected to a data pin of an Arduino, with this data pin configured as the transmit pin of a serial port. As the serial data was transmitted from this pin, the transistor turned on and off an IR LED to match the serial data output. Below is a schematic that shows on the left the IR transmitter circuit and on the right the IR receiver circuit. In Transmitter Circuit – Arduino2, the transistor Q1 is connected to the transmit pin of a serial port and switches the IR LEDs on and off. I have added a second transistor, Q2, and this transistor is switched on and off at a constant frequency of 38 kHz. Because Q2 is in series with Q1, the serial data from Q1 is modulated by Q2 at 38 kHz. Now the serial data is transmitted on a 38 kHz carrier and can be detected by the IR receiver circuit, Receiver Circuit – Arduino 1, on the right-hand side of the schematic below.

IR_CircuitsThe IR receiver circuit is taken from the TSOP382 data sheet. The components R1 and C1 are recommended to prevent electrical overstress.

The Arduino sketch to drive the transmitter circuit:

/*
  IR_Transmitter
  
  Author: Terry Field
  Date  : 20 June 2014
  
  Board: Arduino Uno
  
  Purpose: This demonstration program uses PWM to 
  output a 38 kHz carrier on pin 11 and serial data on
  pin 2. The 38 kHz output is used to modulate the 
  serial output from pin 2 so that it can be detected
  by an IR receiver. 
*/

#include <SoftwareSerial.h>

const int rxPin = 3;  
const int txPin = 2; 

const long IR_CLOCK_RATE = 38000L;

#define pwmPin 11   // IR Carrier

SoftwareSerial displayPort(rxPin, txPin); // Rx, Tx

void setup()  {
  // set the data rate for the Serial port
  displayPort.begin(1200);
  
  // toggle on compare, clk/1
  TCCR2A = _BV(WGM21) | _BV(COM2A0);
  TCCR2B = _BV(CS20);
  // 38kHz carrier/timer
  OCR2A = (F_CPU/(IR_CLOCK_RATE*2L)-1);
  pinMode(pwmPin, OUTPUT);
}

void loop ()
{
    displayPort.print("hello\n");
    delay(1000);
}

The code in the setup() function above is using a Pulse Width Modulation (PWM) technique to output a 38 kHz square wave on pin 11.

The Arduino sketch to read output from the IR receiver and output it to the default serial port for inspection in the Arduino IDE’s Serial Monitor is below:

/*
  Program: SerialReader
  
  Author: Terry Field
  Date  : 20 June 2014
  
  Board: Arduino Uno
  
  Purpose: This program reads from a serial port and 
           outputs it to another.
  
 */

#include <SoftwareSerial.h>

const int rxPin = 2;
const int txPin = 255;

// set up a new serial port
SoftwareSerial inputPort =  SoftwareSerial(rxPin, txPin, true);

void setup()  {
  // set the data rate for the SoftwareSerial port
  inputPort.begin(1200);
  Serial.begin(9600);
  Serial.println("\nStarting to listen...\n");
  delay(200);
}


void loop() 
{
  if (inputPort.available())
  {
    Serial.print((char)inputPort.read());
  }
}

Make sure you set the Serial Monitor’s baud rate to 9600.

In my next post I will describe a simple protocol to increase the robustness of the data transfer from the transmitter to the receiver.

The photo below shows on the left the transmitter circuit and on the right the receiver circuit:

IR_breadboard

X11 Forwarding from a Headless Raspberry Pi, to an Ubuntu 12.04 Desktop

I have several Raspberry Pi’s on my network that run headless, i.e. without monitor or keyboard. When I want to run an XWindows application on the Raspberry Pi and have the window appear on my Ubuntu desktop, I run ssh (secure shell client) with the -X option set, and the application’s connection to the X11 display is automatically forwarded from the Raspberry Pi to my desktop’s XWindows server. The connection is encrypted as all ssh connections are.

The command is:

ssh -X user@hostname

It’s that simple. The -X option takes care of setting the DISPLAY environment variable for you.

Note that man page for ssh mentions that enabling X11 forwarding should be enabled with caution for reasons of security. For this X11 forwarding to work, the remote machine has to make a connection back to the local X11 display, and an attacker could take advantage of this connection to perform activities such as keystroke logging. Make sure you trust the machines you use X11 forwarding with.

Test Drivers

My first job in the late 80’s, early 90’s was working at a small start-up company, as part of a three-person development team. The product we were building was a Point-of-Sale (POS) scanning system, targeted at large, high-volume grocery store chains. I was responsible for the software that ran on the “front-end” register used by the cashier and developed this application in Turbo Pascal. The other two members of the team were responsible for the “back-end” store system, running UNIX and written in C.

At this time our company was breaking new ground; there were practically no PC-based grocery scanning systems in existence, and companies like IBM and National Cash Register (NCR) were just developing POS hardware that could run Microsoft’s DOS and software developed in high-level languages like Pascal and C. We practically had to design and build all of the software components ourselves – the only commercial library we used was for our database. We also had to develop our own polling protocol for the RS-485 network that connected the registers to the UNIX server. RS-485 is a serial hardware implementation similar to RS-232 but it allows “multi-drop”, i.e. multiple devices to be connected to the same wire at the same time. Each register was assigned a unique number and the backend server broadcast the registers’ numbers in turn over the network, and a register would respond with its request when its number was called.

The supermarket chain we were targeting as our first major customer had large stores that had upwards of twenty-five registers. It was critical that our network, back-end server, and front-end registers could cope with the transaction volume responsively and reliably. For a supermarket usually the only interaction a customer has with the staff is at the register, and so the check-out process has to be smooth and painless. We had signed an agreement to pilot our software and IBM’s hardware at a local store, and part of the plan was to send a busload of cashiers and a pallet of food to our office warehouse, to test the store system we had set up on folding tables. The system worked for the most part but we did have issues and a second round of manual testing was required. We did our first store installation and were successful; our small company won a $15M contract to install our POS software and IBM hardware in the rest of the supermarket chain.

Our existing testing process for a new release of software (and we did release often in the early days), was to have our customer support representatives and installers, at most five people, perform “ad-hoc” testing for a couple of hours. Our customer support team leader had worked in the supermarket industry before and would perform more systematic testing, but we were not testing the loading on the server and we had no repeatability in our test environment.

Several months later, after installing a handful of stores, we had reports of performance issues at one of the new sites. Our performance goal was to be able to poll each register, and if the cashier had scanned an item, return the item price and description, all within 0.3 seconds. My boss was considering setting up another store system in our warehouse, which required IBM shipping us loaner equipment, or packing up our development systems so we could camp out at the store and try to measure or recreate the performance issue.

At this time I carried a pager on week nights and weekends, and was getting “feedback” on the system issues our customers were experiencing. I had looked at the code and through inspection was able to find some possible defects, but what concerned me was that these defects were slipping through our testing. We had no way of testing the performance of the system on a large scale and our customer, Save-On-Foods, was also very concerned that our small company would not be able to fix the issues that were affecting its customers.

My solution for our testing gap was to create a “register simulator” that provided automation; I modified the register software to be able to respond to polls on the network for either itself or a group of registers and respond to the polls using input from a text file. I also added a “key-stroke logger” so that our support people could record their testing sessions as they created sales on the register, and the resulting text file could be copied, edited, and expanded. Now one register could impersonate a fleet of registers, and we had the beginnings of a repeatable test process.

With the new register simulator we were able to instrument our back-end server under load and solve the performance issue encountered by the newly-installed store. The register simulator also allowed us to tune our protocol for better responsiveness. We were polling each register in the store in a round-robin fashion, but depending on how busy the store was, many of the registers would be idle for long periods of time. One improvement was to create a “fast poll list” for active registers and a “slow poll list” for registers that were idle, and we would poll the fast poll list more frequently than the slow poll list. We were also able to simulate the effects of new message types on the network. For example, when I interfaced to the debit card networks, I found that to send all the data required by the bank in one packet required too much time and starved the other registers from receiving polls for their requests. I subsequently sent the debit card request in several packets.

While automated testing and simulation are not radical concepts, at the time there was no World Wide Web or Google to search for solutions. If the register was slow, it was my responsibility to lead the investigation. My innovation had a low cost of implementation suitable for a start-up company that needed to be agile, and helped to rapidly mature our ad-hoc testing process. With this innovation we were also able to solve some of the difficult performance issues that negatively affected a customer’s experience in the store, and regain our credibility with our own customer.