×

We are still shipping! When you place an order, we will ship as quickly as possible. We are actively monitoring the COVID-19 situation; please see all updates here. Thank you for your continued support.

Enginursday: A New Sensory Experience with the Cthulhu Shield

Using Sapien LLC's Arduino Shield and Electrode array we can expand our human senses.

Favorited Favorite 6

Many years ago I worked in the medical field helping those with physical and mental disabilities. Assistive/adaptive technology can be both prohibitively expensive and proprietary. This makes it hard to afford, and difficult to adapt to a particular need. When I heard that a local company was creating "the world's first open source sensory substitution/sensory augmentation development platform for Arduino," I grabbed my RedBoard Qwiic and eagerly started planning a project.

Redboard Qwiic

The Cthulhu Shield

For those unfamiliar with the Cthulhu Shield, it is an R3 footprint Arduino shield able to output to and take input from an electrode array that is placed on your tongue. This allows for an individual to control things like a mouse with only their tongue. It also allows for data output to the electrode array, where it produces sensation through biphasic voltage pulses on the electrode. The resulting sensation is akin to Pop Rocks candy. I wanted my project to be portable, so I powered my RedBoard with a simple 9V battery.

Cthulhu Shield

Cthulhu Shield

SEN-15897
$75.00

alt text

If you are interested in the science behind it, or want to learn more about the company I recommend viewing their Kickstarter Page.

Electrode Array
Here we go!

Sensory Substitution with Distance Sensing

I wanted to test out how well I could rely on a sensor in place of my eyes to navigate the building. I used our VL53l1x Distance Sensor and set three different thresholds. If a detected object was under 3 feet away, the electrodes would be set to an intense output; between 3-10 feet, a medium output; and no output if the object is over 10 feet away.

alt text

What is it like to be a bat?

#include <Cthulhu.h>

#include <ComponentObject.h>
#include <RangeSensor.h>
#include <SparkFun_VL53L1X.h>
#include <vl53l1x_class.h>
#include <vl53l1_error_codes.h>

#include <Wire.h>
#include "SparkFun_VL53L1X.h"



int started = 0;



SFEVL53L1X distanceSensor;
//Uncomment the following line to use the optional shutdown and interrupt pins.
//SFEVL53L1X distanceSensor(Wire, SHUTDOWN_PIN, INTERRUPT_PIN);


Cthulhu mycthulhu; //creating and instance of Cthulhu



int array[18];

//array to hold which electrodes should be on or off.
int trodes[] = {1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0};

//pulse period for each electrode, in microseconds. Can be manipulated with Pp and IN to change the intensity of the sensation.
int  PP[] = {10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10};

//length of positive pulse for each electrode, in microseconds. Can be manipulated with PP and IN to change the intensity of the sensation.
int  Pp[] = {9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9};

//inner burst number (how many pulses in each inner burst). Can be manipulated with PP and Pp to change the intensity of the sensation.
int  IN[] = {3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3};

//inner burst period.  In microseconds. Can change quality, or 'feel' of sensation.
int  IP[] = {150, 150, 150, 150, 150, 150, 150, 150, 150, 150, 150, 150, 150, 150, 150, 150, 150, 150};

//Outer burst number. Can change quality, or 'feel' of sensation.
int  ON[] = {5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5};


//Requirements:
//Pp must be less than PP.                    CheckWaveform Error 1.
//(PP*IN) must be less than IP.               CheckWaveform Error 2.
//IP*IN must be less than 2000 microseconds.  CheckWaveform Error 3.


void setup(void)
{
  Serial.begin(9600);

}

void loop(void)
{
  Wire.begin();

  mycthulhu.Begin(); //Initialize Cthulhu library
  oldone();

  distance();
}





void oldone()
{



  //mycthulhu.Begin(); //Initialize Cthulhu library
  //check our waveform parameters and verify that they meet requirements
  int myerror = mycthulhu.CheckWaveform(trodes, PP, Pp, IN, IP, ON);

  if (myerror == 0) //if our waveform parameters are ok, update the stimulation parameters and perform one 36ms stimulation cycle.
  {
    mycthulhu.UpdateStimuli(trodes, PP, Pp, IN, IP, ON); //update waveform parameters
    mycthulhu.Stimulate(); //create the stimulation pulsetrain on the electrodes
  }
  else
  {
    //if we entered an invalid waveform, tell us which rule we violated so we can fix it
    Serial.print("Error in waveform parameters. CheckWaveform Error: "); Serial.println(myerror);
  }
  // Wire.end();
  delay(100); // increased delay to avoid eagleye conflict.
}

void distance ()
{




  distanceSensor.begin();



  distanceSensor.startRanging(); //Write configuration bytes to initiate measurement
  int distance = distanceSensor.getDistance(); //Get the result of the measurement from the sensor
  distanceSensor.stopRanging();

  Serial.print("Distance(mm): ");
  Serial.print(distance);

  float distanceInches = distance * 0.0393701;
  float distanceFeet = distanceInches / 12.0;

  Serial.print("\tDistance(ft): ");
  Serial.print(distanceFeet, 2);

  Serial.println();
  if ( distanceFeet < 3)
  {
    for (int i = 0; i < 18; i++)
    {
      trodes[i] = 1;
      PP[i] = 40;
      Pp[i] = 39;
      ON[i] = 8;

    }
  }

  if (( distanceFeet < 10) && (distanceFeet > 3))
  {
    for (int i = 0; i < 18; i++)
    {

      trodes[i] = 1;
      PP[i] = 10;
      Pp[i] = 9;
      ON[i] = 5;

    }
  }
  if ( distanceFeet > 10)
  {

    for (int i = 0; i < 18; i++)
    {
      trodes[i] = 0;
      PP[i] = 10;
      Pp[i] = 9;
      ON[i] = 5;

    }
  }

}
SparkFun Distance Sensor Breakout - 4 Meter, VL53L1X (Qwiic)

SparkFun Distance Sensor Breakout - 4 Meter, VL53L1X (Qwiic)

SEN-14722
$21.95
8

Sensory Augmentation with Eagle Eye IR sensor

When I was thinking of what new sense I wanted to have, I immediately thought of a certain alien hunter - a predator, if you will - that used infrared vision to track its prey. Snakes also have a pit organ that allows them to use IR.

I used our Eagle Eye IR sensor, which conveniently already uses a grid array, and I reduced the resolution from 64 points to 16 for the Cthulhu Shield. This allowed for some positional sensation. If a "hot" object approached from my left into the field of view I would first feel it on the left side of my tongue. As the object moved through my FOV, or as I moved my sensor toward the object, I could feel when it was centered in front of the sensor.

Naturally, I attached the sensor to a mask, placed the electrodes in my mouth, grabbed a Nerf gun and started the hunt.

alt text
Get to the Chopper!

#include <Cthulhu.h>
#include <Wire.h>
int started = 0;

#include <SparkFun_GridEYE_Arduino_Library.h>


#define HOT 30
#define COLD 20

// This table can be of type int because we map the pixel
// temperature to 0-3. Temperatures are reported by the
// library as floats
int pixelTable[64];
//int transarray[64];
GridEYE grideye;



Cthulhu mycthulhu; //creating and instance of Cthulhu



int array[18];


//two arrays I used to generate geometric patterns

int cElectrodeMap[5][4] =
{
  {0, 0, 1, 0},
  {2, 3, 4, 5},
  {6, 7, 8, 9},
  {10, 11, 12, 13},
  {14, 15, 16, 17}
};

int on[5][4] =
{
  {0, 0, 0, 0},
  {0, 0, 0, 0},
  {0, 0, 0, 0},
  {0, 0, 0, 0},
  {0, 0, 0, 0}
};


//array to hold which electrodes should be on or off.
int trodes[] = {0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0};

//pulse period for each electrode, in microseconds. Can be manipulated with Pp and IN to change the intensity of the sensation.
int  PP[] = {40, 40, 40, 40, 40, 40, 40, 40, 40, 40, 40, 40, 40, 40, 40, 40, 40, 40};

//length of positive pulse for each electrode, in microseconds. Can be manipulated with PP and IN to change the intensity of the sensation.
int  Pp[] = {39, 39, 39, 39, 39, 39, 39, 39, 39, 39, 39, 39, 39, 39, 39, 39, 39, 39};

//inner burst number (how many pulses in each inner burst). Can be manipulated with PP and Pp to change the intensity of the sensation.
int  IN[] = {3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3};

//inner burst period.  In microseconds. Can change quality, or 'feel' of sensation.
int  IP[] = {150, 150, 150, 150, 150, 150, 150, 150, 150, 150, 150, 150, 150, 150, 150, 150, 150, 150};

//Outer burst number. Can change quality, or 'feel' of sensation.
int  ON[] = {8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8};


//Requirements:
//Pp must be less than PP.                    CheckWaveform Error 1.
//(PP*IN) must be less than IP.               CheckWaveform Error 2.
//IP*IN must be less than 2000 microseconds.  CheckWaveform Error 3.

void setup(void)
{

}

void loop(void)
{


  mycthulhu.Begin(); //Initialize Cthulhu library
  oldone();

  Wire.begin();


  Serial.begin(115200);
  eagle();
}

void oldone()
{



  //mycthulhu.Begin(); //Initialize Cthulhu library
  //check our waveform parameters and verify that they meet requirements
  int myerror = mycthulhu.CheckWaveform(trodes, PP, Pp, IN, IP, ON);

  if (myerror == 0) //if our waveform parameters are ok, update the stimulation parameters and perform one 36ms stimulation cycle.
  {
    mycthulhu.UpdateStimuli(trodes, PP, Pp, IN, IP, ON); //update waveform parameters
    mycthulhu.Stimulate(); //create the stimulation pulsetrain on the electrodes
  }
  else
  {
    //if we entered an invalid waveform, tell us which rule we violated so we can fix it
    Serial.print("Error in waveform parameters. CheckWaveform Error: "); Serial.println(myerror);
  }
  //Wire.end();
  delay(100); // increased delay to avoid eagleye conflict.
}

void eagle ()
{
  int x = 0;
  //Wire.begin();
  grideye.begin();

  // loop through all 64 pixels on the device and map each float value to a number
  // between 0 and 3 using the HOT and COLD values we set at the top of the sketch
  for (unsigned char i = 0; i < 64; i++) {
    pixelTable[i] = map(grideye.getPixelTemperature(i), COLD, HOT, 0, 3);
  }


  // loop through the table of mapped values and print a character corresponding to each
  // pixel's temperature. Add a space between each. Start a new line every 8 in order to
  // create an 8x8 grid
  for (unsigned char i = 1; i < 64; i++) {
    if (pixelTable[i] == 0) {
      Serial.print(".");
      if ( (i % 4 ) == 0)
      {
        trodes[i / 4] = 0;
      }
    }

    else if (pixelTable[i] == 1) {
      Serial.print("o");
      if ( (i % 4 ) == 0)
      {
        trodes[i / 4] = 1;
      }
    }
    else if (pixelTable[i] == 2) {
      Serial.print("0");
      if ( (i % 4 ) == 0)
      {
        trodes[i / 4] = 1;
      }
    }
    else if (pixelTable[i] == 3) {
      Serial.print("O");
      if ( (i % 4 ) == 0)
      {
        trodes[i / 4] = 1;
      }
    }
    Serial.print(" ");
    if ((i + 1) % 8 == 0) {
      Serial.println();
    }
  }

  // in between updates, throw a few linefeeds to visually separate the grids. If you're using
  // a serial terminal outside the Arduino IDE, you can replace these linefeeds with a clearscreen
  // command
  Serial.println();
  Serial.println();


  // toss in a delay because we don't need to run all out
  delay(100);


}
SparkFun Grid-EYE Infrared Array Breakout - AMG8833 (Qwiic)

SparkFun Grid-EYE Infrared Array Breakout - AMG8833 (Qwiic)

SEN-14607
$40.95

Moving Forward with the Cthulhu Shield

It takes time and practice to start building sensitivity to the position and strength of the electrodes on the array. I changed the output settings on the electrodes to produce as much sensation as I could to help with my uninitiated tongue. I plan on reducing this as I get better at parsing out strength and position of the electrodes. I also experienced an issue when using I2C devices where if I completely bit down on the electrode it would cause the board to freeze where it was in the loop. As a result I needed to gingerly place it on my tongue or block one of the electrodes with some plastic to prevent the freeze up. I am not sure why this occurred, but that is what oscilloscopes and GitHub issue requests are for!

Cthulhu Shield attached to Nerd

Ready to sense a whole new world!

With a whole world of sensors and data out there, the limit to the Cthulhu Shield is our imagination. If you could give yourself a new sense what would it be? The ability to sense total Volatile Organic Compounds? Use GPS to help guide yourself to your destination?


Comments 3 comments

  • Do you provide any guidance or information on galvanic isolation when connecting humans to electronics? While these projects are very interesting, a well meaning project could get someone fried.

    http://www.idc-online.com/technical_references/pdfs/electronic_engineering/safetyisoloationrules.pdf

    • Yes and thank you for the informative Link! Sapien LLC has a Safety Information Page and the Cthulhu Shield includes a grounding cable. I agree though, when connecting a project to one's body there should be proper considerations surrounding safety.

  • I'm pretty sure that shocking my tongue to know how far away something is, is not something that I would be interested in, however when you get it to the point that I can taste a nice juicy steak broiled to perfection or a nice creamy vanilla ice-cream or even a Reeces peanut butter cup...let me know.

Related Posts

Recent Posts

Tags


All Tags