Retrofitting InMoov with Interactive Eyes..... aka >reducing the servo count.

I Picked up a couple of round 240*240 pixel OLED display modules, as luck would have it they scale easily into the InMoov face frame-work.

Driving the display is an ESP32 using SPI bus for control... meaning that two displays can be controlled with it.

It uses the #include <Arduino_GFX_Library.h>

Here is the stripped down code for a simple generic ESP32 Eye

#define JPEG_FILENAME "/240test.jpg"
#include <Arduino_GFX_Library.h>
#define TFT_CS 17
#define TFT_DC 16
#define TFT_RST 4
Arduino_DataBus *bus = new Arduino_ESP32SPI(TFT_DC, TFT_CS, 18 /* SCK */, 23 /* MOSI */, -1 /* MISO */, VSPI /* spi_num */);
Arduino_GC9A01 *gfx = new Arduino_GC9A01(bus, TFT_RST, 0 /* rotation */, true /* IPS */);
#include <SPIFFS.h>
#include "JpegClass.h"
static JpegClass jpegClass;
// pixel drawing callback
static int jpegDrawCallback(JPEGDRAW *pDraw)
{
gfx->draw16bitBeRGBBitmap(pDraw->x, pDraw->y, pDraw->pPixels, pDraw->iWidth, pDraw->iHeight);
return 1;
}

void setup()
{
Serial.begin(115200);
// Init Display
gfx->begin();
gfx->fillScreen(BLACK);

#ifdef TFT_BL
pinMode(TFT_BL, OUTPUT);
digitalWrite(TFT_BL, HIGH);
#endif

if (!SPIFFS.begin())
{
    Serial.println(F("ERROR: File System Mount Failed!"));
    gfx->println(F("ERROR: File System Mount Failed!"));
}
else
{
    unsigned long start = millis();

    // read JPEG file header
    jpegClass.draw( &SPIFFS, (char *)JPEG_FILENAME, jpegDrawCallback, true /* useBigEndian */,
        0 /* x */, 0 /* y */, gfx->width() /* widthLimit */, gfx->height() /* heightLimit */);
    Serial.printf("Time used: %lu\n", millis() - start);
}
}

void loop()
{
}

The library supports BMP,Jpeg.....

;Png,Mjpeg and even .....

gifs.

The examples above uses the "SPIFFS" file system.

It is a simple case of uploading your program, then uploading the graphics into the ESP's SPIFF file system.

If you have not used the SPIFFS system on your ESP32 before then it is essential to install the driver for it into your Arduino IDE, you only need to do this once. Tutorial Here :- Randomnerds

Alternate HTTP Server (aka local access)

#define JPEG_FILENAME1 "/blueeye.jpg"
#define JPEG_FILENAME2 "/browneye.jpg"
#define JPEG_FILENAME3 "/darkbrowneye.jpg"
#include <Arduino_GFX_Library.h>
#define TFT_CS 17
#define TFT_DC 16
#define TFT_RST 4
Arduino_DataBus *bus = new Arduino_ESP32SPI(TFT_DC, TFT_CS, 18 /* SCK */, 23 /* MOSI */, -1 /* MISO */, VSPI /* spi_num */);
Arduino_GC9A01 *gfx = new Arduino_GC9A01(bus, TFT_RST, 0 /* rotation */, true /* IPS */);
#include <SPIFFS.h>
#include "JpegClass.h"
static JpegClass jpegClass;
// pixel drawing callback
static int jpegDrawCallback(JPEGDRAW *pDraw)
{
gfx->draw16bitBeRGBBitmap(pDraw->x, pDraw->y, pDraw->pPixels, pDraw->iWidth, pDraw->iHeight);
return 1;
}

#include <WiFi.h>

const char* ssid = "Wifiname";
const char* password = "password";

WiFiServer server(80);

void setup()
{
Serial.begin(115200);
WiFi.begin(ssid, password);

    while (WiFi.status() != WL_CONNECTED) {
        delay(500);
        Serial.print(".");
    }
      Serial.println("");
    Serial.println("WiFi connected.");
    Serial.println("IP address: ");
    Serial.println(WiFi.localIP());

    server.begin();

// Init Display
gfx->begin();
gfx->fillScreen(BLACK);

#ifdef TFT_BL
pinMode(TFT_BL, OUTPUT);
digitalWrite(TFT_BL, HIGH);
#endif

if (!SPIFFS.begin())

{
Serial.println(F("ERROR: File System Mount Failed!"));
gfx->println(F("ERROR: File System Mount Failed!"));
}

}

void loop()
{
WiFiClient client = server.available(); // listen for incoming clients

if (client) {                             // if you get a client,
    Serial.println("New Client.");           // print a message out the serial port
    String currentLine = "";                // make a String to hold incoming data from the client
    while (client.connected()) {            // loop while the client's connected
      if (client.available()) {             // if there's bytes to read from the client,
        char c = client.read();             // read a byte, then
        Serial.write(c);                    // print it out the serial monitor
        if (c == '\n') {                    // if the byte is a newline character

          // if the current line is blank, you got two newline characters in a row.
          // that's the end of the client HTTP request, so send a response:
          if (currentLine.length() == 0) {
            // HTTP headers always start with a response code (e.g. HTTP/1.1 200 OK)
            // and a content-type so the client knows what's coming, then a blank line:
            client.println("HTTP/1.1 200 OK");
            client.println("Content-type:text/html");
            client.println();

            // the content of the HTTP response follows the header:
            client.print("Click <a href=\"/blueeye\">here</a> for Blue Eye<br>");
            client.print("Click <a href=\"/browneye\">here</a> for Brown Eye<br>");
            client.print("Click <a href=\"/darkbrowneye\">here</a> for Dark Brown Eye<br>");
            // The HTTP response ends with another blank line:
            client.println();
            // break out of the while loop:
            break;
          } else {    // if you got a newline, then clear currentLine:
            currentLine = "";
          }
        } else if (c != '\r') { // if you got anything else but a carriage return character,
          currentLine += c;      // add it to the end of the currentLine
        }

        // Check to see if the client request was "GET /H" or "GET /L":
        if (currentLine.endsWith("GET /blueeye")) {
           jpegClass.draw( &SPIFFS, (char *)JPEG_FILENAME1, jpegDrawCallback, true /* useBigEndian */, 0 /* x */, 0 /* y */, gfx->width() /* widthLimit */, gfx->height() /* heightLimit */);delay(1000);
        }
        if (currentLine.endsWith("GET /browneye")) {
           jpegClass.draw( &SPIFFS, (char *)JPEG_FILENAME2, jpegDrawCallback, true /* useBigEndian */, 0 /* x */, 0 /* y */, gfx->width() /* widthLimit */, gfx->height() /* heightLimit */);delay(1000);
        }
        if (currentLine.endsWith("GET /darkbrowneye")) {
           jpegClass.draw( &SPIFFS, (char *)JPEG_FILENAME3, jpegDrawCallback, true /* useBigEndian */, 0 /* x */, 0 /* y */, gfx->width() /* widthLimit */, gfx->height() /* heightLimit */);delay(1000);
        }
      }
    }
    // close the connection:
    client.stop();
    Serial.println("Client Disconnected.");
}
}

Simple Two eye display driver code ( needing only one ESP32)

#define JPEG_FILENAME1 "/blueeye.jpg"
#define JPEG_FILENAME2 "/browneye.jpg"
#include <Arduino_GFX_Library.h>
#define TFT_DC 16
#define TFT_SCL 18
#define TFT_MOSI 23
#define TFT_MISO -1

#define TFT_CS1 17    // Display 1
#define TFT_RST1 4    // Display 1
#define TFT_CS2 26    // Display 2
#define TFT_RST2 27   // Display 2

Arduino_DataBus *bus1 = new Arduino_ESP32SPI(TFT_DC, TFT_CS1, TFT_SCL, TFT_MOSI, TFT_MISO , VSPI /* spi_num */);
Arduino_GC9A01 *gfx1 = new Arduino_GC9A01(bus1, TFT_RST1, 0 /* rotation */, true /* IPS */);
Arduino_DataBus *bus2 = new Arduino_ESP32SPI(TFT_DC, TFT_CS2, TFT_SCL, TFT_MOSI, TFT_MISO , VSPI /* spi_num */);
Arduino_GC9A01 *gfx2 = new Arduino_GC9A01(bus2, TFT_RST2, 0 /* rotation */, true /* IPS */);

#include <SPIFFS.h>
#include "JpegClass.h"
static JpegClass jpegClass;
// pixel drawing callback
static int jpegDrawCallback1(JPEGDRAW *pDraw)
{
gfx1->draw16bitBeRGBBitmap(pDraw->x, pDraw->y, pDraw->pPixels, pDraw->iWidth, pDraw->iHeight);
return 1;

}
static int jpegDrawCallback2(JPEGDRAW *pDraw)
{
gfx2->draw16bitBeRGBBitmap(pDraw->x, pDraw->y, pDraw->pPixels, pDraw->iWidth, pDraw->iHeight);
return 1;
}
void setup()
{
Serial.begin(115200);
// Init Displays
gfx1->begin();
gfx1->fillScreen(YELLOW);
gfx2->begin();
gfx2->fillScreen(RED);

if (!SPIFFS.begin())

{
    Serial.println(F("ERROR: File System Mount Failed!"));
    gfx1->println(F("ERROR: File System Mount Failed!"));
}
else
{
    // read JPEG file header
    jpegClass.draw( &SPIFFS, (char *)JPEG_FILENAME1, jpegDrawCallback2, true /* useBigEndian */,
        0 /* x */, 0 /* y */, gfx2->width() /* widthLimit */, gfx2->height() /* heightLimit */);
    // read JPEG file header
    jpegClass.draw( &SPIFFS, (char *)JPEG_FILENAME2, jpegDrawCallback1, true /* useBigEndian */,
        0 /* x */, 0 /* y */, gfx1->width() /* widthLimit */, gfx1->height() /* heightLimit */);
      }
}

void loop()
{
}

Now its time to refine the graphics , make a dual OLED holder and re-test.

The advantage of an Interactive system is that the display could also be used for Debug (during start up) or also feedback for control actions.....

Any other ideas out there ?.... note below and we see what can be done....

Oh my gosh ! ... So cool, and

Oh my gosh ! ... So cool, and yes - so much potential for displaying all sorts of info...
Heh I can imagine the veins increasing with red ...
then a red text dump of errors :D

I'm curious if you've thought of the communication to/from the esp32 ?
will you be doing websockets ?
I've been working quite a lot on Mqtt and MqttBroker service in MRL lately...

My house sensors are beginning to utilize mrl for communication configuration and coordiantion.

I have sensors which currently publish to Mrl's MqttBroker,
and can easilly imagine 2 way communication with your esp32 eyes.

publish blink message to topic /eyes/blink and blink go the eyes
publish text message to /eyes/text .. and it displays text
many possiblities ! (I'm really starting to like mqtt)

Update HTTP Jpeg & gifs

Framework works.... now finicky work on the graphics....

Intergrated the Gif and JpegClass's (shoehorn time).

Here's a sneak peak of the

Here's a sneak peak of the new MqttBroker service in MRL
Your esp32 would connect to mrl, and subscribe to /rightEye/blink topic perhaps then from MRL or potentially any device that can send mqtt messages - you'd be winking ;)

HTTP Server ala "local"

I tend to keep things local... so current setup is an ESP32 HTTP server, however in this format anything is possible I guess.

Broker !!

I looked into MQTT .... however could only find cloud service (off site) brokers....

Are Cload brokers the only way to go or are there local brokers that can be instanced?

edit :- did I miss something.... does MRL act as a local broker...?

.... G digs deeper......

You got it .. Yeah, I've

You got it ..
Yeah, I've been working on it for a while ...
Ya Mrl can now "be a" broker - implementation is done, and I have a worky UI(s) too
Need to do some documentation now.

Mqtt Brokers I think are really quite good for orchestrating lots and lots of little IoT devices...
HTTP is nice from a browser - but its very synchronous ..
e.g. send a request ----> get a response back

With Mqtt Brokers anything can talk to anything potentially at any time...
You use "topics" which are names - and you send messages to them ..
Anything can subscribe to the topic and anything can send a message to that topic which will be broadcasted to all the subscribers ...

Sweet

Nice job, looks awesome!

X Y movement

Hello

This looks great.

I am working on the same idea using the animated_eyes_2 code for the ESP32 TFT_eSPI library.

How can I change the x y for the eyes on the arduino from servo's to analog voltage?

Could MTTQ be used to send the x y coordinates?

Cheers Richard

Inmoov Ani_Eyes