Intro to working with with ESP8266

originally posted here

tl;dr

I'm building environmental sensors using the ESP8266, a super cheap wifi-capable microcontroller that can run Micropython. This guide walks through writing driver code for the 8266 to collect data from a DHT and soil moisture sensor and beam it back over the network to a server applicaiton that collects the data for use later. The project code is available here.

So, I live in the Midwest, where things get exceptionally cold during the winter. I also enjoy gardening and fresh vegetables. These facts don't mix too well. Thankfully, places like the University of Minnesota have extension programs working with deep winter greenhouses, designed to grow plants in the dead of winter.

My (long term) goal is to build and automate one of these with microcontrollers to monitor and control the environment within the greenhouse. Since I didn't want to be learning how to program microcontrollers alongside building a greenhouse I decided to get a jump start on it.

This post will detail my journey of making an environmental (temperature, humidity, soil moisture) sensor that reports back to a central server over the network.

This guide/devblog/tutorial will condense the information available from the Micropython ESP8266 tutorial, which is a phenomenally good reference, along with some documentation about the sensors I'm using.

Parts

• 1 ESP8266 ESP12
• 1 DHT11 sensor / 1 DHT 22 Sensor
• 1 soil moisture sensor
• 1 USB wall wart + micro cable

I'll assume you have various kinds of jumper wires lying around, along with a laptop.

Software

All the utilities I used were Python based; so I'm assuming you have a working Python 3 install alongside pip. You'll need some pip packages:

pip install adafruit-ampy esptool

You'll also need a copy of the most recent stable micropython firmware for the ESP8266 from the micropython page.

I use the serial tool picocom to talk to the board over USB; you're welcome to use whatever your favorite serial connection tool is, so long as it, y'know, works.

Why the ESP8266?

So, having just started working with microcontrollers I don't know crazy much about them but I'll tell you what I do know about the 8266.

Originally for this project I was going to use an arduino connected via USB to a raspi. The arduino would collect the sensor readings via it's GPIO port and report them back to the raspi via serial on USB. The pi would parse the incoming data, coallate it and send it to a controlling server. There were a couple problems with this deisgn:

1. Required a raspi to power and communicate with the arduino, with a physical connection. This would make it hard to place many monitoring sensors around the greenhouse, since now I need to run cable. Also increases cost, since for every 4 sensor arduinos I'd need one raspi to power and control them.
2. I had to write the program for the arduino in C. This I kind of expected, since C is definetely the most well-suited language for microcontrollers, but also sucks, since C is a pain in the ass to use, especially for a beginner.

I could mitigate Problem 1 by connecting a wifi chip to the arduino, and beaming the data across the network. I didn't relish having to do that with C, but it was technically possible; quick research pointed me in the direction of a chip called the ESP8266, which was apparently really popular with people wanting to use microcontrollers with wifi. What I missed during my first pass was that the ESP8266 is a microcontroller in it's own right. It doesn't need the arduino, and has GPIO capabilities of its own. In fact, the 8266 outclasses the arduino in available memory.

So, I switched tacks from building an arduino connected to a raspi over USB and decided to use a fleet of 8266s beaming data back to a single, central raspi via wifi.

Getting Started

The ESP8266s I ordered off of Ebay claimed to ship with the AT firmwhere, where you can issue commands over serial to the ESP8266 for things like connecting to a network, opening a TCP/IP socket, beaming data, accepting incoming data, etc. Unfortunately for me, either they didn't have included firmware, the firmware wasn't the AT I was expecting I or I was simply too incompetent to correctly use it, so I spent about three hours trying to convince the damn things to do what I wanted and then I ran into the Micropython project.

Following the instructions from their super helpful ESP8266 tutorial, I went hunting for the device in my /dev; running ls /dev/tty* and looking for things that end in USB0 or ACM0 will point you in the right direction.

Once I figured out what port it was on, I copied the Micropython firmware onto the chip:

$ esptool.py --port /dev/ttyUSB0 --baud 460800 write\_flash --flash\_size=detect -fm dio 0 esp8266-20171101-v1.9.3.bin

Then I connected to the chip with a serial terminal:

$ picocom /dev/ttyUSB0 -b115200

And I got greeted with the MicroPython REPL! Fantastic!

Hooking up the sensors

Most of my sensor platforms will be a soil moisture sensor running with a LM393 comparator chip and a DHT 11 or DHT 22 sensor.

The DHTs emit a digital sensor over a single wire interface, so you hook their positive lead to a 3.3v supply from the 8266, hook the negative lead to the ground on the 8266 and the middle data lead to one of the digital GPIO pins on the ESP8266.

The moisture sensor unfortunately only gives an analog signal depending on how moist the soil is, between 0 and 1 volts. This will read as a value between 0 and 1024 using the ADC module within micropython--we'll come back to that later.

As with the DHT sensor, the positive lead goes to 3.3v on the ESP8266, the negative lead goes to Ground and the analog out of the chip goes to the analog in pin of the 8266--there's only one analog input pin, in the upper left of the pinout diagram. It's labeled as "ADC0" or "A0" on the board I'm using.

One thing to note about the LM393 chip, it contains a potentometer that sends a digital signal when the analog value crosses a threshold--a "0" for below, and "1" for above the threshold. The potentometer can be adjusted with a screwdriver and some patience. For my uses, I care more about having the analog-ish value.

Now you'll note by now we've used two of the available three 3.3 rails of the ESP8266. It may be possible to split those lines to multiple directions, but I am not an electrical engineer and have no idea if that would twok, so proceed with caution.

I'm also planning to supply power for the 8266 via it's USB port. Your plans may be different, but I'm a simple fellow.

Writing the program

So, as a bonus of using Micropython, I don't have to write C code. Since I'll be dealing with network interactions and data collection, I'm very happy with this.

Now, the downside is I don't get fine-grained control over memory, and Micrpython inhabits more space on the device's program storage than an equivalent C binary would. I'm not planning to squeeze every little bit of performance out of the chip, so I'm okay with sacrificing a bit of space and management in favor of simpler code.

Micropython offers several out-of-the-box modules to do what I need for sensor data collection and network communication. The binary for the 8266 also includes functions to hook into the network fairly easily. I recommend spending some time in the REPL via a serial link when you first get your binary flashed to get comfortable with the available functionality.

Connecting to the Network

This is perhaps the meat and potatos of the ESP8266, and why you're using it in the first place: the damn thing is 2.4ghz wifi capable. Micropython is aware of this and incudes quick instructions for connecting in the help() function available from the repl, and their docs include this function:

def do_connect():
    import network
    sta_if = network.WLAN(network.STA_IF)
    if not sta_if.isconnected():
        print('connecting to network...')
        sta_if.active(True)
        sta_if.connect('<essid>', '<password>')
        while not sta_if.isconnected():
            pass
    print('network config:', sta_if.ifconfig())

This function, when called from your program, will block execution until the 8266 connects to the wifi network with the specified password.

I've modified this a bit to blink the built-in LED on my board while it connects, so I can get an approximate idea of the state of the device. ( Weird Note: for some reason, led.on() will turn the led off, and led.off() will turn it on. Dunno why.) You'll also note I pull the network name and password out to variables that I load from a config file:

def do_connect():
    global led_state
    sta_if = network.WLAN(network.STA_IF)
    if not sta_if.isconnected():
        print('connecting to network...')
        sta_if.active(True)
        sta_if.connect(NETNAME, NETPASS)
        while not sta_if.isconnected():
            if led_state:
                status_led.on()
                led_state = False
            else:
                status_led.off()
                led_state = True
            time.sleep(0.25)
    print('network config:', sta_if.ifconfig())
    status_led.off()

See? Nice and painless. (Aren't you glad this isn't C?)

Connecting to the Sensors

DHT Sensors

Right, so you're connected to wifi. You're not talking to anything on the network yet, but that's okay, we'll come to that later.

The micropython binary we're using includes a library for using DHT sensors, since they're apparently crazy popular. I have a DHT11, which is slightly cheaper and has disgustingly large errors in reads (2 degrees C!). The DHT22 is a slightly more expensive (DHT11 is ~$2/per, DHT22 is ~$3.5/per), but more accurate, at the cost of how often you can read from the sensor. We'll come to that too.

So, hooking up the DHT sensors is stupid easy. We pull in the machine module, to access the hardware pin, and the DHT module to communicate with whatever's on that pin:

d = dht.DHT11(machine.Pin(5))

Wow, that was easy.

Now, Important note: The number you hand to machine.Pin is the number of the 8266 pin , not of whatever handy board you're using. Pay attention to your pinout diagrams if you're having trouble connecting the sensor.

Anyway, now that we have this DHT11 object we can tell it to measure:

d.measure()

Which updates these values:

d.temperature(); d.humidity();

Another note, on the DHT11, you can only call measure once every second or so. On the DHT22, you can only read once every two seconds. I don't know why this is specifically, or what the consequences are of not obeying that. Again, use caution if you're coloring outside the lines.

Soil Moisture Sensor

As I mentioned before, the soil moisture sensor--as we have it hooked up--will give back an analog signal. We're going to have the ESP8266 interpret that signal as a digital value.

Here, we're making an ADC object off of that pin:
python

adc = machine.ADC(0)adc.read()

adc.read() will yield a value between 0 and 1024. Apparently 1024 means "Dry" and "0" means "wetter than water", since I stuck the sensor in a cup of water and only got a reading of something like 300. I dunno why it's this way, but it is. Use as you will.

Network communication

Okay, so we've got our data, we're connected to the wifi, now we need to take the data from the microcontroller we're on and shove it somewhere useful. Micropython includes a socket module that behaves just like the normal module in [Macro]?Python. In my code, I have the IP and port of the server application recorded in a config file, and so we open the socket and start hammering data out to it:

socket_connected = False

while True:

    while not socket_connected:
        try:
            s = socket.socket()
            s.connect((SERVIP, SERVPORT))
            s.send("Initializing link...\n")
            socket_connected = True
        except OSError:
            print("Socket connection failed... waiting.")
            # pulse LED to indicate problem
            status_led.on()
            time.sleep(0.1)
            status_led.off()
            time.sleep(0.1)
            status_led.on()
            time.sleep(0.1)
            status_led.off()
            time.sleep(0.1)
            status_led.on()
            time.sleep(0.1)
            status_led.off()

    status_led.on()
    d.measure()
    msg = "Soil: " + str(adc.read()) + "; temp: " + str(d.temperature()) + "; hum: " + str(d.humidity()) + "; from " + mac + "\n"
    print(msg)
    try:
        q = s.send(msg)
    except OSError:
        socket_connected = False
    print("Sent ", q, " bytes.")
    status_led.off()
    time.sleep(2)

Walking through this:

We drop into a forever loop, so we'll collect and transmit data until the end of time.

While we don't have an active socket connection, try to make one. If the server isn't listening, the attempt to open a socket will timeout and we'll flash the LED angrily and try again.

(I need to update this code to also check and make sure we still have a wifi link--no point in trying to transmit data if we aren't connected anyway.)

If we have a socket connection, we aren't trapped in that loop and we continue on to the meat of the loop.

We update the temperature sensor, then construct a message to transmit. This message can take whatever form you really want, I'm pretty sure it gets converted to bytes, so if you really wanted you could send raw binary data. Right now I'm just sending strings to make the dbeugging easier for me.

I also append the mac address of this board so I can identify which sensor the readings are coming from, when I'm on the server. I obtain the mac address with this line. which I stole from here:

mac = ubinascii.hexlify(network.WLAN().config('mac'),':').decode()

Once we construct the message I print it, which dumps it to serial output, which makes debugging a cinch, and I write it to the socket.

The sleep at the end gives the temperature sensor a moment to recuperate, and gives us a reasonable rate of data flow.

So, with this, you have a microcontroller environmental sensor that can send data over the network!

You can view all of my project code here.