ESP32 Deep Dive: What Time Is It?
Updated on 2021-03-08
Traipsing through the ESP-IDF to add some sweet sweet real-time clock functionality
Note: Platform I/O will download the esp32_i2c and rtc_ds1307 libraries on your behalf as part of the ds1307_example project, regardless of whether or not you download them from the above. If you have any issues getting Platform I/O to build it after you first open it, in the tasks, open up a new PlatformIO CLI prompt and in it type "pio run" which should force it to download and build everything.
Introduction
Anyone can program an ESP32 using the Arduino framework, but at the cost of flexibility, features, and potentially code size, depending on what you're doing. Getting serious with the ESP32 means forgoing the Arduino framework that wraps it and tinkering with the Espressif IoT Development Framework (ESP-IDF) itself.
The ESP-IDF is the core "operating system" of the ESP32. While not actually an operating system, it does come with one, called FreeRTOS, but it provides features that one comes to expect from them, like fundamental hardware interfacing and I/O, system configuration, and things like that.
The ESP-IDF was written in C, but we'll be using it from C++ with a little bit of massaging.
In this article, we are going to explore how to expose an I2C bus host in master mode to communicate with one or more slave I2C devices on that bus. We will then use these facilities to communicate with a DS1307 real time clock over that bus.
There are already libraries for Arduino that do this, but this is for those of you that don't want to depend on the Arduino framework, or otherwise want to explore the ESP32 in more depth.
Prerequisites
This is a hardware project and as such, you'll need some basic components. Except for the ESP32 devkit itself, most, if not all of these components come with Arduino starter kits:
- An ESP32 based devkit. This project was built using an ESP32-WROOM on a devkit01 board
- A DS1307 real-time clock module operating at +5vdc (+3.3vdc would actually be "cleaner" but I don't have one. Change your power inputs on the module if you do. You'll want to use +3.3vdc on your VIN in that case. In this one, we use +5vdc.
- 2x 4.7k resistors. I've gotten by with the internal pullups sometimes. Othertimes, not so much. External resistors are best but if you don't have them, you can try just using the internal ones. It may or may not work, or if you have my luck, it will work at first, and then your clock will just stop working one day until you put pullups on the bus.
- The obligatory collection of wires and solderless PCB boards, and a microUSB cable.
You'll also need Platform I/O installed since I don't want to force the great, but commercial VisualGDB on you.
Wiring this Mess
Wiring it is pretty easy. The following assumes a +5vdc DS1307 based clock module:
ESP32 GPIO21 ➟ DS1307 SDA ➟ 4.7k resistor #1 ➟ ESP32 +3.3VDC
ESP32 GPIO22 ➟ DS1307 SCL ➟ 4.7k resistor #2 ➟ ESP32 +3.3VDC
ESP32 +5VDC/VIN ➟ DS1307 +5VDC/VIN
ESP32 GND ➟ DS1307 GND
Conceptualizing this Mess
This project consists of two separate libraries and an example. We have the esp32_i2c library used for communication with devices on an I2C bus, and we have the rtc_ds1307 library which uses esp32_i2c to communicate with a DS1307 based real time clock module. In a sense, rtc_ds1307 serves as the example code for esp32_i2c. Finally, we have the example PlatformIO example project that uses the above libraries.
Design Patterns and Considerations
The classes in these libraries use the C++ RAII pattern to handle resource acquisition and release. However, because using this pattern means non-trivial calls must be made from class constructors, the major classes have an initialized() accessor that can be queried to determine if the object was successfully initialized. It should be checked after object creation to ensure that it didn't fail, because...
We don't use exception handling in these libraries, because the stack frame bloat on a little IoT device like the ESP32 is somewhat prohibitive. We use a C-ish mechanism where our objects report the last error as a numeric value. Most members return a bool and if they return false than the last error can be retrieved through the last_error() accessor. All i2c components share the same last error, accessed through esp32::i2c::last_error(). The clock as its own last_error() accessor.
I2C Communication
I2C is a two-wire protocol that allows devices to communicate serially across short distances at frequencies of around 100kHz. They're common, easy to wire, and a single master host can control multiple slave devices.
The downside in terms of hardware is it's not any good for ranges of more than half a meter, if that, and it can get finicky as you start adding more devices to it. The software downside is it can be a difficult to program because the communication protocol is very sparse in terms of standard behaviors one can expect - every device reads and writes registers in a similar fashion, but not necessarily the exact same fashion, for example. The protocol is basically extremely low level and it's up to each device to flesh out the features they need on top of that, however they see fit along a sort of unwritten de facto common guidelines. That "however they see fit" part can translate to headaches.
We're not concerning ourselves with using the ESP32 as a slaved I2C device in this article because it's just not a very common use case, so all of this is focused on the master side of things that hosts the devices.
I2C Commands
The ESP-IDF uses a queuing technique to ensure that a series of reads and writes happens as a unit, which helps keep timing correct and reduces the code burden for doing so. The idea is that instead of reading and writing the bus directly, you issue reads and writes to a "command link" that queues them up, and executes them once they're all teed up and ready, as a batch.
I2C Registers
Registers seem to be a kind of loose standard. There's no specific operation on an I2C bus that acknowledges the existence of something called a "register", or anything in code to indicate they exist. However, every single device I've encountered exposes them by way of a series of read/write handshakes specifying a register id. I'll explain.
On most devices, to read or write a register, you:
- Write a byte: The 7-bit destination I2C address, shifted left, and then binary OR'd with a flag that indicates read (1) or write (0), so like
address<<1|I2C_MASTER_READoraddress<<1|I2C_MASTER_WRITE. - Write a byte: The register id/code value. Each register has some unique one byte identifier and that's what goes here.
- Read or write a series of bytes. The length to be read or written depends on the register and the device. You have to know what to expect, and what it expects.
Greatly complicating this is that each distinct read or write operation can have various ways to handshake/verify it was sent through ACKs and NACKs, and for example, with the clock, when you read a register, the last byte is always read using NACKs but the prior bytes are read using ACKs. I'm not sure how universal this is. It all comes down to the documentation for the hardware you intend to interface with, and getting it wrong can mean timeouts or just non-functional features. It's not for the faint of heart. If you plan to do a lot of development of wrapper classes for I2C based hardware, I recommend getting an oscilliscope, and learning the I2C bus protocol at the hardware level. Seriously.
I've provided some helper functions to facilitate reading and writing these registers, like begin_read() and begin_write() but I don't recommend using read_register() or write_register() yet since I'm not fully confident in their general usefulness.
Start and Stop Signals, and Restarting
One additional wrinkle with these devices, is they expect start and stop signals during batch reads and writes. Often, it goes start➟writes➟stop, but for commands where reading and writing both are taking place during the same batch, you'll need another start signal in there, so it's more like start➟writes➟start➟reads➟stop. I've never seen more than one stop signal in any command batch, but I won't say the situation doesn't exist. My experience is limited in that respect.
DS1307 Module
The DS1307 based real time clock module you have provides a higher precision alternative to the timing capabilities of the ESP32 as well as providing timing while the rest of the circuit is off, via an integrated battery. It also has logic to increment the date and time each second. Taken together, these features create a real-time clock. This module communicates via an I2C interface, as well as two additional pins for power, and one for a square wave output that other devices can use to sync their own timing to an accurate signal.
I'm pretty sure it's possible to use the square wave signal from this clock, by wiring it back into the ESP32 as a "crystal" input source which the ESP32 can use to handle its own timing, making the C library routines for time and such actually have the proper resolution and things that they need to perform as a real clock. Unfortunately, I haven't managed to get it to work yet, and I may need an oscilliscope to track down the issue. Once I do, I'll update the library and article to reflect that, and you'll be able to use the built in C and C++ time libraries with this clock.
Unless you're going to be syncing other hardware to the timing of your clock, the square wave feature isn't especially useful, and we won't be using it in our example. As I suggested above, once I get this syncing feature working with the ESP32, I'll add more here to reflect the changes, including covering the core time facilities in the C and C++ standard libraries it will open up.
Lifetime of Hardware Wrapper Classes
In a normal computer, with a regular operating system, you don't have hardware wrapper classes as such. You have drivers. Drivers virtually always exist from the time you boot up until after your OS powers off. Typically, you'll want to do something similar with anything that wraps a hardware device on this platform. You can do so by declaring your hardware wrapper classes as global variables. I normally don't care for globals, but if there ever was a good case for them it's here. Globals are ever-present and accessible to anyone that knows what to look for. So is hardware. It's a nice fit.
However, I've found that there's an issue with the ESP32's initialization process wherein it is not safe to start establishing communication with I2C slave devices before the app_main() entry point has been hit. Doing so can cause a crash/reboot loop.
This creates a problem with our RAII pattern wherein when we go to initialize or even check for the existence of a slave device from a wrapper class - like say, from the DS1307 clock module via the ds1307 class constructor - it will cause a crash. I am not sure why, except to say that it is apparently unsafe to do I2C bus operations until after app_main() starts. This hamstrings us in terms of our ability to declare the ds1307 clock wrapper as a global variable, because it would cause the constructor to fire too early. That's not acceptable. We must be able to declare this as a global, if we want.
We work around this problem by actually not initializing the device from the constructor. Instead, we simply take all the data we need to initialize it as constructor arguments and then save them for later. Upon first use of any of the clock functions, or if initialize() is called explicitly, at that point the clock is initialized. This isn't perfect. It fudges, if not breaks the RAII pattern since "resource acquisition is initialization" is no longer true - the initialization is lazy. At least it plays nicely with RAII so our code that uses it doesn't have to concern itself much with the difference. Just know that you won't actually know if the clock is even present until you either try to actually use it for the first time, or you call initialize().
When you write wrappers for other devices in the future, bear the above in mind, as your wrappers will probably have to take a lazy initialization approach as well.
Coding this Mess
Using the ds1307 Wrapper Class
I find leading with code to be effective, so here it is:
#include <iostream>
#include <i2c_master.hpp>
#include <ds1307.hpp>
extern "C" {void app_main();}
using namespace std;
using namespace esp32;
using namespace rtc;
// compile time date portion
// adapted from dc42's solution:
// https://stackoverflow.com/questions/17498556/c-preprocessor-timestamp-in-iso-86012004
constexpr uint8_t compile_year = (uint8_t)((__DATE__[7] - '0')
* 1000 + (__DATE__[8] - '0')
* 100 + (__DATE__[9] - '0')
* 10 + (__DATE__[10] - '0')
-1900);
constexpr uint8_t compile_month = (uint8_t)((__DATE__[0] == 'J') ?
((__DATE__[1] == 'a') ?
1 : ((__DATE__[2] == 'n') ?
6 : 7)) // Jan, Jun or Jul
: (__DATE__[0] == 'F') ? 2 // Feb
: (__DATE__[0] == 'M') ?
((__DATE__[2] == 'r') ?
3 : 5) // Mar or May
: (__DATE__[0] == 'A') ?
((__DATE__[2] == 'p') ?
4 : 8) // Apr or Aug
: (__DATE__[0] == 'S') ? 9 // Sep
: (__DATE__[0] == 'O') ? 10 // Oct
: (__DATE__[0] == 'N') ? 11 // Nov
: (__DATE__[0] == 'D') ? 12 // Dec
: 0)-1;
constexpr uint8_t compile_day = (uint8_t)((__DATE__[4] == ' ') ?
(__DATE__[5] - '0') : (__DATE__[4] - '0')
* 10 + (__DATE__[5] - '0'));
constexpr uint8_t compile_hour = (uint8_t)(__TIME__[0]-'0')
*10+(__TIME__[1]-'0');
constexpr uint8_t compile_minute=(uint8_t)(__TIME__[3]-'0')
*10+(__TIME__[4]-'0');
constexpr uint8_t compile_second=(uint8_t)(__TIME__[6]-'0')
*10+(__TIME__[7]-'0');
// initialize our master i2c
// host:
i2c_master g_i2c;
// initalize our clock
ds1307 g_clock(&g_i2c);
void app_main() {
tm tm;
// we're going to set the time and
// date to the time and date this
// code was compiled:
tm.tm_year = compile_year;
tm.tm_mon = compile_month;
tm.tm_mday = compile_day;
tm.tm_hour = compile_hour;
tm.tm_min = compile_minute;
tm.tm_sec = compile_second;
tm.tm_isdst = -1;// for daylight savings. we don't know
if(!g_clock.set(&tm)) {
cout << "Error setting clock: "
<< g_clock.last_error()
<< " "
<< esp_err_to_name(g_clock.last_error())
<< endl;
}
// now just print the date and time every second
while(true) {
if(!g_clock.now(&tm)) {
cout << "Error reading clock: "
<< g_clock.last_error()
<< " "
<< esp_err_to_name(g_clock.last_error())
<< endl;
} else {
cout << asctime(&tm);
}
// delay for 1 second
vTaskDelay(pdMS_TO_TICKS(1000));
}
}A big portion of this is simply cracking apart the DATE and TIME macros the compiler defined for us. They return their info in a human readable string and that's not what we want, so the code in there parses it out for us at compile time, which is one of my favorite features of C and C++ - schlepping mundane tasks like this from runtime/initialization to compile-time where they belong. Things like that make me happy.
After that, once the app starts, we just shove all those values we got above into a tm structure and set the clock with it.
Note: I've had issues, particularly after being irresponsible with my voltages or pin assignments to the clock in a project and it getting basically reset and taken out of the running state (g_clock.running() == false) due to me tinkering with it on the bench. This is a hardware, not software issue. However, once it's no longer in the running state, the only way I've found to bring it back to the running state is to set the clock again through software. If your clock will initialize but won't give you time, and it reports that it's not running() you should set the clock. If anyone knows of another way to kickstart the DS1307 clock after it is no longer in the running state, please let me know in the comments.
Anyway, after we set it we just retrieve the time in a loop and print it once a second.
How It Works: The I2C Communication
When we created the clock, right before we did so, we created an i2c_master instance:
// initialize our master i2c
// host:
i2c_master g_i2c;
// initalize our clock
ds1307 g_clock(&g_i2c);Upon creation, the i2c_master initializes the specified I2C "host slot" with your specified pins and settings. We've used the defaults, which is why there are no parameters to the constructor. We'll look at the constructor now:
i2c_master(
i2c_port_t i2c_port=I2C_NUM_0,
gpio_num_t sda=GPIO_NUM_21,
gpio_num_t scl=GPIO_NUM_22,
bool sda_pullup=true,
bool scl_pullup=true,
uint32_t frequency=100000,
int interrupt_flags=0
) : m_initialized(false) {
m_configuration.mode=i2c_mode_t::I2C_MODE_MASTER;
m_configuration.sda_io_num = (int)sda;
m_configuration.scl_io_num = (int)scl;
m_configuration.sda_pullup_en = sda_pullup;
m_configuration.scl_pullup_en = scl_pullup;
m_configuration.master.clk_speed=frequency;
esp_err_t res = i2c_param_config(i2c_port, &m_configuration);
if (res != ESP_OK) {
i2c::last_error(res);
return;
}
res = i2c_driver_install(i2c_port, m_configuration.mode, 0, 0, interrupt_flags);
if (res != ESP_OK) {
i2c::last_error(res);
return;
}
m_port=i2c_port;
m_initialized=true;
}I don't normally like using that many parameters for a constructor. I'd prefer a ref struct to avoid hammering the stack but I didn't want to take away the optional arguments, which I think are a critical feature of this code. Without them, setting up a bus would be very difficult. Given that this initialization typically happens once, the small performance hit isn't of much concern. The settings are suitable for the primary I2C bus slot 0 on GPIO pins 21 (SDA) and 22 (SCL) which is what we're using in this case. We shouldn't need the internal pullup resistors here on SDA and SCL even though they default to true - we already included our own in the circuit that are much more reliable - but having them doesn't hurt, and will hopefully limp along for those of you that were naughty and skipped adding the 2 4.7k resistors to your circuit. Leave the frequency unless you know you need to change it.
What this does is it simply builds a configuration struct with all your parameters, and stashes it for later, and then it tells the ESP-IDF to use it to configure parameters (I don't actually know what this does - only that you need to do it), and then installs the driver. If there's an error at any point, it sets the last_error() result and then returns early. Finally, it stores the final piece of configuration information - the host port, or slot and indicates that it has been initialized.
If you look through the source code for i2c_master, you'll see the destructor doesn't do a whole lot, which might be surprising given the setup we did in the constructor. As far as I can tell, there's no way to uninstall or otherwise tear down an I2C host slot once it has been installed. It's on until reboot. I suppose it makes sense, give that it's basically just a way to configure a hardware bus, but it's something to be aware of, since you can't just dynamically create and destroy host slots as you please, using the ESP-IDF.
Other than that, the only other thing of primary interest is the execute() method:
bool execute(const i2c_master_command& command,
TickType_t timeoutTicks=portMUX_NO_TIMEOUT) {
esp_err_t res = i2c_master_cmd_begin(m_port,command.handle(),timeoutTicks);
if(ESP_OK!=res) {
i2c::last_error(res);
return false;
}
return true;
}I wrote about queuing up read and write operations as a batch command, and then sending it off to be executed. This is that last bit. We'll get to the batch commands in a moment. All this does is execute the commands given the specified timeout.
Note: I strongly suggest avoiding the default value of no timeout here, and frankly the main reason I haven't removed it as a default is I don't have a saner value to offer. The only time no timeout is the most reasonable choice is if your application is such that if the device isn't communicating it represents catastrophic failure of the entire application, so any halt that would happen is moot - in other words, no timeout is okay if a failure to communicate with the device would end the application anyway.
Packaging Read and Write Operations as Commands
All read and write operations are contained in i2c_master_command instances. You create one of these, at any point, and fill it by issuing read and write operations to it, and finally, execute it using i2c_master's execute() method when you're finished.
When reading values, you must specify a pointer that gives the address of the destination data that will be read. Note I said "that will be!" - these pointers you give it are not even accessed until execute() is called, and it's up to you to make sure they still point to valid locations at that time. This is pretty easy as long as you always create your i2c_master_command instance and then use it with execute() in the same method where you created it, right after you're done populating it. Here's an example. This writes one byte to the "control register" (id of 7) of the DS1307 clock, which is used to set the integrated square wave's cycle frequency:
esp32::i2c_master_command cmd;
if(!cmd.start()) {
last_error(esp32::i2c::last_error());
return false;
}
if(!cmd.begin_write(m_address,true)) {
last_error(esp32::i2c::last_error());
return false;
}
if(!cmd.write(0x07,true)) {
last_error(esp32::i2c::last_error());
return false;
}
if(!cmd.write((uint8_t)value)) {
last_error(esp32::i2c::last_error());
return false;
}
if(!cmd.stop()) {
last_error(esp32::i2c::last_error());
return false;
}
if(!m_i2c->execute(cmd,5000/portTICK_PERIOD_MS)) {
last_error(esp32::i2c::last_error());
return false;
}
return true;Here, aside from the error handing, and the propogation of the most recent I2C error to the clock's last_error(), we have start➟begin_write➟write➟write➟stop and then we execute.
We're expecting ACKs after each write, which I've found is the most common case. We've used begin_write() above to tell the command we're going to target the device at the I2C addresss indicated by m_address for a register write operation. We use the next write to tell it which register to target (0x07). Finally, any following writes until a start() or stop() call are sent to that register.
I gave it a 5 second timeout on the execute() call, but I've never actually had it take that long to fail. It's a fairly arbitrary value. Basically, I picked the longest timeout that didn't make me want to gouge my own eyes out waiting for it, but in practice, with this device, that timeout won't be hit in any situation I've thrown at it. I haven't tried it while it's actually on fire though, yet.
Reading is slightly more complicated, because there's usually another start call in there, plus there are just extra considerations for reading data. Here's a corollary to the above code - this retrieves the current square wave cycle frequency value, using a register read operation. In this case, this particular register on this device is kind of funky about reads. It expects you to write to it, but write 0 bytes of data, before then reading the register:
esp32::i2c_master_command cmd;
if(!cmd.start()) {
last_error(esp32::i2c::last_error());
return false;
}
if(!cmd.begin_write(m_address,true)) {
last_error(esp32::i2c::last_error());
return false;
}
if(!cmd.write(0x07,true)) {
last_error(esp32::i2c::last_error());
return false;
}
if(!cmd.start()) {
last_error(esp32::i2c::last_error());
return false;
}
if(!cmd.begin_read(m_address,true)) {
last_error(esp32::i2c::last_error());
return false;
}
uint8_t r;
if(!cmd.read(&r,I2C_MASTER_NACK)) {
last_error(esp32::i2c::last_error());
return false;
}
if(!cmd.stop()) {
last_error(esp32::i2c::last_error());
return false;
}
if(!m_i2c->execute(cmd,5000/portTICK_PERIOD_MS)) {
last_error(esp32::i2c::last_error());
return false;
}Notice how as soon as we wrote the register value 0x07, we called start() instead of writing a value to that register? That's the funny business I mentioned above.
A normal register read would start with start➟begin_read➟read...➟stop
The clock expects start➟begin_write➟write➟start➟begin_read➟read➟stop
I emphasized the funny business above. The clock has a funny way of querying it. This is the sort of silliness that I alluded to at the beginning of the article. Devices are unique like snowflakes, so make sure you know yours if you intend to write a wrapper for it.
I'm not omniscient, nor did I slave over this device with an oscilliscope. What I did, is I rather shamelessly scanned code from several examples, and from existing libraries (even though they use an entirely different I2C framework) and collated and cross referenced that with information I gleaned from the manual for a DS1307, before fiddling and massaging what I ended up with to come up with the appropriate I2C I/O for this gadget.
That should cover enough to both give you a usable real time clock library, and a path to create wrappers for additional I2C based devices now that we've unpacked how they operate. Happy creating!
History
- 8th March, 2021 - Initial submission