For my m8rScript project I really want to use Bonjour. It would give me a way of finding many ESP8266 devices on the network without having to nail down IP addresses. The concept is simple: you give your device a name, like yardweather. Then you type yardweather.local into your web browser and up pops the web page for your device. Additionally, you can broadcast a message on your network asking if any device supports a given service, and any device that does sends back information telling you about itself. So you can know how you are allowed to talk to it. Simple, right?
NO
I tried lots of options for the ESP8266. BUT NONE OF THEM WORKED! To make matters worse The documentation is terrible. In fact, maybe the reason none of the implementations work is because of the bad documentation.
Ok, I’m being a little unfair. All of the implementations work somewhat, but none have all the features to let you achieve the Nirvana of being able to both find out how to contact the device and what the device is capable of. So I started on a trek, first to understand what mDNS is all about and second to write an implementation for the ESP8266.
What is mDNS?
It goes by many names: Multicast DNS is it’s official IETF name, as documented here. Apple invented it and they call it Bonjour. They originally wanted to call it Rendezvous, but that name was taken, so they switched to the equally French Bonjour. And all this falls under the umbrella of Zero Configuration Networking (zeroconf).
When I started looking around I would search for “mdns packet format” and things like that and it would keep coming back with DNS hits. It didn’t occur to me at the time that mDNS IS actually a particular flavor of the Domain Name System (DNS), which is pretty much the backbone of the web. With that new understanding, a lot of the terminology and descriptions made a lot more sense. You ask a question (“what is the IP address for this domain name“) and get back an answer (“1.2.3.4”). Then you can contact that IP address and start a conversation. The two interesting bits about mDNS are a) the types of questions you ask and the responses you get back, and b) the fact that it uses multicast. Normally you ask DNS questions of a server with a particular IP address, which is added to your computer when it is first set up. Multicast just sends its questions out into the universe and then waits a while for answers to come back, often from many different sources.
For the ESP8266 the multicast part is easy. It uses a networking technology called UDP, which is well supported on the ESP8266. The hard part is knowing which questions you need to respond to and what answers you need to give. There’s no source of information that lays all that out clearly, so I had to piece it all together from a number of sources. For some things, I just had to start sending out test questions and then use wireshark to look at the answers being given back.
I work at Apple, so everything in my house is Apple compatible. And Bonjour was invented by Apple, so every Apple product speaks Bonjour. Lots of non-Apple products, like printers and TV sets speak it to. So when I started sending out questions there was no shortage of answers. It took quite a while, but I finally figured out the correct responses and where in the response packets the answers had to go.
Note: Microsoft and Android don’t speak mDNS out of the box, so if you own one of those devices you’ll have to try one of the 3rd party products that can do it for you.
Bonjour does two things that are interesting to me. The first is called Announcement. This is where a questioner asks “what is the IP address of yardweather.local?”. And you need to respond with the IP address of your device. This is simple and most of the existing implementations get this right. The harder part is Discovery, This is where the questioner asks “what device supports the http service” and any device that does responds with its IP address, the port it’s listening on and other relevant information. This is a function that none of the implementations supported correctly. Most didn’t even try and the ones that did either had malformed packets (as reported by Wireshark) or sent them in the wrong section of the response, so the Bonjour implementation on the Mac didn’t recognize them.
Starting over
I knew I wanted to write my own fresh mDNS implementation. I just needed a starting point. There are many ESP8266 toolkits and many of them have mDNS built in. Even the official SDK from Espressif has one. And there are a few standalone implementations out there. So there was plenty of code to look at. And it was mostly useful. Even though none of the code was fully functional, I saw all the basics of packet encoding and decoding. So I took what I saw and made my own MDNSResponder class. It lets you setup a hostname (so <hostname>.local works). Then it lets you create one or more services. This is where you say you support the http service on port 80, for instance. In my case I wanted to respond that I support a custom protocol for talking to the ESP8266 file system, so I could talk to my IoT devices from a macOS app I am writing.
The code is available on github as part of m8rscript, my custom scripting language made specifically for the ESP8266. The code is still pretty fresh, but already it’s paying dividends. I wrote a m8rsim macOS app which listens for mDNS events. If a ESP9266 with my code is running when m8rsim starts up it will send a PTR response, which macOS recognizes and can talk to the ESP8266 using its Bonjour name. Not only that, but m8rsim is always listening, so if another ESP8266 comes online it will announce itself and will get added the list of known devices. It really makes ESP8266 development much easier than ever before for me.
Stay tuned for move developments with my m8rscript/m8rsim system
Leave a Reply