Multicast DNS Tribulations

For my m8rScript project I really want to use Bonjour. It would give me a way of finding many ESP8266 devices on the network without having to nail down IP addresses. The concept is simple: you give your device a name, like yardweather. Then you type yardweather.local into your web browser and up pops the web page for your device. Additionally, you can broadcast a message on your network asking if any device supports a given service, and any device that does sends back information telling you about itself. So you can know how you are allowed to talk to it. Simple, right?

NO

I tried lots of options for the ESP8266. BUT NONE OF THEM WORKED! To make matters worse The documentation is terrible. In fact, maybe the reason none of the implementations work is because of the bad documentation.

Ok, I’m being a little unfair. All of the implementations work somewhat, but none have all the features to let you achieve the Nirvana of being able to both find out how to contact the device and what the device is capable of. So I started on a trek, first to understand what mDNS is all about and second to write an implementation for the ESP8266.

What is mDNS?

It goes by many names: Multicast DNS is it’s official IETF name, as documented here. Apple invented it and they call it Bonjour. They originally wanted to call it Rendezvous, but that name was taken, so they switched to the equally French Bonjour. And all this falls under the umbrella of Zero Configuration Networking (zeroconf).

When I started looking around I would search for “mdns packet format” and things like that and it would keep coming back with DNS hits. It didn’t occur to me at the time that mDNS IS actually a particular flavor of the Domain Name System (DNS), which is pretty much the backbone of the web. With that new understanding, a lot of the terminology and descriptions made a lot more sense. You ask a question (“what is the IP address for this domain name“) and get back an answer (“1.2.3.4”). Then you can contact that IP address and start a conversation. The two interesting bits about mDNS are a) the types of questions you ask and the responses you get back, and b) the fact that it uses multicast. Normally you ask DNS questions of a server with a particular IP address, which is added to your computer when it is first set up. Multicast just sends its questions out into the universe and then waits a while for answers to come back, often from many different sources.

For the ESP8266 the multicast part is easy. It uses a networking technology called UDP, which is well supported on the ESP8266. The hard part is knowing which questions you need to respond to and what answers you need to give. There’s no source of information that lays all that out clearly, so I had to piece it all together from a number of sources. For some things, I just had to start sending out test questions and then use wireshark to look at the answers being given back.

I work at Apple, so everything in my house is Apple compatible. And Bonjour was invented by Apple, so every Apple product speaks Bonjour. Lots of non-Apple products, like printers and TV sets speak it to. So when I started sending out questions there was no shortage of answers. It took quite a while, but I finally figured out the correct responses and where in the response packets the answers had to go.

Note: Microsoft and Android don’t speak mDNS out of the box, so if you own one of those devices you’ll have to try one of the 3rd party products that can do it for you. 

Bonjour does two things that are interesting to me. The first is called Announcement. This is where a questioner asks “what is the IP address of yardweather.local?”. And you need to respond with the IP address of your device. This is simple and most of the existing implementations get this right. The harder part is Discovery, This is where the questioner asks “what device supports the http service” and any device that does responds with its IP address, the port it’s listening on and other relevant information. This is a function that none of the implementations supported correctly. Most didn’t even try and the ones that did either had malformed packets (as reported by Wireshark) or sent them in the wrong section of the response, so the Bonjour implementation on the Mac didn’t recognize them.

Starting over

I knew I wanted to write my own fresh mDNS implementation. I just needed a starting point. There are many ESP8266 toolkits and many of them have mDNS built in. Even the official SDK from Espressif has one. And there are a few standalone implementations out there. So there was plenty of code to look at. And it was mostly useful. Even though none of the code was fully functional, I saw all the basics of packet encoding and decoding. So I took what I saw and made my own MDNSResponder class. It lets you setup a hostname (so <hostname>.local works). Then it lets you create one or more services. This is where you say you support the http service on port 80, for instance. In my case I wanted to respond that I support a custom protocol for talking to the ESP8266 file system, so I could talk to my IoT devices from a macOS app I am writing.

The code is available on github as part of m8rscript, my custom scripting language made specifically for the ESP8266. The code is still pretty fresh, but already it’s paying dividends. I wrote a m8rsim macOS app which listens for mDNS events. If a ESP9266 with my code is running when m8rsim starts up it will send a PTR response, which macOS recognizes and can talk to the ESP8266 using its Bonjour name. Not only that, but m8rsim is always listening, so if another ESP8266 comes online it will announce itself and will get added the list of known devices. It really makes ESP8266 development much easier than ever before for me.

Stay tuned for move developments with my m8rscript/m8rsim system

The Internet of Things

If you have even one nerd bone in your body, you’ve heard about the Internet of Things or IoT. It’s all the rage. There used to be a joke about connecting your toaster to the internet. It was a metaphor for just how crazy things might get someday if left unchecked. Well, that day is here

Posted in IoT

m8rScript

The Internet of Things is really taking off. The ESP8266 has made it possible to add a powerful process with wifi to any project for a couple of bucks. Whether you doing a one off art project, or are building a home automation gizmo that you plan on funding with Kickstarter, the ESP8266 can be at the core.

I’ve gotten really excited about this new chip and it’s potential, so naturally I decided to write my own scripting language. When you say that sort of thing in the circles I run around in or when you read it in ESP8266 forums, you generally get a lot of push back. Once the rolling of the eyes and general incredulity settle down, you are likely to get any combination of these 3 responses:

  1. Anyone who writes a new scripting language for any reason on any platform is nuts. There’s nothing you could possibly do that hasn’t already been done.
  2. The ESP8266 already has support for many scripting languages already, based on excellent memory efficient and powerful standards (Javascript, Python, Lua, Basic). Trying to add something new, especially a language not based on an accepted standard, is just nuts.
  3. Scripting languages on a platform like ESP8266 incur useless overhead. Native code FTW. You’re nuts.

TL;DR I’m nuts.

So what motivation could I possibly have for doing this? I’ll push back on each of the points above:

  1. There are plenty of reasons for writing a custom language specific to a particular platform. In the case of the ESP8266, it is both much more powerful that other solutions that have come before it (like the AVR chip in Arduino) and much less powerful than traditional computers for which most of these languages were written. So it makes sense to make a language that enhances its advantages while minimizing its constraints.
  2. All the languages currently on the ESP8266 were originally written for more powerful platforms. Even though the versions being ported are generally “micro” versions, they still push the limits of the platform. For instance, they typically leave less than 25KB of ram storage, It’s hard to do anything in that space. And they generally use garbage collection that adds even more overhead to the management of that tiny memory space.
  3. Native programming is fine as long as you want to upload new code every time you make a change. You might think this problem is solved with OTA. But how do you test your changes without crashing the device, which would require you to bring it back to the shop, crack it open and flash new firmware over the serial port. OTA was designed for mass deployment, not experimentation.

One of the great advantages of m8rscript is that I wrote it to be cross-platform. It runs on both ESP8266 and Mac. That has allowed me to write a macOS app which can simulate m8rscript code before deploying on the device.

To be continued…

Easy Library Table

LibraryTableCompMy kids are in high school now so I thought it was time they had nicer desks than the ones we got them at Ikea. I wanted to make something that would last, that they could take with them to college and beyond. I also decided I didn’t want a desk with drawers on the side, just one small drawer at the top for pencils and papers. This was both because they already have roll around cabinets for bigger things and because I didn’t want to complicate this project with a lot of drawers, which are harder to build.

library-cWhat I really wanted was not a desk at all, but a Library Table. I’ve always like Craftsman style furniture, so I went looking for plans. The internet is a great place for finding just about anything and I quickly came upon two sets of free plans as a starting point:

Screen Shot 2014-07-10 at 7.31.23 AMScreen Shot 2014-07-10 at 8.04.02 AM

 

 

 

 

 

I liked features of both of these desks. But both were pretty complicated, using mortise and tenon joints and wooden drawer slides. I can use a table saw and other power tools, but I’m no master craftsman, so I set out to simplify the plans and come up with something I could do a good job building.

Joinery

The first thing I needed was a joinery system. I was really worried about this because all the joinery I saw being used in the plans I looked at seemed beyond my abilities. So I did some research and decided on using dowels for the main leg supports, rabbet joints for the drawers and biscuits for everything else.

Wood

When it comes to wood there are lots of choices. You want to select an attractive hardwood for durability. But that doesn’t narrow it down much. Here in the San Francisco Bay Area there are many different woods available. If you go down to Southern Lumber you’ll see an incredible selection. But you’ll also see eye popping prices. I did a sample pricing using cherry wood. It would have been a beautiful desk for around $1500 or so. Not only is it expensive, but at that price I would have been terrified with every cut.

redoakSo I went down to Home Depot and found that they had a good supply of red oak. Oak is great wood, if not very exciting. It’s hard, but not very difficult to work with. By contrast, I have some Ipé from a deck I had installed. It is a hardwood too. But it’s super difficult to cut and produces sawdust that’s a real irritant. Your eyes water, your skin itches and you just can’t wait for the whole ordeal to be over.

The oak at Home Depot was reasonably priced. I was able to get everything I needed for around $300. And it had nice straight boards with good grain at a variety of lengths and widths, all 3/4″ thick. They also have maple and cherry (and a little walnut). But they’re more expensive with much less variety and quality of boards. So red oak it was.

My one complaint about the Home Depot boards is that they put one or two stickers on each one. They are paper not plastic, so they rip easily and you really have to work at it to get them off. At first I thought I could just sand them off. But that’s much harder than it seems. So I recommend you get into a Zen place and peel off the stickers when you first buy the boards. The glue left behind is easy to sand off and you’ll be able to pick either side of all boards.

In the plans below I show all the wood and sizes needed. You’ll have to cut these out of various lengths of boards to get what you need. I’ll leave it up to you how you do that. Home Depot has lots of 7′ boards in 3 1/2″ and 5 1/2″ widths. But they have lots of other lengths and widths. If you have a way to transport it, you might be more efficient with a 10 or 12 foot piece of lumber. Just think about what combination of pieces you can get out of each board to minimize waste (both of wood and money).

Clamps

In the process of doing a few woodworking projects I’ve found that you can never have enough clamps. Seems like every time I start a new project I buy 2 or 3 new clamps. My favorites for long edge clamping are pipe clamps. pipe clampThey are pretty cheap and you buy a piece of metal pipe to make them any length you want. I have 4: 2 using half inch pipe and 2 with 3/4 inch, all 48″ long. They can really apply a lot of pressure and are long enough to clamp the legs to the cross members of the desk.

I also have several C clamps, some bar clamps in 1, 2 and 3 foot lengths, a some smaller (and less useful) spring clamps. Make sure you have enough of each.

Building the Desk

The Top

To make the top, you need to glue together enough boards to get the 24″ width needed, using biscuits for strength. I used all 5 1/2″ stock, 5 board wide, and cut off the extra 3 1/2″. I recommend starting with the top. You want to select the nicest boards, laying them side by side to see which have the most consistent color and grain and flipping them to see which side is nicer. I found that if you do this and use a light colored varnish you can’t really see any seams at all.

The Legs

The legs are 2″ x 2″ and 29″ long. Since all the wood from Home Depot is 3/4″ thick, I glued 3 boards together to get that thickness. Lots of glueing surface, so no need for any joinery Here. I actually glued together 3 – 5 1/2″ x 30″ boards and then cut them lengthwise into 2 – 2″ x 2 1/4″ legs. My plan was to put a veneer on the face of the legs with with the seams. You make the veneer by cutting a 1/8″ wide strip from a 3 1/2″ wide board and glueing a piece of that to each side. But I found I could put the veneer on one of the faces since the other would be facing to the back. That made it easier to cut the legs to exactly 2″ x 2″, without having to worry about the exact thickness of the veneer.

End Pieces

Once the legs are completed it’s time to attach them to the end pieces and the slats. I recommend you sand everything before starting assembly just to make it easier to get into all the corners. The easiest way to proceed is to drill the dowel holes in the top end piece and legs. In fact you’ll want to drill the holes for the cross pieces. That will be 4 holes for the back piece on both sides and two holes for each of the two front pieces on both sides. This will make it a lot easier to do the next step. There’s one last bit of work that needs to be done before assembling the front, so you might as well do it now. Cut the vertical biscuit hole for the right and left front pieces in the front legs. This will be between the two front cross pieces.

Now mark the centers of the attachment points for the slats and cut the biscuit holes for them, as well as for the bottom end piece. Now glue and clamp the slats to the top and bottom pieces and before they are dry, glue and clamp the end pieces to the legs. Assuming all the cuts are straight, everything should be perfectly square. But biscuits have a bit of slop, so make sure the slats like up with the center line marks you made. Set these aside and let them dry, then you’re ready for the next step.

Cross Pieces

Now it’s time to connect the cross pieces, making a structure that will stand on its own. Since you’ve already made the dowel holes, the only cutting here is the biscuit holes for the bottom cross brace in back. Once that’s done you can glue and clamp the two front top cross pieces, the back top cross piece and the back bottom cross piece. Let everything dry and you’re ready for the next step.

Front and Drawer Runners

These pieces fit together and attach to the desk using biscuits. After cutting the biscuit holes, glue the runners to the back then glue in the front right and left pieces. You might want to put a bit of glue on the top and bottom cross pieces, too. But everything should stay pretty solid from here.

The Drawer

The drawer uses locking rabbet joints in the front so it withstands repeated opening, and a standard rabbet in the back. Follow the directions in the link above to cut the slot in the front piece and the locking slots in the front of the side pieces. Then cut a standard 1/2″ x 1/4″ rabbet in the back of the side pieces. Once that’s done, setup the table saw to cut a slot lengthwise 1/4″ from the bottom of all sides and 1/4″ deep. This is for the bottom to slide into. Next glue and nail the sides to the front, then slide in the bottom plywood, then glue and nail the back to the sides. You just made a drawer!

Now is the tricky part. Assuming you got the measurements right, you should have just enough space for the drawer sliders to fit between the drawer sides and the slide supports in the desk. I had to shim a bit, using a couple of washers, to get things to fit just right. Also be careful to make the amount of space above and below the drawer the same. Take your time you should be able to get a nice smooth drawer operation with no rubbing of wood on wood.

Drawer Facade

There is a facade on the front of the drawer to hide the slides. I made a front as a single 3 1/2″ x 36″ piece and then cut it in 3 pieces to get the two side pieces and the facade. That way, the grain matches nicely. But you’ll have to trim the facade so it fits with some clearance. I started by taking 1/16″ off of all 4 sides and continued trimming tiny bits until there was good clearance, but not a big gap.

Next I drilled 2 holes in the front drawer piece (the piece behind the facade). Now I held the facade against the drawer front until there was the same clearance on all 4 sides, marked the positions of the 2 screws, and drilled pilot holes. Don’t drill all the way through the facade. I use blue masking tape on the drill bit 1/2″ down just to make sure. Then screw the facade on with 1″ wood screws and you’re set. Test the fit and do any final adjustments to the slides.

As a last step, drill and install your drawer pull. I used a simple burnished metal knob because it fit the look and only needed one hold to install. But the choice is yours.

Attach the Top

188101You attach the top using screw, not glue, to allow for expansion. I used figure 8 fasteners. If you can find them use the ones specifically designer for desks. They fit wood screws nicely without the need for additional washers. To install them select your locations on the desk frame (I chose 6 locations: 2 each on the front and back and 1 on each end). Then use a Forstner bit of the appropriate size and drill deep enough so the fastener sits flush with the top of the frame. Now go around, drill pilot holes and screw the fasteners into the frame.

UnknownForstner bits are not cheap, but they are really useful. They let you drill holes with flat bottoms and are perfect for this sort of counter sinking work. They’re not cheap, but you can buy them individually. So just get one of the right size for this job and build up your collection later. I bought a couple and then asked for more sizes for Christmas 🙂

Next place the top on the frame and adjust it so it’s even on all 4 sides. Then mark the location of the holes in the fasteners, drill pilot holes and screw the top to the frame. You desk is now built!

Sand and Varnish

Hopefully you’ve been doing some preliminary sanding before putting everything together. If so, you just need to do a light final sanding and then wipe everything down with a damp cloth. Wait a few minutes for it to dry and then put on a coat of polyurethane varnish. A quart is plenty. Let that coat dry, lightly sand the entire desk, and give a pretty good sanding to the top. Then wipe off and apply a second coat. Two coats is good enough, but to get a really smooth top, you can sand and coat only the top once more.

I didn’t actually buy varnish from Amazon. I went down to the hardware store because they have Minwax stain samples. Red oak is a little light and doesn’t have that classic golden oak tone. So I got 4 samples and stained a scrap piece to see which one looked best. I ended up going with Golden Oak (bit surprise!).

Cut List

Library Table Exploded

  • Legs – 2″ x 2″ x 29″ (3 – 2″ x 3/4″ x 29″, cut to side with 1/8″ veneer on one side)
  • Top side – 3/4″ x 5 1/2″ x 17 1/2″
  • Bottom side – 3/4″ x 3 1/2″ x 17 1/2″
  • Side slats – 1/2″ x 3″ x 16″
  • Top – 24″ x 46″ x 3/4″ (3 – 5 1/2″ wide, 2 – 3 3/4″ wide)
  • Top back cross piece – 5 1/2″ x 36″
  • Bottom back cross piece – 3 1/2″ x 36″
  • Front cross pieces – 2 – 2″ x 36″
  • Front facing piece – 3 1/2″ x 36″, cut into 2 – 8″ side pieces and 1 – 20″ facade
  • Drawer slide supports – 2 – 3″ x 19 3/8″ x 3/4″
  • Drawer front – 2 3/4″ x 17″ x 1/2″
  • Drawer back – 2 3/4″ x 17″ x 1/2″
  • Drawer sides – 2 3/4″ x 19″ x 1/2″
  • Drawer bottom – 17″ x  18″ x 1/2″ birch (or any hardwood) plywood, one side appearance grade
  • Drawer slides
  • Drawer knob – to suit your taste

Enjoy Your New Desk!

Joinery for the Everyday Woodworker

08350-7Probably the most important skill for any woodworker is joinery. Attaching pieces of wood to each other is required on most any project and if it needs to withstand any kind of load you need a good strong joint. There have been lots of joinery techniques through the centuries, but modern techniques (together with modern glue) allow you to make stronger joints than every before.

Modern glues are really strong, but you need a lot of surface area for a strong joint, which is why techniques like mortise and tenon are so popular – lots of surface area on both sides of the tenon for the glue to hold onto. But I tried making one once, using a drill and chisel for the mortise and it ended up looking sloppy and was too loose. There are tools which make the process easier, but they are really expensive.

I’ve searched around and have found a couple of techniques which have worked well in a variety of projects. They’re very repeatable and while you have to invest in good tools to do them, they’re not that expensive and they use tools you’ll want to have around the shop anyway.

A word about glue

The purpose of most joinery techniques is to create enough surface area for glue to adhere and produce a strong joint. So choice of glue is very important. There are plenty of articles on the web comparing different kinds of glue. The one here was the most interesting to me. I found that a normal carpenter’s glue, like Titebond works well, cleans up easy and is reasonably priced. I’ve tried Titebond III, but it’s waterproof and I found that if it gets on any exposed surface you’ll see spots when you put on varnish. So I use it for outdoor projects, but I stick with Titebond I for everything else. I’ve heard good things about Gorilla Glue for woodworking, so I’ll probably try it someday. But for now Titebond is it for me.

Dowels

My first joint of choice is dowels. They require simple tools (a drill and doweling jig) and if you use enough dowels can produce a joint as strong as mortise and tenon. I use these when maximum strength is need, like in table legs.

The trick with dowel joints is repeatability. You need the exact same spacing in both pieces or things won’t fit right. There are many jigs on the market that let you get the right spacing, at various price points. They all have fixed spacing for the dowels, but many only let you have a fixed distance from the edge of the board. To change that you need to use shims and that leads to inaccuracy. img_9273So I went with the Jessem Doweling Jig because it variable edge spacing with detents so you can get nice repeatable results. It also has hardened steel guide bushings so it will (hopefully) last through lots of doweling jobs.

If you buy this or any jig, make sure to get a really big bag of dowels. You go through them pretty fast. The only downside to this jig is the fact that it doesn’t have a built-in clamp. So you have to use a wood clamp, which makes it a bit hard to line up just right. If I could do it over, I might have opted for the DowelMax, but it was much more expensive and harder to setup precisely.

The Jessem uses 3/8″ dowels, which are typically 2″ long. This is good for most jobs. They also make a 1/4″ add-on for smaller jobs and 1/2″ when you really need a lot of holding power. They have videos and instructions on how to do many kinds of joints. It’s been working great for me.

Biscuits

If you’ve ever watched The New Yankee Workshop you know that Norm Abrams loves biscuit joinery. This is a technique which requires a single purpose tool. But I find that I do biscuit joints so much that it’s well worth the investment. I found a great comparison of popular models. From this I settled on the Porter Cable 557. 81eJB3bQdzL._SL1500_It has a great fence which gives very repeatable results, has plenty of adjustments for biscuit size and positioning, and is reasonably priced on Amazon.  Plus it comes with a nice case to keep it in. Make sure you buy plenty of biscuits, you’ll go through them fast. I went with a big bag that has a variety of sizes.

I use biscuits both to join wood edge-to-edge for table tops and for edge joints that don’t need as much structural strength as, say, a table leg. For instance, I used them for the slats in my oak Library Table.

Drawer Joints

Building drawers has always scared me. The precision of dovetail joints and the complexity of the jigs required to get repeatable results feels beyond my abilities. But the great thing about the internet is that it can show you all kinds if alternatives with videos to make it look like something you could actually do. My favorite so far is the Locking Rabbet Joint. Screen Shot 2014-07-13 at 11.21.12 AMI used this on the drawer for my library table and it worked great. The drawer is strong and straight and looks pretty cool when you have the drawer open.

I have a router table, but for the joints I did I used a table saw like it shows in the article. You just need to make sure the saw height is right. I have this nice little tool for this which is basically a piece of aluminum with various marked height steps.spin_prod_206235701 It is inexpensive and even comes with a companion router height gauge. You can get it at Amazon.

Once you have the height you want (I always make a pass on some scrap wood to make sure), you need to make the appropriate slits in the wood as shown in the article. When attaching, I used small brads in addition to glue for extra strength. But some people would say that’s overkill. With this type of joint, you need a façade on the front of the drawer, an extra piece of wood to cover up the joints. But for my drawer, I needed a façade that was a bit wider than the drawer to hide the slider anyway.

The Rest

There are many other types of joints and I’m sure I’ll try some of them in the future. But for now, these joints make me feel like I could build anything!