OSX Lion as a Home Server Howto

In Home Server Bound I described my decisions in moving from a dedicated server to a home server, based on OSX Lion Server. Here I describe all the steps I took to make this a reality.

1. Get the Hardware

First off, get a Hackintosh. What? No! I actually read a post that suggested this. Do you really want that pain? No, you don’t. Just get a good recent Macintosh with at least a Core 2 Duo processor. A Core Duo or earlier won’t work because they won’t run Lion. And you really do want to run Lion. Even a recent Mac Mini would do. I’m using an iMac with a Core 2 Duo, 3GB of RAM and a 750GB disk. That is plenty for all but the most heavy server usage.

2. Get Lion

If you’re lucky you have Snow Leopard on your target server. In that case make sure it’s up to date and it will have the Mac App Store. Use that to purchase and install Lion and Lion Server. If you’ve already purchased Lion for another Mac, you can download it for your server for free. Otherwise it’s $30, and Lion Server will cost another $50. It’s worth it. Don’t ask for cheaper alternatives, just do it. While you’re at it download and install Lion Server Admin Tools 10.7, which is free, onto any Mac you want to use to remotely manage your server. Then use Software Update to make sure you have all the latest bits.

3. Get Your Server on the Internet

a. Get your server on your internal home network

First a word about routers. If you don’t have a router with a firewall, or if your firewall is turned off STOP RIGHT HERE. Are you kidding me? Have you seen the people out there trying to get all up in your digital grill? Go buy an Airport Extreme and set it up and don’t let me catch you out after dark again. A Time Capsule is an Airport Extreme with a backup disk included, so that works, too. Conversely, an Airport Extreme with a USB disk attached is the same as a Time Capsule. Backup is yet another issue, which we’ll discuss later.

In fact, let me go so far as to say, just get an Airport Extreme. It’s so much easier than any other router. Why do you want to punish yourself? Just get an Airport Extreme. The rest of this tutorial will assume you have an Airport Extreme equivalent.

Your server needs a hardwired IP address in your local address. My network is set up to do DHCP from 10.0.1.10 to 10.0.1.200. So I’ve given my server the address 10.0.1.202, which is outside this range. Just go into Network Settings on the server, select Using DHCP with manual address and enter the IP address 10.0.1.202.

b. Get a dynamic IP address

These days it’s really hard to get a static IP address. If you’re on DSL you might be able to add one, but it will cost you. And if you’re on cable, you’re probably out of luck unless you get a business account, which is really expensive. Fortunately, these days you can get by with a dynamic IP address, thanks to places like DynDNS and No-IP. They have free plans where they will give you a domain name (for DynDNS it ends in dyndns.org, like mysuperawesomewebsite.dyndns.org) and then route that to your dynamic IP address, whatever it happens to be.

To do this (for free) you need to install client software (DynDNS / No-IP) on some Mac in your home network. This Mac has to be on all the time so it can periodically communicate with the dynamic DNS service to update the IP address when it changes. Don’t install the client on your server yet, since you be messing with it. Put it on some other machine in your network.

Now you should be able to go into a terminal, type

    ping mysuperawesomewebsite.dyndns.org

and see your current IP address getting pinged.

c. Get your own domain name

mysuperawesomewebsite.dyndns.org might be fine for right now, but eventually you’ll want your own domain name. Go to name.com, get a user account and buy a domain name. For now, why not just get a .info domain, like mysuperawesomewebsite.info, which only costs $3.99 a year. Name.com is a great service that will handle all your DNS needs, which you’re going to need in this case. After you’ve bought your domain, click on it in your account panel and then click on DNS Record Management in the panel on the right. Then in Add DNS Record select CNAME, and in the right box (the one next to the box with ‘300’ in it) type your dynamic IP domain name (mysuperawesomewebsite.dyndns.org). It takes a few hours for these changes to be made. But when it’s done, you should be able to type

    ping mysuperawesomewebsite.info

and see your current IP address pinged just like before.

So now users can get to you from the outside. Now you need to let them in.

4. Let the Outside World In

Assuming you’ve done as I’ve said and have your machine at 10.0.1.202, then you need to open up a few ports. For now let’s open these:

  • 22 – Remote Login – SSH, so you can remotely login from a terminal
  • 25 – SMTP Mail, so clients can send mail through the server
  • 80 – Web Server
  • 993 – IMAP Mail with SSL, so clients can read their mail securely

Open Airport Utility, click on the picture of your router and click on Manual Setup. Then click on the Internet tab at the top and click on the NAT tab under that. Then click on Configure Port Mappings and click on the ‘+’ sign. Choose one of the services above. It should fill in the proper port numbers. You just need to fill in the Private IP Address. Do this for each service above. then click on Update so the changes are uploaded to your router. You should now be able to remotely login to your server.

On your server, in Preferences->Sharing, turn on Remote Login. While you’re at it, turn on Web Sharing, too. Now you should be able to go to http://mysuperawesomewebsite.info from anywhere on the internet and see the default Lion Server web page. You should also be able to go to a Terminal and type:

    ssh myusername@mysuperawesomewebsite.info

and login to your server.

5. Get Mail Going

TBD

6. Install MySQL

TBD

7. Install WordPress

TBD

8. What Else?

TBD

 

 

Home Server Bound

The marrin.com site (and friends) is currently hosted at ESecureData. We have a dedicated server box with an Atom processor, up in Vancouver. It’s a great little server and has been working for several years. But it’s not without its quirks, sometimes going down for no apparent reason. And about a year ago I had to get entirely new hardware. But the folks at ESecureData have always been great and were willing to give me that new server at my old, very attractive price.

But even at the great price I am getting, it’s still pretty expensive. But my needs are pretty specific and no VPS or other hosting solution has ever come close to giving me the capabilities of a dedicated box at the price I’m paying.

Long ago I tried hosting marrin.com on my own home server, but that experience was a dismal failure. I got hacked multiple times and getting (and keeping) everything working was extremely painful. But that was about 10 years ago and times change. I have more experience now, so I know what is needed to run a server. And there are way more services available, so things like backup MX and DNS can be farmed out at a reasonable cost.

On the Road Again

So I’ve decided to experiment again with a home server. Oh, and one other thing has changed. I work for Apple now and so I know way more about Mac OSX and that OS has come a long way in the realm of easy-to-use servers.

Here are my needs:

  1. Serve email for me and my family (wife, 2 kids, and a couple of other family members)
  2. Redirect mail for most of my family, so they can all have an @marrin.com address.
  3. Handle a couple of small mailing lists
  4. Really, really good spam filtering.
  5. Host git repositories, along with gitweb
  6. Host several websites (videomonkey.org, mermaidtoes.com, marrin.org and avr.marrin.org) using WordPress
  7. Generally be a repository for making random files available to me or other I choose

Spam, spam, spam, spam, spam

I get lots of spam. I’ve had my @marrin.com email address for 17 years and I’ve never been careful about “letting it get out”. I have friends who will carefully use an alternate email address whenever they register at an online site, and have changed their email address completely every few years, just stay ahead of the spammers. I refuse to do that and instead rely on really good spam software.

So good spam software is really important to me and has been the showstopper issue that has kept me from using any managed hosting service to date. Spam software is either too hard to configure, not nearly good enough at stopping spam, or gets way too many false positives (I’m looking at you, gmail).

But there is really great spam software out there call SpamAssassin. It’s very configurable, constantly updated with new rules to detect spam, and ties into all the blacklisting and other techniques for thwarting spam. I literally get several hundred spams per day and I only ever see about 10. That’s well below my pain threshold and allows me to not worry about how spammers get my email address. And I can configure SpamAssassin to automatically discard spam above a threshold at which I’m confident it’s really spam, and put spam that’s just slightly spammy into a separate folder. I can look there from time to time to see if there are any false positives. But false positives are so rare that I only look at that folder every couple of months and then only when there’s some reason to believe I might have missed an email (which is never the case). I’ve gotten about 3 false positives in the last 5 years. SpamAssassin is just about perfect.

Most hosting services don’t use SpamAssassin and those that do don’t give you nearly enough control, so I have my own server.

Bringing it Home

A server at home is almost everything a dedicated remote server is, except for three things:

  1. They’re “on the backbone” (theoretically have higher bandwidth)
  2. They are located in a temperature controlled, uninterruptable power environment.
  3. They have a dedicated (static) IP address.

My home isn’t air conditioned, but here in the Bay Area that’s rarely an issue, and I can buy a UPS for power outages, which is a rare occurance here. As far as bandwidth goes, my needs are not great. And measurements of my ESecureData server shows that it is really no faster than my home internet connection. I’m sure this is a combination of my excellent connection and the necessary bandwidth limiting ESecureData must do. Either way, I don’t see the disadvantage.

There was a day when a static IP address was essential for a server if you ever wanted it to be universally accessible. But these days dynamic IP services can be had for free, or for $30/year if you want good support and convenience. So that should not be an issue.

OSX Lion Server

So I embarked on setting up a server. I have a decent dual-core iMac available, so I’m using that for initial experiments. I’ve installed Lion as well as OSX Lion Server, which is a $50 add-on from the app store. I’ve also installed Server Admin Tools on my personal Mac, which is how you remote configure the server. This is all sooooo much easier to use than WebMin on Linux. But what do I have available?

Well, first I was pleasantly surprised that Lion Server uses SpamAssassin as it’s built-in spam software. It even has a GUI for configuration. It’s not quite powerful enough (bug posted) but I can break into the config file when needed. Postfix is the mail server used, along with dovecot for imap clients. I’m very familiar with dovecot, but I’ve always used SendMail, so Postfix was something new. The good news there is that, so far, it’s been easy to setup mail accounts with Postfix and to get them working with SSL and everything. I still have some experimentation to do in order to find out how to do mail aliasing and mailing lists, but all looks promising right now.

Git is well supported on OSX (we use it every day at work), as is ssh, scp and all the other usual suspects for server access, so all that should be simple.

What About WordPress

WordPress is the only sticking point. OSX has always come with MySQL, which is needed by WordPress. But Lion dropped that, apparently due to some licensing issues with Oracle. Installing MySQL is possible but this little snag got me thinking. What about trying to work without MySQL? Lion ships with Postgresql and some of the alternatives to WordPress can run on that. Even WordPress has the ability to use Postgresql is you work at it. So I went down that road.

Fortunately I only burned up a day on this. I read several scary posts about how Postgresql on WordPress is not really ready for prime time, so I didn’t even try going there. Then I looked at Plone, Joomla, and Drupal, 3 very popular alternatives to WordPress. I read many horror stories about how hard it is to work with and customize Joomla, so I looked no further in that direction. Then I installed and got Plone running. But compared to WordPress I found it confusing and lacking in good support. Drupal didn’t seem very friendly either (although I never did install it), so I went back to trying to install MySQL.

It turns out not to be that hard. You just need to glean info from a few places and go through some rigamarole to get the passwords and permissions right. That’s done now and I now have a working installation of MySQL. I even found Sequel Pro, a GUI tool for interacting with MySQL.

As an experiment, I made a tarball of the videomonkey.org site on marrin.com and put that on my home server. Then I used Sequel Pro to export the database for videomonkey.org on marrin.com, and import that into my home server. With some apache, SQL and htaccess magic (which I’ll explain later) I was able to get videomonkey.org fully up and running at http://marr.in.

To be continued…

Etherclock

I’ve been planning on building an ethernet connected clock for some time now. It started as a project for my daughter, Mikayla. But my wife went out and bought both her and my other daughter Leah little LED alarm clocks for $10 at Walgreen’s which took away my ability to make her a $50 clock 🙂

Not to worry. We always need more clocks. So I repurposed Etherclock for the family room. We had a Squeezebox serving the purpose of a really accurate clock in that room. But I felt bad to be putting it to such a meager use and my friend Jon wanted a second one, so I sold it to him and started on Etherclock.

The Hardware

First I chose the Tuxgraphics ethernet board. This is a nice, small board that has a mega168 and an ENC28J60 for ethernet. The ethernet chip runs at 25MHz and can output a clock signal which is 1/2 that, which drives the mega168 at 12.5MHz. That’s slower than it’s maximum rate of 16MHz, but it was plenty fast enough for this project. And it was an easy matter to get a precise 1sec time reference using the 16 bit timer/counter of the AVR. Plus it has a nice little prototyping area on one side, and an LED for some early stage debugging.

The Tuxgraphics ethernet board

I wanted the clock to have nice big digits, so I decided on 0.8″ red 7 segment LEDs (LSD8161). I wanted them to be bright enough and have nice even brightness, so I opted for a shift register with constant current outputs. The MAX6969 is a great part for this. It has 16 outputs, so I only needed 2 parts to drive 4 digits, and I didn’t have to worry about multiplexing. I just needed 4 pins: data, clock, enable and latch.

Testing the MAX6969 LED driver

I set the current on the MAX6969 to maximum and used the enable signal to adjust the brightness. I breadboarded one digit to test the serial interface and brightness control. It was convenient to use my STK500 board for the testing phase, using the same mega168 at the same frequency as the final project. I also used this setup to test the IS474 light detector, used to automatic brightness adjustment. The IS474 outputs an analog signal from 0 to 5v, which I brought into the ADC0 pin of the Mega168. The incoming light level determines the percentage of time the LEDs are on, so they’re brighter in a bright room, and get dim when the room light decreases.

The Software

I knew I wanted to try writing code for Etherclock in C++, partly because I have used it for many years at work and partly because I read so many AVR articles claiming it was a bad idea 🙂 So I started writing Marrinator, a C++ library to interface with all the internal and external hardware used in the project. For a while I even toyed with writing my own toolchain, but things have gotten better since then and I abandoned that project. I made up my own Makefile and was running everything from the command line for a while. But I finally settled on using Xcode for development. It is perfectly capable of using an external Makefile and toolchain for building and made it much easier to find and fix compile errors.

Just for fun I played around with the Arduino development system a bit. It’s not hard to get Arduino to work with non-standard boards. But I found that the libraries and development system didn’t really give me much from my roll-your-own approach, so I went back to Xcode.

Getting firmware to the board

When I started the project I was using an STK500 both for hardware prototyping an downloading the firmware to the chip. When it came time to move to the real hardware I had a problem. The STK500 is perfectly capable of programming external hardware, using the standard 6 pin ICSP connector. But the Tuxgraphics board has a 5 pin inline connector instead. So I built what I called The Abomination to interface the two.

The Abomination

Now, on the face of it, it’s not so bad. It’s just a 6 pin IDC to 5 pin inline adaptor, embedded in epoxy. And this worked fine with the STK500. But later on I bought a Dragon, thinking I would be able to use DebugWire for some hardware debugging. That hasn’t happened (yet), but I also started using the Dragon to program the Tuxgraphics board. Of course, that didn’t work. Just look at The Abomination. 6 pin IDC to 5 pin inline. Where’s the missing pin? Turns out it carries VCC, and the Tuxgraphics board has no connection for it. That was fine with the STK500, but the Dragon wants to see VCC to know what voltage it should be programming at. No VCC, no programming.

So I simply added a 6 pin IDC connector to the prototyping area of the Tuxgraphics board, brought in the 5 ICSP pins, added in VCC and was in business.

Tuxgraphics board with the newly added 6 pin ICSP connector. Also notice the connectors for the display board and the button

Note to board designers. If you don’t have a compact board (like the LilyPad or FIO) please add a standard ICSP connector 🙂

Display Board

I put the 2 MAX6969 chips, the IS474 light detector and the 7 segment LEDs on a separate display board, mounted at a right angle to the main board. This made for a nice compact design which didn’t need any additional mounting for the display.

The finished display board. The Magic Mouse is NOT part of the design 🙂

The parts were mounted on a piece of perfboard, which meant soldering. Lots of soldering.

The rat’s nest that is the back of the display board

I used to really like soldering. But display boards like this are an entirely different experience. It consists of around 50 wires, or 100 connections. Shockingly, it worked perfectly the first time I wired it up, which is clearly a tribute to steady hand and high quality tools – or the beer.

Finishing the hardware

The only other parts and connections were a button on top of the case and the power connection at the rear. I didn’t use a polarized connector for power, but I will always try to do that in the future. I haven’t fried anything (yet), but I worry every time I connect power.

The power jack with its scary unpolarized connector
The button mounted in the top. Always use a connector for something like this, so you can disassemble it easily.

The back panel has cutouts for power and the ethernet connector. For ethernet I drilled a round hole and then used the dremel and a file to square it the best I could. The cable fits fine, so I didn’t worry about the look too much.

The case is a Context Engineering split body aluminum box. This is basically 2 pieces of extruded aluminum that fit together, so you can cut them to any length. I cut it to the perfect size with a band saw, which was easy once I had the right blade! But you might notice in the picture above that the cut is a little ragged. Next time I’ll add a fence using a piece of wood and a couple of clamps. I did this for the plexiglass front and it came out perfect.

The perfectly cut plexiglass front

Once the hardware was done, I just had to write all the software, which I’ll discuss in a separate article.

Etherclock in its new home, next to the Apple TV