Life the universe and everything

Poor mans failover

with 18 comments

This ‘How-to’ blog is primarily intended for webmasters with database driven websites (but also covers effective but cheap IP failover for all webmasters) who need to achieve high availability without high expense and is based on my own experience.

So why am I writing this? Well … at the time of writing I am struggling to find any resources for cheap, high availability and fail-over techniques which cover the failover scenarios a web master might face. The articles I’ve found also tend to be highly technical. I hope I can offer you a method which offers effective failover and allows you to maintain high availability using simple, effective and cheap resources.

A word of warning! We all know that 5 nines of availability is the goal (99.999% uptime) but that kind of availability does not come cheap. This article is intended for website owners who can live with a few minutes of downtime now and again but would like to avoid a few hours.

Here is the summary of main points/requirements:

  1. You must control your own DNS
  2. You need multiple websites and/or multiple databases
  3. You pay for a DNS failover service

DNS nameservers

You must have control of your DNS nameservers in order to specify a DNS failover service. I have shared hosting which comes with its own name servers but I use a managed DNS service at ZoneEdit for my primary name server and list my hosts DNS as secondary name server. Experience has taught me never to keep all my eggs in one basket. I use Zone Edit because they offer the failover service.

[update]I have since swapped to www.dnsmadeeasy.com as they offer a 2-4 minute failure detection/recovery time[/update]

Multiple Websites

Here’s my scenario. I have shared hosting for my database driven (MySQL) website and I have a server in my office which has the same files. I use this server for development and testing and it makes sense to use it as my failover server for when my main website fails. I also have an installation of MySQL on this server. The office has broadband with a fixed IP address.

OK, so not everyone has the luxury of a development server … but you do have the option of very cheap hosting at a different hosting company!!! If you need high availability you need somewhere else to take over when your primary site goes down. You simply need to ensure that the failover website has a reasonably up to date version of the live site.

Databases

I backup my live MySQL database every day and I ‘restore’ the latest backup to my office server as often as I remember.

I learned to my cost that database downtime accounts for the majority of my sites total down-time. Here’s where having a secondary database gives you another added advantage. With most database connections you can write script (php, asp etc) which checks the availability of a database and if it fails you instantly and seamlessly connect to your backup database and the page still executes on the primary website. I use such a script as a global connection which is used on every page. The classic ASP version is:

Set objConn=Server.CreateObject (“ADODB.Connection”)
objConn.open connstring, “”, “”
If objConn.errors.count>0 Then
objConn.close
objConn.open connstring2, “”, “”
End If

If my live database goes down (which it does) my users are unaffected because my scripts instantly switch to the secondary database.

Failover Service

For at most around $11 per year (multi-credit purchase is much cheaper) the ZoneEdit failover service will retrieve a specified page from your website every 10-15 minutes. It uses simple regular expressions to check for valid html content. If it cannot retrieve the page it dynamically changes the DNS pointers for your website to the IP address which you specify in the failover service. In my case the office IP address but could simply be a second hosting company IP address.

What happens then?

New visitors to the site will mostly be directed to the backup site. Visitors who were on-line at the time will lose the connection. This is because their browser will continue to attempt to access the ip address it has cached for the domain. This is another slight compromise.

Conclusion

Given a secondary website and/or database on a different server we can use simple scripting to connect seamlessly to a secondary database should the primary fail. Using a DNS IP failover service can provide high availability (10 mins average detection time at failover) by redirecting users to a specified secondary ip address.

This is my guerilla method for cheap fail-over and it has proved very reliable. I hope it is of use.

Advertisements

Written by barnabyfry

March 22, 2007 at 5:24 pm

Posted in How to

18 Responses

Subscribe to comments with RSS.

  1. Good solution.. I’ve heard that some isp ignore dns TTL and just set there own TTL, effectively preventing this solution from working properly have come across this? ..have you

    Ross

    November 3, 2008 at 2:26 pm

  2. Hi Ross,

    That is true. I think there will always be circumstances beyond our control. From my own experience, I can confirm that when the failover kicks in the backup server starts to get several hits a minute which seems to suggest that most traffic is successfully redirected.

    If I am online when it fails I find that if I close the browser then re-open it I pick up the new IP address straight away. So I think this could be a smaller problem than it first appears.

    barnabyfry

    December 2, 2008 at 11:05 am

  3. Hi Barnaby,

    Someone actually did a study in 2007 on how long it takes to reach a full switchover from the viewpoint of resolving caches.

    Try these two links:

    http://edgedirector.com/app/fail.htm

    and

    http://blog.pyromod.com/2007/09/viability-of-dns-failover.html

    In general, the caches were observed to be quite well behaved in that they respected the ttl expiry, and did go out and get the changed address in teh expected time frame.

    HTH,

    Spenser

    spenser

    June 27, 2009 at 7:34 am

  4. I just started looking into this for some websites that I need to keep up. How do you like the DNSMadeEasy service? Any others that you would recommend?

    Thanks

    LGR

    January 22, 2010 at 4:48 am

  5. I have a slightly different solution.

    I want my clients’ sites to be up all the time, but I don’t want to mess around with DNS, its too much out of my control.

    I am writing a PHP script that could be placed on a “free webhost”, such as the free credit webhost with a godaddy domain name, that will test a website through a cURL function and when successful then proceed to forward the visitor to that site, or otherwise forward the visitor to second, third or more hosts, depending on which one responds.. simply going down the list until one is active. You could even have the page reload until a successful connection is made.

    Obviously, this will do nothing for when godaddy goes down, but it will allow me more control over what happens when my own servers go down.

    This first version will check each time to make sure the host is active before forwarding the visitor to the verified host.

    If anyone is interested, you may soon download it from my website. Search for “php host failover script”

    I’d be interested in hearing suggestions for features.

    Frank

    April 27, 2010 at 1:42 am

    • Does your method involves multiple domain names? like example.com redirects to example2.com or example3.com ?

      imbeware

      March 29, 2012 at 5:14 pm

  6. Great article very helpful am already looking at zone edit.

    I was with Rackspace cloud sites previously but it was too expensive to justify, I never had downtime with them though.

    Now have two cheap hosting accounts so something like this sounds like a good option.

    Dan

    September 5, 2011 at 10:47 pm

  7. FWIW – I launched a SaaS service to do this a couple of years ago. Its cheaper than anyone else and allows you to keep your DNS where it is (by providing DNS Failover via a CNAME record pointed a host record that we control). We also offer unlimited Free DNS hosting – the thing to watch for with other failover vendors is that they require you to migrate your DNS to them first – and then they bill you off DNS query volume… if you have short TTLs for faster failover that results in a much high DNS query volume (we can effectively provide failover at 5 second TTL levels if you have solid hosting). Anyone looking to save some money should check out at: http://www.DNSHAT.com

    Thanks,
    -Scott (Founder)

    DNS HAT

    January 26, 2012 at 2:25 am

  8. Hello there! I could have sworn I’ve been to this blog before but after reading through some of the post I realized it’s new to me.
    Nonetheless, I’m definitely glad I found it and I’ll be book-marking and checking back
    often!

  9. After checking out a few of the blog posts on your web site, I
    seriously appreciate your technique of blogging. I book-marked it to my
    bookmark site list and will be checking back in the near future.
    Please visit my web site as well and tell me what you think.

    groppy.com

    April 18, 2013 at 5:37 pm

  10. Hello, just wanted to mention, I enjoyed this post.

    It was inspiring. Keep on posting!

    ciaonet.altervista.org

    April 28, 2013 at 10:45 am

  11. Excellent post. I was checking continuously this blog and I am impressed!
    Very helpful info particularly the last part 🙂 I care for
    such information much. I was looking for this certain information for a long time.
    Thank you and best of luck.

    Liliana

    June 13, 2013 at 3:48 am

  12. Woah.. I’m really enjoying the template/theme of this blog.. It’s simple, yet effective. A lot of times it’s hard to get that “perfect balance” between user friendliness and appearance.. I must say you’ve done a fantastic job with this. In addition, the blog loads extremely quick for me on Safari. Superb Blog! North Platte Roofing Pro, 3765 E. Cavalry Hills Dr., North Platte, NE, 69101, US, 308-221-8734

    Alana McNair

    June 28, 2013 at 9:08 am

  13. Oh – and fairy dust to the top after the second coat later in the day to finish it off!
    Planning a small remodel trailer homeing may take
    extra planning, but the Velvet Underground were an inspiration’ – and he talks to him with a fan’s gratitude, promising to reopen on August 18 with a new face.

    bathroom warehouse

    August 4, 2013 at 3:34 am

  14. An interesting discussion is worth comment. I do think that you ought to
    publish more on this topic, it might not be a taboo subject but typically
    folks don’t talk about such subjects. To the next! Cheers!!

    Yolanda

    August 4, 2013 at 2:49 pm

  15. Factor these in when you you could make your decisions on lenders and loan types super real the
    gop’s decision to sponsor students loan bill that calls for
    the elimination in the prevention and public health fund
    can be a brilliant political move.

    super real

    January 25, 2014 at 2:21 pm

  16. whoah this weblog is wonderful i really like studying your posts.
    Stay up the good work! You know, lots of individuals are hunting round for this information, you
    can help them greatly.


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: