Archive for the ‘Linux’ Category

Linode Dynamic DNS Bash Script

Monday, May 11th, 2009

So Mark Walling was working on an ash script for his router running OpenWRT to update Linode's DNS Manager with his IP address. I liked the idea of a simple shell script to update without needing to install libraries for Perl or Python.  I took his script and tried to adapt it to get the DOMAINID and RESOURCEID using sed or awk.  Those utilities seem great for manipulating multiline files of text, but I wasn't getting anywhere trying to parse one line of JSON from wget.  So I used perl to extract the id numbers.  I believe OpenWRT has some sort of perl with limited functionality, so maybe this will work with that or at least be easily adapted.  This could also be easily modified to use curl instead of wget. I suspect someone out there will find this useful.

#!/bin/bash
#
# Script to update Linode's DNS Manager for a given name.
#

# Things you need to change.
APIKEY=$(cat ~/.linode-apikey)
LASTIP="/tmp/lastip"
DOMAIN="domain.com"
SOAEMAIL="hostmaster@domain.com"
STATUS="1"
RRTYPE="A"
RRNAME="home"
IFACE="eth0"

# Shouldn't need to change anything below here.

WGET="wget -qO - https://api.linode.com/api/"
NEWIP=$(ifconfig $IFACE | head -n2 | tail -n1 | cut -d: -f2 | cut -d' ' -f1)
test -e $LASTIP && OLDIP=$(cat $LASTIP) || OLDIP=""

if [ x"$OLDIP" = x"$NEWIP" ]; then
  logger "No IP address change detected. Keeping $NEWIP"
else
   DOMAINID=$($WGET --post-data "api_key=$APIKEY&action=domainList" | \
        perl -e 'if ( =~ /"DOMAIN":"'"$DOMAIN"'","DOMAINID":([0-9]+),/) { print $1; }')
   RESOURCEID=$($WGET --post-data "api_key=$APIKEY&action=domainResourceList&DomainID=$DOMAINID" | \
        perl -e 'if ( =~ /"RESOURCEID":([0-9]+),"DOMAINID":'"$DOMAINID"',"TYPE":"'"$RRTYPE"'","NAME":"'"$RRNAME"'"/) { print $1; }')
   $WGET --post-data "api_key=$APIKEY&action=domainResourceSave&ResourceID=$RESOURCEID&DomainID=$DOMAINID&Name=$RRNAME&Type=$RRTYPE&Target=$NEWIP"; echo
   $WGET --post-data "api_key=$APIKEY&action=domainSave&DomainID=$DOMAINID&Domain=$DOMAIN&Type=master&Status=$STATUS&SOA_Email=$SOAEMAIL"; echo
   echo $NEWIP > $LASTIP
   logger "Updated IP address to $NEWIP"
fi

RHEL Yum Dependency Failure

Friday, May 8th, 2009

It's been awhile since I've posted anything here. I have been quite busy lately and other things seem to take precedence. I have upgraded my Asterisk install and would like to write a post about some of the changes I've made on that front. Although the real reason I'm writing this post is because of a problem I ran into at work with yum under RHEL.

While doing the latest batch up updates, I ran into a dependency problem with kernel-headers and glibc-headers. Here is what I got when I tried yum update.

--> Running transaction check
---> Package kernel-headers.i386 0:2.6.18-128.1.10.el5 set to be updated
---> Package kernel-devel.i686 0:2.6.18-128.1.10.el5 set to be installed
--> Processing Dependency: kernel-headers for package: glibc-headers
--> Processing Dependency: kernel-headers >= 2.2.1 for package: glibc-headers
---> Package kernel.i686 0:2.6.18-128.1.10.el5 set to be installed
--> Finished Dependency Resolution
glibc-headers-2.5-34.i386 from installed has depsolving problems
  --> Missing Dependency: kernel-headers >= 2.2.1 is needed by package glibc-headers-2.5-34.i386 (installed)
glibc-headers-2.5-34.i386 from installed has depsolving problems
  --> Missing Dependency: kernel-headers is needed by package glibc-headers-2.5-34.i386 (installed)
--> Running transaction check
---> Package kernel.i686 0:2.6.18-128.el5 set to be erased
--> Processing Dependency: kernel-headers for package: glibc-headers
--> Processing Dependency: kernel-headers >= 2.2.1 for package: glibc-headers
---> Package kernel-devel.i686 0:2.6.18-128.el5 set to be erased
--> Finished Dependency Resolution
glibc-headers-2.5-34.i386 from installed has depsolving problems
  --> Missing Dependency: kernel-headers >= 2.2.1 is needed by package glibc-headers-2.5-34.i386 (installed)
glibc-headers-2.5-34.i386 from installed has depsolving problems
  --> Missing Dependency: kernel-headers is needed by package glibc-headers-2.5-34.i386 (installed)
Error: Missing Dependency: kernel-headers is needed by package glibc-headers-2.5-34.i386 (installed)
Error: Missing Dependency: kernel-headers >= 2.2.1 is needed by package glibc-headers-2.5-34.i386 (installed)

While googling parts of the error message, I came across a post on a Chinese website. The Google translator wasn't that helpful, but it seems that yum clean all will clear out all the local cached info and that fixes the problem.

Compiling Dahdi Modules on Linode

Thursday, January 1st, 2009

Happy New Years everyone! This holiday I decided to work on setting up Asterisk, which is an open source pbx system. I had some issues where I wasn't getting any audio and it became apparent that I needed to compile the dahdi_dummy kernel module for a timing source. Since my VPS account is hosted at Linode, where they run Xen, I'm able to build my own kernel modules and my own kernel images if I wanted.

I was using the 2.6.18.8 kernel and everything built fairly easily under that, but modprobe had a segmentation fault when I loaded the dahdi_dummy module. Linode recently made the 2.6.28 kernel available, so I decided to give that a try instead.  I prefer to use the kernels that Linode provides for the stability and support.  Linode makes the kernel source available from which the kernels are built.  I did run into a few glitches though.  The first was that the dahdi modules were looking for bounds.h, which did exist.  It seems that "make clean" removes some of the generated header files.  Running "make prepare" solves that quickly (thanks caker).  I then found that the scripts/genksyms/genksyms binary was compiled for 64bit Linux.  Since I'm using a 32bit distribution, that didn't work out so well.  That part of the kernel tree needs to get cleaned and rebuilt in order to compile the dahdi modules.  After messing around awhile, I ended up rebuilding the entire kernel just to get a few small files.  I decided it's be best to write a small script to only clean the sections I need and make this easier next time around.

The script I used is below.

#!/bin/bash
#
# Script to clean genksyms in the kernel source and prepare the source to be
# used when compiling modules for things like asterisk.  This was needed to
# remove binaries in the scripts tree because the Linode kernels are built on
# a 64bit server and my Linode is using a 32bit distribution.
#
# Copyright (c) 2009 Patrick Hennessy
#
# Permission is hereby granted, free of charge, to any person
# obtaining a copy of this software and associated documentation
# files (the "Software"), to deal in the Software without
# restriction, including without limitation the rights to use,
# copy, modify, merge, publish, distribute, sublicense, and/or sell
# copies of the Software, and to permit persons to whom the
# Software is furnished to do so, subject to the following
# conditions:
#
# The above copyright notice and this permission notice shall be
# included in all copies or substantial portions of the Software.
#
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
# EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES
# OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
# NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT
# HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY,
# WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
# FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
# OTHER DEALINGS IN THE SOFTWARE.
#

# Set this to where your kernel source is placed.  I usually have a symlink
# for both of the locations below.
#
# KSOURCE=/lib/modules/$(uname -r)/build
#
KSOURCE=/usr/src/linux

# Simple test for the Makefile.
#
if [ ! -e $KSOURCE/Makefile ]; then
        echo "Can't find $KSOURCE/Makefile.  Make sure $KSOURCE exists."
        exit
fi

# Patch the Makefile to clean the scripts tree when doing clean.
#
patch -d $KSOURCE << END
--- ../2.6.28-linode15.orig/Makefile    2008-12-29 15:36:08.000000000 -0500
+++ Makefile    2009-01-01 03:08:19.000000000 -0500
@@ -1174,7 +1174,7 @@
 #
 clean: rm-dirs  := \$(CLEAN_DIRS)
 clean: rm-files := \$(CLEAN_FILES)
-clean-dirs      := \$(addprefix _clean_,\$(srctree) \$(vmlinux-alldirs) Documentation)
+clean-dirs      := \$(addprefix _clean_,\$(srctree) \$(vmlinux-alldirs) Documentation scripts)

 PHONY += \$(clean-dirs) clean archclean
 \$(clean-dirs):
@@ -1194,7 +1194,7 @@
 #
 mrproper: rm-dirs  := \$(wildcard \$(MRPROPER_DIRS))
 mrproper: rm-files := \$(wildcard \$(MRPROPER_FILES))
-mrproper-dirs      := \$(addprefix _mrproper_,Documentation/DocBook scripts)
+mrproper-dirs      := \$(addprefix _mrproper_,Documentation/DocBook)

 PHONY += \$(mrproper-dirs) mrproper archmrproper
 \$(mrproper-dirs):
END

# Run make targets to prepare the tree.
#
make -f $KSOURCE/Makefile clean
make -f $KSOURCE/Makefile oldconfig
make -f $KSOURCE/Makefile scripts/genksyms/
make -f $KSOURCE/Makefile prepare

This should be useful for other Linode users building dahdi or other kernel modules under 32bit Linux distributions.

[2009-Jan-01 EDIT:  The reason I didn't do "make mrproper" or "make distclean" is that I needed the Module.symvers file, which is created during the kernel compile.  That file appears to be used to add symbols when building modules.  Even while using difference force options, I was getting "no symbol version for struct_module" errors.  Once that Module.symvers file was present, I was able to build and load the modules as expected.]

[2009-Jan-02 EDIT: You may be using a different version of gcc than what the Linode kernel was compiled with. In that case, you'll need to pass the "--force-vermagic" option to modprobe when loading the module.]

[2009-Jan-02 EDIT: Updated the script to run oldconfig after the clean stage.]

Using Linode DNS Manager for Google Apps

Monday, December 8th, 2008

I've hosted a few websites for other people and have used Google Apps to handle mail and calendar needs. This way I'm just using my Linode to only host the webpages. After manually creating the Google Apps DNS records a few times, I decided it'd be best to write a script using Linode's API to automate the process.

Since I haven't written a Perl script from scratch in awhile, I thought it would be good to use that. Michael Greb, who works at Linode, published a Perl module on CPAN called "WebService::Linode". Mike also posted some really helpful information regarding the installation of Perl modules under Ubuntu Linux. Instead of downloading, compiling, and installing the module the old fashioned manual way, you can use dh-make-perl. The dh-make-perl program will do all of that for you and create a Debian package file to install. This way if the CPAN module is ever upgraded or becomes part of the distro, it'll be handled automatically. I used the following commands to install the module (I also had to do the same thing for JSON because my installed version was too old).

$ dh-make-perl --cpan JSON --build
$ dh-make-perl --cpan WebService::Linode --build
$ sudo dpkg -i libjson-perl_2.12-1_all.deb libwebservice-linode-perl_0.02-1_all.deb

On the first run it will prompt for the Linode API key and store it in ~/.linode-api. Subsequent runs will simply read the key from that file. Below is an example of how it's usage.

$ googleapps-dns.pl -h
Usage:
    googleapps-dns.pl [ -d domainname ] [ -m ] [ -c ] [ -f ] [ -v ] [ -h ]

Options:
            -d domainname
                    Specify the domain name for adding the records.  This field is
                    required.
            -c      Add Google Chat's Jabber and XMPP records to route external
                    chat program to Google's services.
            -g      Add CNAMES that point calendar.domainname, docs.domainname,
                    mail.domainname, sites.domainname, and start.domainname to
                    ghs.google.com.
            -m      Add MX and SPF records for routing mail to Google Apps.
            -f      Force deletion of any conflicting records.
            -v      Enable verbose debugging messages.
            -h      Display help and options.

$ googleapps-dns.pl -d example.com -g -m -c
Would you like to delete any conflicting dns records? [Y/n] y
Please enter your Linode API Key: xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
$

I hope this is helpful for other people managing sites hosted on Google Apps. The code is below.

#!/usr/bin/perl
#
# googleapps-dns.pl
#
# Script to populate the Linode's DNS Manager for Google Apps.
#
# Copyright (c) 2008 Patrick Hennessy

#
# Permission is hereby granted, free of charge, to any person obtaining a copy
# of this software and associated documentation files (the "Software"), to deal
# in the Software without restriction, including without limitation the rights
# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
# copies of the Software, and to permit persons to whom the Software is
# furnished to do so, subject to the following conditions:
#
# The above copyright notice and this permission notice shall be included in
# all copies or substantial portions of the Software.
#
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
# THE SOFTWARE.
#

use strict;
use warnings;

use Getopt::Long;
use Pod::Usage;
use WebService::Linode::DNS;

# File to store apikey.  If undefined or blank, use ~/.linode-apikey.
#
my $apiKeyFile = '';

# Default resource record templates.
#
my %mailTemplate = ( resourceid => 0,  name => '', type => 'MX' );
my %xmppTemplate = ( resourceid => 0, name => '_xmpp-server', protocol => '_tcp', type => 'SRV', weight => 0, port => 5269 );
my %jabberTemplate = ( resourceid => 0, name => '_jabber', protocol => '_tcp', type => 'SRV', weight => 0, port => 5269 );
my %ghsTemplate = ( resourceid => 0, type => 'CNAME', target => 'ghs.google.com' );

# Google Apps DNS Records
#
my @googleMailRR = (
        { %mailTemplate, target => 'aspmx.l.google.com', priority => 10 },
        { %mailTemplate, target => 'alt1.aspmx.l.google.com', priority => 20 },
        { %mailTemplate, target => 'alt2.aspmx.l.google.com', priority => 20 },
        { %mailTemplate, target => 'aspmx2.googlemail.com', priority => 30 },
        { %mailTemplate, target => 'aspmx3.googlemail.com', priority => 30 },
        { %mailTemplate, target => 'aspmx4.googlemail.com', priority => 30 },
        { %mailTemplate, target => 'aspmx5.googlemail.com', priority => 30 },
        { %mailTemplate, type => 'TXT', target => 'v=spf1 a mx include:aspmx.googlemail.com -all' },
);

my @googleXMPPJabberRR = (
        { %xmppTemplate, target => 'xmpp-server.l.google.com', priority => 5 },
        { %xmppTemplate, target => 'xmpp-server1.l.google.com', priority => 20 },
        { %xmppTemplate, target => 'xmpp-server2.l.google.com', priority => 20 },
        { %xmppTemplate, target => 'xmpp-server3.l.google.com', priority => 20 },
        { %xmppTemplate, target => 'xmpp-server4.l.google.com', priority => 20 },
        { %jabberTemplate, target => 'xmpp-server.l.google.com', priority => 5 },
        { %jabberTemplate, target => 'xmpp-server1.l.google.com', priority => 20 },
        { %jabberTemplate, target => 'xmpp-server2.l.google.com', priority => 20 },
        { %jabberTemplate, target => 'xmpp-server3.l.google.com', priority => 20 },
        { %jabberTemplate, target => 'xmpp-server4.l.google.com', priority => 20 },
);

my @googleGHSRR = (
        { %ghsTemplate, name => 'calendar' },
        { %ghsTemplate, name => 'docs' },
        { %ghsTemplate, name => 'mail' },
        { %ghsTemplate, name => 'sites' },
        { %ghsTemplate, name => 'start' },
);

# =============================================================================
# Do not change anything below this line.
# =============================================================================
#
die "This script needs to be run interactively." if (! -t STDIN);

# Handle options.
#
my ($chat, $debug, $domainName, $force, $ghs, $help, $mail);
my $result = GetOptions(
        'd=s'           => \$domainName,
        'c'             => \$chat,
        'f'             => \$force,
        'g'             => \$ghs,
        'm'             => \$mail,
        'v'             => \$debug,
        'help|h|?'      => \$help
);

# Usage
#
pod2usage(1) if $help;
pod2usage(1) if ! $domainName;
pod2usage(1) if (! $chat) && (! $ghs) && (! $mail);

# Prompt to see if we should delete conflicting records or just add the new Google records.
#
if (! defined($force)) {
        $force = parseresp(prompt("Would you like to delete any conflicting dns records? [Y/n] "));
}

# Get API key and create api object.
#
my $apiKey = getApiKey($apiKeyFile);
my $api = new WebService::Linode::DNS( apikey => $apiKey );

# Get domain object.
#
my $domain = $api->domainGet( domain => $domainName ) ||
        die "ERROR: Domain not found, please create the initial records first.\n";

# XMPP+Jabber Records
#
processRRs($api, $domain, \@googleXMPPJabberRR, $force) if $chat;

# Mail Records
#
processRRs($api, $domain, \@googleMailRR, $force) if $mail;

# GHS Records
#
processRRs($api, $domain, \@googleGHSRR, $force) if $ghs;
# Save the domain.
#
$api->domainSave(%$domain) || die "Couldn't save $domainName\n";

# All done.
#
exit(0);

# Subroutines
#
sub processRRs {
        my $api = shift;
        my $domain = shift;
        my $resourceRecords = shift;
        my $force = shift;

        # Remove any existing records.
        #
        if ($force) {
                for my $nRecord (@$resourceRecords) {
                        for my $eRecord (domainGetRRs($api, $domain->{domainid}, $nRecord->{type})) {
                                if (($eRecord->{type} =~ m/mx/i) && ($eRecord->{name} eq "")) {
                                        deleteRR($api, $eRecord);
                                } elsif (($eRecord->{type} =~ m/txt/i) && ($eRecord->{target} =~ m/v=spf/i)) {
                                        deleteRR($api, $eRecord);
                                } elsif (($eRecord->{type} =~ m/srv/i) && ($eRecord->{name} =~ m/_xmpp-server/i)) {
                                        deleteRR($api, $eRecord);
                                } elsif (($eRecord->{type} =~ m/srv/i) && ($eRecord->{name} =~ m/_jabber/i)) {
                                        deleteRR($api, $eRecord);
                                } elsif (($eRecord->{type} =~ m/cname/i) && ($eRecord->{target} =~ m/ghs.google.com/i)) {
                                        deleteRR($api, $eRecord);
                                }
                        }
                }
        }

        # Create new records.
        #
        for my $record (@$resourceRecords) {
                debug("SAVING RR: TYPE=$record->{type} NAME=$record->{name} TARGET=$record->{target}");
                my $rr = $api->domainResourceSave( 'domainid' => $domain->{domainid}, %$record ) ||
                        die "Could not save TYPE=$record->{type} NAME=$record->{name} TARGET=$record->{target}\n";
        }
}

sub deleteRR {
        my $api = shift;
        my $record = shift;

        debug("DELETING RR: TYPE=$record->{type} NAME=$record->{name} TARGET=$record->{target}");
        my $result = $api->domainResourceDelete( resourceid => $record->{resourceid} ) ||
                warn "WARNING: Could not delete TYPE=$record->{type} NAME=$record->{name} TARGET=$record->{target}\n";

        return $result;
}

sub domainGetRRs {
        my $api = shift;
        my $domainId = shift;
        my $domainRRType = shift;
        my @domainRRs = ();

        my $rrs = $api->domainResourceList( domainid => $domainId ) ||
                die "ERROR: Could not retrieve domain resource list.\n";
        for my $rr (@$rrs) {
                push(@domainRRs, $rr) if ($rr->{type} =~ m/^$domainRRType$/i);
        }

        return @domainRRs;
}

sub getApiKey {
        my $apiKeyFile = shift;
        my $apiKey;

        if (!length($apiKeyFile)) {
                $apiKeyFile = (getpwnam(getlogin()))[7] . '/.linode-apikey';
        }

        if (-f $apiKeyFile) {
                chomp ($apiKey = `cat $apiKeyFile`);
        } else {
                $apiKey = prompt("Please enter your Linode API Key: ");
                system "echo '$apiKey' > $apiKeyFile";
        }

        return $apiKey;
}

sub parseresp {
        my $response = shift;
        my $val = 0;

        chomp($response);
        $val = 1 if ($response =~ m/^y/i );
        $val = 1 if ($response eq "");

        return $val;
}

sub prompt {
        my $promptstring = shift;

        print $promptstring;
        chomp (my $retstring = );

        return $retstring;
}

sub debug {
        my $msg = shift;

        print STDERR $msg, "\n" if $debug;
}

__END__

=head1 NAME

googleapps-dns.pl - Create Google Apps DNS Records in Linode DNS Manager

=head1 SYNOPSIS

googleapps-dns.pl [ -d domainname ] [ -m ] [ -c ] [ -f ] [ -v ] [ -h ] 

=head1 OPTIONS

        -d domainname
                Specify the domain name for adding the records.  This field is
                required.
        -c      Add Google Chat's Jabber and XMPP records to route external
                chat program to Google's services.
        -g      Add CNAMES that point calendar.domainname, docs.domainname,
                mail.domainname, sites.domainname, and start.domainname to
                ghs.google.com.
        -m      Add MX and SPF records for routing mail to Google Apps.
        -f      Force deletion of any conflicting records.
        -v      Enable verbose debugging messages.
        -h      Display help and options.

=head1 DESCRIPTION

B provides a wizard to populate DNS records in the Linode
DNS manager for Google Apps.  It requires that the domain already exist.  It
would create the standard DNS entries which can be pruned manually later.

=cut

Linode Rocks

Thursday, July 31st, 2008

So I've been using Linode for awhile to host my site.  Linode is a Linux based VPS provider and they do an excellent job.  They are constantly enhancing their management interface and increasing resources in their plans.  They've just added a referral program as well.  So if anyone is interested in trying out Linux or want their own virtual machine, please use this link and it'll use my referral code when signing up.