\: vim:syntax=tex
\: $Id: caspar.azm,v 1.55 2009-11-14 07:21:19 joostvb Exp $
\: this is a manpage in zoem format.  see http://micans.org/zoem/ and man_zmm(7)

\def{"man::synstyle"}{long}
\def{"man::defstyle"}{long}
\import{man.zmm}

\import{./include.zmm}

\set{"man::name"}{caspar}
\set{"man::html-title"}{caspar}
\set{"man::section"}{7}

\"man::preamble"
\${html}{\"man::maketoc"}

\def{"docbook.mk"}{\it{docbook.mk}}
\def{"pod.mk"}{\it{pod.mk}}
\def{"caspar.mk"}{\it{caspar.mk}}
\def{caspar}{\bf{caspar}}
\def{"install.mk"}{\it{install.mk}}

\sec{name}{NAME}
  \par
  caspar - Makefile snippets for common tasks

\sec{synopsis}{SYNOPSIS}

In a Makefile, write either

\verbatim{
  include caspar/mk/caspar.mk
}

or

\verbatim{
  include caspar/mk/docbook.mk
}

or

\verbatim{
  include caspar/mk/pod.mk
}
.

\sec{description}{DESCRIPTION}

Caspar offers Makefile snippets for common tasks, like installing
(configuration) files, or typesetting LaTeX, DocBook XML and DocBook SGML
documents.
  \par
The typesetting functionality is delivered by \"docbook.mk" and \"pod.mk".
This is documented in \sibref{caspar-typesetting}{caspar-typesetting(7)}.
  \par
The installing-stuff functionality is delivered by \"caspar.mk".
(That's what the rest of the manual will talk about.)  It enables one to run
'make install' from within a tree which typically holds configuration files,
managed using CVS (or Subversion or any other version control system, for that
matter).
  \par
It is useful in cases like this: all configuration files of some host are under
version control, and, after commiting a change to CVS, you want to be able to
easily install the new configuration file on the host.
  \par
With \caspar, all you have to do is specify the hostname in one place, and
specify the name of the target directory in each CVS directory.
  \par

\sec{usage}{USAGE}

Within a CVS tree, create a file \it{include/install.mk}, with contents like
e.g.

\verbatim{
 csp_UHOST = root@some.host.somewhere
 include caspar/mk/caspar.mk
}

.  Within each other directory of the CVS tree which holds files, create a
Makefile, which looks like e.g.

\verbatim{
 csp_DIR = /some/dir/ectory/
 include ../../include/install.mk
}
.

If you'd like to use the \it{install-recursive} target too, in directories
which hold subdirectories (but not files), you'll have to create a Makefile
which looks something like

\verbatim{
 include ../../include/install.mk
}

.  From within the CVS tree, one can call:

\verbatim{
  make <filename>-install
  make install
  make load
  make
  make install-recursive
}

Calling \tt{make install} (or \tt{make}) now will scp all files in the current
directory to the remote location.  The \it{install-recursive} target descends
down the tree, and calls \tt{make install} in each subdirectory.

\par

Of course, you'll have to be able to ssh directly as root to the remote host to
get this working (if you'd like to use \tt{csp_UHOST =
root@some.host.somewhere}).  If you don't like this, and would like to have a
\tt{PermitRootLogin no} in your \tt{/etc/ssh/sshd_config}, you can use
csp_sucp(1).  See below.

\sec{variables}{VARIABLES}

The variables one can set in a calling Makefile are:

\begin{itemize}{
  {contiguous}{1}
  {w1}{7}
}
  \item{\tt{csp_UHOST}}    user@host, reachable using \tt{$(csp_PUSH)} (which
                           is scp by default)
  \item{\tt{csp_UHOSTS}}   space separated list of user@host items, reachable
                           using \tt{$(csp_PUSH)}
  \item{\tt{csp_DIR}}      directory on host, reachable using function
                           \tt{$(csp_PUSH)}
  \item{\tt{csp_PUSH}}     make function for pushing files to remote location.
                           Usually, this is a wrapper around a script or
                           program.  The function will be passed 4 arguments:
                           local \it{filename}, \it{[user@]host}, remote
                           \it{directory} and (a possibly empty) extra argument.
                           \it{[user@]host} will be set
                           to all elements of \tt{$(csp_UHOSTS)};
                           \it{directory} will be set to \tt{$(csp_DIR)}.  The
                           fourth argument will be set to \tt{$(csp_XARG)}.
                           Currently, \tt{$(csp_scp_FUNC)}, \tt{$(csp_cp_FUNC)}
                           and \tt{$(csp_sucp_FUNC)} are supported as push
                           plugins.  If \tt{csp_PUSH} is unset, the default
                           \tt{$(csp_scp_FUNC)} is used.
  \item{\tt{csp_LOAD}}     targets which should depend on the `load' target.
  \item{\tt{csp_BUILD}}    targets which should depend on the `build' target.
  \item{\tt{csp_CP}}       cp binary, just "cp" by default
  \item{\tt{csp_SCP}}      scp binary, just "scp" by default
  \item{\tt{csp_SUCP}}     script wrapping sudo in ssh, "csp_sucp" by default
  \item{\tt{csp_EXTRAFILES}} extra files which should be installed.
    Can be used to include files starting with a dot.
  \item{\tt{csp_TABOOFILES}} files which should never be installed.  Set to
    \tt{Makefile .%.swp %~ #%# pod2htmd.tmp pod2htmi.tmp} by default.
  \item{\tt{csp_TABOOFILES_ADD}} extra files which should never be installed;
    added to list in \tt{csp_TABOOFILES}.
  \item{\tt{csp_TABOOFILES_SKIP}} files which should be installed, even if
    in initial \tt{csp_TABOOFILES} list.  Removed from \tt{csp_TABOOFILES}
    list.
  \item{\tt{csp_TABOODIRS}}  directories to exclude in install-recursive target.
    set to \tt{CVS .svn} by default.
  \item{\tt{csp_TABOODIRS_ADD}, \tt{csp_TABOODIRS_SKIP}}  see
    \tt{csp_TABOOFILES} equivalents.
  \item{\tt{csp_UHOSTS_SUBSET}} override csp_UHOSTS: don't push to csp_UHOSTS, but
    to the intersection of this space separated list of user@host items and
    csp_UHOSTS.
\end{itemize}

\par

The following variables might get phased out or removed soonish:

\begin{itemize}{
  {contiguous}{1}
  {w1}{7}
}
  \item{\tt{csp_CPFLAGS}}  extra arguments to pass to cp invocation, none by
    default
  \item{\tt{csp_SCPFLAGS}} extra arguments to pass to scp invocation, e.g.
    '\tt{-i .ssh/id_rsa-root}'
  \item{\tt{csp_CPDIR}}    directory to which we can cp(1), instead of scp
    (CoPy DIRectory)
  \item{\tt{csp_CPDIRS}}   space separated list of directories to which we can
    copy (CoPy DIRectorieS)
\end{itemize}

\sec{examples}{EXAMPLES}

Some examples:

\cpar{Using csp_UHOST}

This is the simplest way to use caspar.  \it{Makefile} is

\verbatim{
  csp_UHOST = root@some.host.somewhere
  csp_DIR = /etc/
  include caspar/mk/caspar.mk
}

Now, running "make" will scp all files in the current directory to
\tt{root@some.host.somewhere:/etc/}.

\cpar{More hosts, not scp but sudo via ssh}

\it{Makefile} is

\verbatim{
  csp_UHOSTS = root@some.host.somewhere root@some.other.host
  csp_PUSH = $(csp_sucp_FUNC)
  csp_DIR = /etc/
  include caspar/mk/caspar.mk
}

Now, running "make" will use csp_sucp to install all files in the current
directory to both root@some.host.somewhere:/etc/ and
root@some.other.host:/etc/.  If a file named \tt{fstab} is present in the
current directory, running "make fstab-install" will just install that one
file.

If you need to sudo(1) to another user on the remote host, add something like

\verbatim{
 csp_XARG = postgres
}

.  (If such a username is not supplied, sudo (and csp_sucp) use the default:
root.)

\cpar{Using csp_CP and csp_LOAD}

\it{username/etc/Makefile} is

\verbatim{
  csp_UHOST = dummy
  csp_PUSH = $(csp_cp_FUNC)
  csp_DIR = $(HOME)/etc/
  csp_LOAD = crontab-load
  include ../include/install.mk

  crontab-load:
        crontab $(csp_DIR)/crontab
}

while \it{../include/install.mk} is just

\verbatim{
  include caspar/mk/caspar.mk
}
.

Setting \tt{csp_PUSH} to \tt{$(csp_cp_FUNC)} causes cp(1) to get executed
by "make install" (not scp(1)).

Setting \tt{csp_LOAD} causes "make load" to execute the crontab command.  Just
running "make" is OK too, since "make" calls both "make install" and "make
load".

\cpar{Using csp_DIR and csp_LOAD}

\it{etc/Makefile} is

\verbatim{
  csp_DIR = /etc/
  csp_LOAD = aliases-load
  include ../include/install.mk

  aliases-load:
      ssh $(csp_UHOST) "cd /etc; postalias aliases; postfix reload"
}

while \it{../include/install.mk} is

\verbatim{
  csp_UHOST = root@some.host.somewhere
  include caspar/mk/caspar.mk
}

If you don't like the explicit "ssh" in your Makefile, you might want to
introduce a variable for that.  E.g. add \tt{my_EXEC = ssh}
to \it{../include/install.mk}, and write

\verbatim{
  aliases-load:
      $(my_EXEC) $(csp_UHOST) "cd /etc; postalias aliases; postfix reload"
}

in \it{etc/Makefile}.  This will make it easier to change to another push
mechanism, once that would be needed.


\cpar{Using csp_BUILD: building files locally}

If you'd like to build some files locally from local sources, before installing
the just build files, do e.g.

\verbatim{
 csp_UHOST = root@some.host.somewhere
 csp_DIR = /etc/
 csp_TABOOFILES_ADD = sshd_config.m4
 csp_BUILD = my-build

 include caspar/mk/caspar.mk

 my-build: sshd_config

 sshd_config: sshd_config.m4
     m4 $< > $@
}

List all source files in csp_TABOOFILES_ADD: this way, they won't get installed
on the csp_UHOST.


\cpar{Using csp_sucp_FUNC and csp_LOAD}

If you'd like to use csp_sucp and want a `load' target, do something like:

\verbatim{
 csp_PUSH = $(csp_sucp_FUNC)
 csp_UHOST = foobar.example.com
 csp_DIR = /etc/uruk/
 csp_LOAD = rc-load

 include caspar/mk/caspar.mk

 rc-load:
     ssh $(csp_UHOST) "sudo invoke-rc.d uruk force-reload"
}


\cpar{Adding a "check" target}

If you want to do some syntax check on the remote host, before loading the just
installed configuration file (and have a "make check" thing), do

\verbatim{
  csp_UHOST = foobar.example.com
  csp_DIR = /etc/
  csp_LOAD = check my-load

  include caspar/mk/caspar.mk

  check:
      ssh $(csp_UHOST) do-check-stuff

  my-load:
      ssh $(csp_UHOST) do-load-stuff
}

This way, "make load" won't cause the file to load if the check fails (which is
probably what you want).  Running "make" will perform "install", "check" and
"load".


\cpar{Combining the csp_LOAD target with multiple hosts; building files remotely}

You'll have to loop over csp_UHOSTS to execute load-command.  Here's an example
doing some preprocessing on the remote hosts too.

\verbatim{
  csp_DIR = /etc/ssh/
  csp_UHOSTS = root@some.host.somewhere root@some.other.host
  csp_LOAD = sshd_config-load
  include caspar/mk/caspar.mk

  sshd_config-load:
      for suh in $(csp_UHOSTS); do \\
        ssh $$suh "cd $(csp_DIR); \\
         m4 sshd_config.m4 > sshd_config && \\
         PATH=$$PATH:/sbin /etc/init.d/ssh restart"; \\
      done
}

(You might want to use a variable like \tt{my_EXEC} for ssh, see above.)

\cpar{Using the csp_TABOOFILES_{ADD,SKIP} variables; another way to perform remote builds}

Using the csp_TABOOFILES_{ADD,SKIP} variables is handy if you want to
\it{install} a \it{Makefile}, instead of using it:  Create \it{Makefile} just
as you'd like to have it installed on the remote location.  Now, create
\it{GNUmakefile} as e.g.

\verbatim{
  csp_TABOOFILES_SKIP = Makefile
  csp_TABOOFILES_ADD = GNUmakefile

  csp_DIR = /etc/foobar/
  csp_UHOST = root@some.host.somewhere
  include caspar/mk/caspar.mk

  load:
        ssh $(csp_UHOST) "make -C $(csp_DIR)"
}

(You might want to use a variable like \tt{my_EXEC} for ssh, see above.)

Now, \tt{make install} and \tt{make load} will do the right thing.

\cpar{Using the csp_EXTRAFILES variable}

Using the csp_EXTRAFILES variable is handy if you want to install files
with a leading dot.   E.g.:

\verbatim{
  csp_EXTRAFILES = .bashrc
  csp_UHOST = root@some.host.somewhere
  csp_DIR =
  include caspar/mk/caspar.mk
}

\cpar{Overriding csp_UHOSTS}

Supply e.g.

\verbatim{
 csp_UHOSTS = root@localhost root@some.host.somewhere
}

in \it{install.mk}, to install on multiple hosts.  Run

\verbatim{
 make filename-install csp_UHOSTS=joe@otherhost
}

to install filename as \tt{joe@otherhost}, instead of the default as given in
\it{install.mk}.  If you want to enable passing csp_UHOSTS as a shell
environment variable, you'll have to use conditional assignment in your
Makefile:

\verbatim{
 csp_UHOSTS ?= root@localhost root@some.host.somewhere
}

This allows it to run

\verbatim{
 % export csp_UHOSTS=foo@bar
 % make filename-install
}

to install on \tt{foo@bar}.


\cpar{Overriding csp_UHOSTS in a smart way: csp_UHOSTS_SUBSET.  Using multiple
groups of hosts.  Recursive make made easy.}

If you have lots of subdirectories holding information for lots of groups of
hosts, while this run you just want to install for a small group (or 1) hosts,
csp_UHOSTS_SUBSET is useful.

Suppose your casparized tree looks like

\verbatim{
 Makefile

 apache/include/install.mk
 apache/etc/apache2/Makefile
 apache/etc/apache2/envvars

 php/include/install.mk
 php/etc/php4/apache/Makefile
 php/etc/php4/apache/php.ini

 grub/include/install.mk
 grub/boot/grub/Makefile
 grub/boot/grub/menu.lst
 logrotate/include/install.mk
 logrotate/etc/Makefile
 logrotate/etc/logrotate.conf

 nrpe/include/install.mk
 nrpe/debian/etc/default/Makefile
 nrpe/debian/etc/default/nagios-nrpe-server
}

The file \it{apache/etc/apache2/Makefile} is:

\verbatim{
 csp_DIR = /etc/apache2/
 include ../../include/install.mk
}

(all other \it{Makefile}s are similar).

The file \it{apache/include/install.mk} is

\verbatim{
 csp_UHOSTS = root@a root@b
 include caspar/mk/caspar.mk
}

The file \it{php/include/install.mk} is the same.  The files
\it{grub/include/install.mk} and \it{logrotate/include/install.mk} are

\verbatim{
 csp_UHOSTS = root@d root@e root@f root@g
 include caspar/mk/caspar.mk
}

The file \it{nrpe/include/install.mk} is

\verbatim{
 csp_UHOSTS = root@d root@e root@f root@n
 include caspar/mk/caspar.mk
}

The toplevel \it{Makefile} is

\verbatim{
 dirs = $(patsubst %/Makefile,%,$(shell find * -mindepth 1
  -name Makefile))

 all:
     for i in $(dirs); do $(MAKE) -$(MAKEFLAGS) -C $$i; done

 install
     for i in $(dirs); do $(MAKE) -$(MAKEFLAGS) -C $$i install; done

 load
     for i in $(dirs); do $(MAKE) -$(MAKEFLAGS) -C $$i load; done
}

(we don't feel like sticking a \it{Makefile} in all non-leaf nodes of our tree).

\par

Now, when running "\tt{csp_UHOSTS_SUBSET='root@e root@f root@m root@n' make}"
in the toplevel, caspar just takes the intersection of csp_UHOSTS_SUBSET and
csp_UHOSTS for each csp_UHOSTS list. So, caspar will not push anything for
\it{apache/} and \it{php/}. The files \it{grub/boot/grub/menu.lst} and
\it{logrotate/etc/logrotate.conf} will get pushed to \tt{root@e} and
\tt{root@f} only.  The file \it{nrpe/debian/etc/default/nagios-nrpe-server}
will get pushed to \tt{root@e}, \tt{root@f} and \tt{root@n}.

\par

This is often better than just overriding csp_UHOSTS on the commandline (or in
your shell's environment): if the intersection of the original csp_UHOSTS and
your new csp_UHOSTS is empty, chances are big you've just forgotten to clean
your environment.

\cpar{Using sudo locally for installing files}

If you'd like to install files like

\verbatim{
  sudo cp foo.rc /etc/foobar/
}

you could set up your \it{Makefile} as

\verbatim{
 csp_DIR = /etc/foobar/
 csp_UHOST = dummy
 csp_PUSH = sudo cp $(1) $(3)
 include caspar/mk/caspar.mk
}

This is like csp_sucp, but without the ssh wrapping: it works on localhost
only.

\cpar{Plugging your own install script in caspar}

If your script \tt{foobar} should be called as e.g.

\verbatim{
  foobar --file=fstab --user@host=joe@some.host \\
    --dir=/etc/ --debuglevel=3
}

then make sure your \it{Makefile} features something like

\verbatim{
  csp_foobar_FUNC = foobar --file=$(1) --user@host=$(2) \\
    --dir=$(3) --debuglevel=$(4)
  csp_PUSH = $(csp_foobar_FUNC)
  csp_XARG = 3
}

You can now use \tt{csp_UHOST} and \tt{csp_DIR} just as you're used to.


\cpar{More advanced tricks}

When you don't want to ssh to \tt{root@some.host.somewhere} directly, you could
do

\verbatim{
 sudo rsync -az /path/to/your/config_archive /etc
}

on some.host.somewhere (e.g. from cron).

\sec{files}{FILES}

\par
 \it{caspar/mk/caspar.mk}, \it{caspar/mk/docbook.mk}, \it{caspar/mk/pod.mk}

\sec{environment}{ENVIRONMENT}

For \"caspar.mk": \tt{csp_CP}, \tt{csp_LOAD},
\tt{csp_SCP}, \tt{csp_UHOST}, \tt{csp_PUSH}, ...

\sec{bugs}{BUGS}

Very likely, GNU Make is not the best tool for doing the stuff \"caspar.mk" is
doing.

See also TODO, distributed with the caspar package.  (And online at
\httpref{http://mdcc.cx/pub/caspar/caspar-latest/TODO}.)

\sec{trivia}{TRIVIA}

Caspar is named after Caspar the Friendly Ghost, since that's the title
of the Daniel Johnston song I was listening to when deciding to package
my homegrown scripts.

\sec{author}{AUTHOR}

\"man::author"

\sec{seealso}{SEE ALSO}

\sibref{caspar-typesetting}{caspar-typesetting(7)}
\sibref{csp_helper}{csp_helper(1)}
  \par
The caspar homepage is at \httpref{http://mdcc.cx/caspar/} .
  \par
The document
\aref{http://non-gnu.uvt.nl/pub/uvt-unix-doc/packaging/}{"Versiebeheer en
software-packages: Waarom en Hoe"} (in Dutch) describes some of the reasons why
people might want to use tools like caspar.
  \par
Jeroen Hoppenbrouwers blogs about the way he uses caspar, in
\aref{http://www.hoppie.nl/pub/node/79}{"Using Subversion and Caspar to
maintain a Linux host"}.
  \par
Lots of tools overlap (partly) with caspar in their functionality.  Here's a
list.  A big part of it was collected by Ray Miller
(\httpref{http://users.ox.ac.uk/~raym/}) of Oxford University, and published in
the article "Configuration Management with Subversion, YAML and Perl Template
Toolkit" in the SANE 2006 (\httpref{http://www.sane.nl/sane2006/}) conference
proceedings.

  \par

\: The `cvs-conf' package (\httpref{http://project.tuxfamily.org/cvs-conf},
\: \httpref{http://packages.debian.org/stable/utils/cvs-conf.html}) probably
\: offers about the same functionality as \"caspar.mk".  However, this package
\: seems largely unmaintained (between September 2002 and May 2004, at
\: least).

docbookmk, by Michael Wiedmann (\httpref{http://www.miwie.org/docbkmake/})
offers probably a superset of Caspar's \"docbook.mk" functionality.
  \par
latex-make by the LaTeX Utils project on
\httpref{http://gforge.inria.fr/projects/latex-utils/} seems to provide similar
functionality as \"docbook.mk" for LaTeX documents.
  \par
Latexmk by John Collins e.a. on
\httpref{http://www.phys.psu.edu/~collins/software/latexmk-jcc/} is another
implementation of this idea.
  \par
SUP, the Software Upgrade Protocol and it's implementation by Carnegie Mellon
University offers another way to distribute (configuration)files.  Beware
though: between Nov 1996 and June 2004, no new release has been published.  The
Debian (\httpref{ftp://ftp.debian.org/debian/pool/main/s/sup/}) and NetBSD
packages are likely still maintained, though.
  \par
cfengine (\httpref{http://www.cfengine.org/}), by Mark Burgess e.a., builds
expert systems to administrate and configure large computer networks: it
delivers a very big superset of caspar's installation mechanism.
  \par
PIKT (\httpref{http://www.pikt.org/}) is intended primarily for system
monitoring, but does do configuration management too.
  \par
LCFG (\httpref{http://www.lcfg.org/}) is another configuration management
system.
  \par
The Arusha Project (ARK, at \httpref{http://ark.sf.net/}) provides a framework
for collaborative system administration.
  \par
Puppet (\httpref{http://reductivelabs.com/projects/puppet}), also something
like a configuration management system.
  \par
Bcfg2 (\httpref{http://trac.mcs.anl.gov/projects/bcfg2/}) is yet another
configuration management system.
  \par
quattor (\httpref{http://quattor.web.cern.ch/}) is a system administration
toolkit for installation, configuration and management of Unix systems.
  \par
rb3 and friends, as written and used by Ray Miller e.a. at Oxford University,
(\httpref{http://users.ox.ac.uk/~raym/software/configuration-management/}).
  \par
The \aref{http://svk.elixus.org/}{svk} version control system is said to be
quite usable for handling configuration file management (without a separate
install mechanism like caspar).  See also
\aref{http://lists.debian.org/debian-devel/2005/02/thrd2.html#00495}{this
discussion on the Debian development list}.
  \par
On the \httpref{http://www.infrastructures.org/} website on automated (Unix)
system administration, you can find some thoughts on managing configuration
files using a version control system.

\"man::postamble"