$Id: build.txt 14449 2012-03-13 07:47:09Z joostvb $
$URL: https://svn.ilk.uvt.nl/svn/trunk/doc/build-instructions-for-software-packages/build.txt $
Introduction
============
This documents describes how to create Debian packages, Fedora RPM packages and
Fink packages for Mac OSX from the software produced at ILK/TiCC, Tilburg
University. It describes procedures appicable for the TiMBL, Mbt, Dimbl and
Frog software, a.o. This document is mainly useful for ILK/TiCC software
developers.
This document is still in DRAFT status. A possibly up to date copy might be
available from http://ilk.uvt.nl/software-packages/build.txt .
Debian
======
You need a system running Debian unstable to create a package suitable for
uploading to the Debian archive. If you want to upload to the ILK apt repo,
Debian stable or testing is likely more suitable.
Set up build infrastructure
---------------------------
If not yet installed:
$ sudo apt-get install svn-buildpackage devscripts
Get the Debian packaging files:
$ svn co svn+ssh://svn.debian.org/svn/debian-science/packages/timbl
(use "svn co svn://svn.debian.org/svn/debian-science/packages/timbl" if you
don't have write access, use "timblserver", "mbt", "mbtserver", "dimbl", "ucto"
or "frog" instead of "timbl" for the other packages, of course.)
Fetch upstream
--------------
Fetch the current upstream tarball:
$ mkdir timbl/tarballs
$ cd timbl/trunk
See which tarballs are available:
$ uscan --destdir=../tarballs --download-current-version --verbose
(This uses information from debian/watch .) If you're trying to build a newer
upstream version than the latest one mentioned in debian/changelog, adjust this
file to reflect that:
$ dch -i --release-heuristic=changelog
(Fix the version number.) Then run again:
$ uscan --destdir=../tarballs --download-current-version --verbose
Build and commit
----------------
Check if the package properly builds:
$ cd svn/debian-science/package/timbl/trunk
$ DH_VERBOSE=1 svn-buildpackage -uc -us -rfakeroot --svn-ignore --svn-reuse --svn-move --svn-lintian --svn-builder=debuild
(If you don't have write-access to the svn repo, run
$ cd ../tarballs
$ tar zxf timbl_0.4.2.orig.tar.gz
$ cd timbl-0.4.2
$ svn export svn://svn.debian.org/svn/debian-science/packages/ucto/trunk/debian
$ DH_VERBOSE=1 debuild -uc -us -rfakeroot
instead.)
Check the result:
$ ls ..
Inspect all files found there. Viewing a .diff.gz is easy. If however you
don't use old-style .diff.gz, but your package generates .debian.tar.gz, you
might like to view it's contents by running e.g.:
tar -vOzxf timbl_10.8-1.debian.tar.gz 2>&1 | less
Inspect content of binary packages by running e.g.
$ debc ../mbt_3.2.7-2_amd64.changes | less
If lintian complains, run
$ lintian --info ../*.dsc | less
and
$ lintian --info ../*.deb | less
and fix ALL the problems and record the fixes.
Now, let's do some real work:
$ vi debian/control
Check your changes:
$ svn diff
Record your changes in debian/changelog (make sure $DEBEMAIL is set to your
email address as listed in debian/control!):
$ dch -i --release-heuristic=changelog
(Or use plain
$ vi debian/changelog
.) Commit your change to SVN, using your debian/changelog entry as commit message:
$ debcommit --diff; debcommit --confirm
Now build as indicated above, and check the quality.
Intermezzo: updating to new 3.0 dpkg source format
--------------------------------------------------
If your package lacks a debian/source directory, create it:
$ svn mkdir debian/source
$ echo '3.0 (quilt)' > debian/source/format
$ svn add debian/source/format
$ dch -i --release-heuristic=changelog 'updated package to use 3.0 source format'
$ debcommit --diff; debcommit --confirm
See also http://wiki.debian.org/Projects/DebSrc3.0 and dpkg-source(1).
Get ready for Release
---------------------
Once the package is suitable for release:
$ svn-buildpackage --svn-tag -rfakeroot --svn-reuse --svn-move --svn-builder=debuild
(This assumes you have gpg setup correctly, and your key is suitable for
signing the release. If not, run "svn-buildpackage --svn-tag -rfakeroot -uc -us"
instead.)
NB! Never modify files in the tag directory.
The svn-buildpackage run ends in something like:
dch -D UNRELEASED -i NOT RELEASED YET
which means you'll have a pending commit in your working directory. _Do_ commit it!
E.g.:
$ svn commit -m 'start next release'
Envoi trunk/debian/changelog
.
Upload to Debian
----------------
If you have upload rights: upload the package:
$ cd build-area
$ dupload --to anonymous-ftp-master *changes
Uploading to ILK apt archive and rebuilding the index file
----------------------------------------------------------
If you don't want to wait for propagation of the packages through Debian's
mirrors and Ubuntu, upload to the ILK apt archive:
Make sure the upload dir is empty:
$ ssh zeus
$ cd /var/www/apt.ticc.uvt.nl/upload
$ rm -rf *
Copy generated files *.{changes,deb,diff.gz,dsc,orig.tar.gz} (or e.g.
timbl_6.2.0.pre1*) to upload dir at zeus:/var/www/apt.ticc.uvt.nl/upload/.
Afterwards, on zeus run
$ cd /var/www/apt.ticc.uvt.nl
$ make dists/lenny/Release
PGP-sign the new Release file. You will need the secret key and add it to your
keyring. Also a password is needed ;)
$ cd /var/www/apt.ticc.uvt.nl/dists/lenny
$ gpg --no-random-seed-file -bao Release.gpg Release
.
Rebuilding from unchanged sources
---------------------------------
In order to link against wheezy libs:
joostvb@bruhat:~% gpg --armor --export-secret-keys 'Joost van Baal-Ilić' | ssh virbalas gpg --import
Build only source, don't sign:
joostvb@bruhat:~/sv...ence/packages/ucto/trunk% svn-buildpackage -S -uc -us --svn-tag -rfakeroot --svn-reuse --svn-move --svn-builder=debuild
joostvb@bruhat:~/sv...ence/packages/ucto/trunk% scp ../ucto_0.4.9-4.d* virbalas:build/
On testing/unstable host, build binary:
joostvb@virbalas:~/build% dget http://apt.ticc.uvt.nl/t/libfolia_0.5-2.dsc
of
sloot@zeus:~/a$ dget http://ftp.debian.org/debian/pool/main/m/mbt/mbt_3.2.7-3.dsc
joostvb@virbalas:~/build% ls
libfolia_0.5-2.debian.tar.gz libfolia_0.5-2.dsc libfolia_0.5.orig.tar.gz
joostvb@virbalas:~/build% dpkg-source -x libfolia_0.5-2.dsc
Installeer eventueel nodige build-dependencies:
root@virbalas:~# apt-get -V install cdbs pkg-config libicu-dev libxml2-dev
Maak n nieuwe versie zoals bv mbt (3.2.7-3lenny1) unstable; urgency=low;
daarvoor pas je debian/changelog aan:
sloot@zeus:~/a/mbt-3.2.7$ dch -lilk --release-heuristic=changelog 'Rebuild on squeeze'
(of, met t handje:
sloot@zeus:~/a/mbt-3.2.7$ vi ./debian/changelog
)
Daarna bouw je de .deb:
joostvb@virbalas:~/build/libfolia-0.5% debuild -sa -k0B86B067
of, als je niet wilt signen, en wel de upstream tarball in je upload wilt:
sloot@zeus:~/a/mbt-3.2.7$ debuild -sa -uc -us
Als je verder wilt bouwen, doe je b.v.
root@virbalas:~# dpkg -i ~joostvb/build/libfolia1-dev_0.5-2_amd64.deb ~joostvb/build/libfolia1_0.5-2_amd64.deb
en daarna:
joostvb@virbalas:~/build/ucto-0.4.9% debuild -sa -k0B86B067
joostvb@bruhat:~/sv...nce/packages/ucto/wheezy% scp virbalas:build/\*ucto\* .
joostvb@bruhat:~/sv...cience/packages/libfolia% scp virbalas:build/libfolia\* .
root@virbalas:~# apt-get remove libfolia1 libfolia1-dev cdbs pkg-config libicu-dev libxml2-dev
root@virbalas:~# apt-get autoremove
Rebuilding from unchanged sources: already tagged release from svn
------------------------------------------------------------------
joostvb@bruhat:~/sv...e/packages/libfolia/tags% ln -s ../tarballs
joostvb@bruhat:~/sv...ages/libfolia/tags/0.6-1% uscan --destdir=../tarballs --download-current-version --verbose
joostvb@bruhat:~/sv...ages/libfolia/tags/0.6-1% svn-buildpackage -rfakeroot --svn-reuse --svn-move --svn-builder=debuild
, and, if you want to force inclusion of upstream source, use -sa:
$ svn-buildpackage -sa -rfakeroot --svn-reuse --svn-move --svn-builder=debuild
, and, if you need to include more, use -v3.2.5-3 (changelog >> 3.2.5-3)
Signing after the fact
----------------------
$ echo DEBSIGN_KEYID=0B86B067 >> ~/.devscripts
$ debsign ../ticcutils_0.4-5_amd64.changes
Alternative: syncing ILK apt archive with Debian archive
--------------------------------------------------------
On zeus:
cd /var/www/apt.ticc.uvt.nl/upload
dget http://ftp.debian.nl/debian/pool/main/t/timblserver/timblserver_1.2-4.dsc
wget http://ftp.debian.nl/debian/pool/main/t/timblserver/timblserver_1.2-4_amd64.deb
Be sure to copy the relevant .changes file to zeus too. Joost likely has a
copy in bruhat:svn/svn.debian.org/debian-science/packages/timblserver/.
More information
----------------
Generic official Debian packaging documentation:
http://www.debian.org/doc/manuals/maint-guide/
http://www.debian.org/doc/manuals/developers-reference/
Official svn-buildpackage documentation:
http://svn-bp.alioth.debian.org/
Tips for specific ways of working with svn-buildpackage:
http://pkg-perl.alioth.debian.org/subversion.html
http://workaround.org/debian-subversion-pbuilder
https://trac.cyconet.org/debian/wiki/SvnBuildPackage
Inofficial tutorials:
Building Tutorial by Margarita Manterola e.a. at http://wiki.debian.org/BuildingTutorial
"Debian Packaging Tutorial" by Lucas Nussbaum at http://www.lucas-nussbaum.net/blog/?p=676
See also svn+ssh://svn.debian.org/svn/debian-science/packages/timbl/trunk/debian/README.devel
RPM
===
See also https://svn.ilk.uvt.nl/svn/trunk/sources/Timbl-pkg/rpm/log . (FIXME: merge that one)
Building {,src}rpm from spec and .tar.gz
----------------------------------------
joostvb@bruhat:~/svn.ilk.uvt.nl/Frog/trunk% vi frog.spec
joostvb@bruhat:~/svn.ilk.uvt.nl/Frog/trunk% svn commit -m 'new upstream' frog.spec
joostvb@bruhat:~/svn.ilk.uvt.nl/ucto/trunk% scp ucto.spec fedora:rpmbuild/SPECS/
joostvb@fedora.mdcc.cx:~/rpmbuild/SOURCES% egrep -i sourc\|vers ../SPECS/ucto.spec
joostvb@fedora.mdcc.cx:~/rpmbuild/SOURCES% wget http://software.ticc.uvt.nl/ucto-0.4.1.tar.gz
(FIXME: find way to automate download. tried this:
joostvb@fedora.mdcc.cx:~/rpmbuild/SOURCES% rpm -q --specfile ../SPECS/ucto.spec
ucto-0.4.1-1.x86_64
joostvb@fedora.mdcc.cx:~/rpmbuild/SOURCES% rpm -q --qf %{description} --specfile ../SPECS/ucto.spec
Ucto can tokenize UTF-8 encoded text files (i.e. separate words from
[...]
use to you.
%
joostvb@fedora.mdcc.cx:~/rpmbuild/SOURCES% rpm -q --qf %{source} --specfile ../SPECS/ucto.spec
(none)
rpm -q -i --specfile ../SPECS/ucto.spec
%
)
joostvb@centos-5.3.mdcc.cx:~/rpmbuild/SPECS% rpmbuild -ba timbl.spec
root@centos-5.3.mdcc.cx:~# rpm -i ~joostvb/rpmbuild/RPMS/i386/timbl-6.3.2-1.i386.rpm
root@fedora.mdcc.cx:~# rpm -U ~joostvb/rpmbuild/RPMS/x86_64/ucto-0.4.1-1.x86_64.rpm
$ rpm -q -l timbl
joostvb@bruhat:~/tmp% scp fedora:rpmbuild/SRPMS/ucto-0.4.1-1.src.rpm .
joostvb@bruhat:~/tmp% scp fedora:rpmbuild/RPMS/x86_64/ucto-0.4.1-1.x86_64.rpm .
joostvb@bruhat:~/tmp% scp ucto-0.4.1-1.* popper:
joostvb@popper:~% scp ucto-0.4.1-1.src.rpm zeus:/var/www/ilk/packages/fedora-core-14-yum-rpm/SRPMS/
joostvb@popper:~% scp ucto-0.4.1-1.x86_64.rpm zeus:/var/www/ilk/packages/fedora-core-14-yum-rpm/14/RPMS/x86_64/
joostvb@zeus.uvt.nl:/var...s/fedora-core-14-yum-rpm% zgrep timblserver repodata/primary.xml.gz | grep location
joostvb@zeus.uvt.nl:/var/www/ilk/packages/fedora-core-14-yum-rpm% createrepo --update `pwd`
25/25 * 14/RPMS/x86_64/mbtserver-0.1-1.fc14.x86_64.rpm
Saving Primary metadata
Saving file lists metadata
Saving other metadata
See also
--------
Date: Thu, 12 Nov 2009 16:22:56 +0100
From: Joost van Baal
To: Antal van den Bosch
Cc: Ko van der Sloot
Subject: timbl-6.2.0-1.fc11.x86_64.rpm (was: Re:
timbl_6.2.0-1_darwin-powerpc.deb)
Message-ID: <20091112152256.GM31021@bruhat.mdcc.cx>
Creating binary RPM from SRPM
-----------------------------
http://ilk.uvt.nl/packages/timbl-6.2.0-1.fc11.x86_64.rpm is er nu, voor Fedora
11 op x68-64.
Daarnaast is er timbl-6.2.0-1.src.rpm; dat zou, met rpmbuild, voor iedere Red
Hat-achtige een binary .rpm moeten kunnen opleveren. Voer daarvoor uit, als
een normale gebruiker:
mkdir -p ~/rpmbuild/{SOURCES,SPECS,SRPMS,RPMS}
cd ~/rpmbuild/RPMS
wget http://ilk.uvt.nl/packages/timbl-6.2.0-1.src.rpm
rpm -i timbl-6.2.0-1.src.rpm
cd ~/rpmbuild/SPECS
rpmbuild -ba timbl*spec
Constructing a yum repo
-----------------------
joostvb@fedora.mdcc.cx:~% cat /etc/fedora-release
Fedora release 14 (Laughlin)
joostvb@fedora.mdcc.cx:~% rpm -q createrepo
le paquetage createrepo n'est pas installé
root@fedora.mdcc.cx:~# yum install createrepo
================================================================================
Paquet Architecture
Version Dépôt Taille
================================================================================
Installation:
createrepo noarch 0.9.8-5.fc14 fedora 81 k
Installation pour dépendance:
deltarpm x86_64 3.6-0.2.20100708git.fc14 fedora 74 k
libxml2-python x86_64 2.7.7-2.fc14 fedora 316 k
python-deltarpm x86_64 3.6-0.2.20100708git.fc14 fedora 28 k
Résumé de la transaction
================================================================================
Install 4 Package(s)
Taille totale des téléchargement : 498 k
Installed size: 2.4 M
Est-ce correct [o/N] :
joostvb@fedora.mdcc.cx:~% mkdir repo && cd repo
joostvb@fedora.mdcc.cx:~/repo% createrepo `pwd`
joostvb@fedora.mdcc.cx:~/repo% find .
.
./repodata
./repodata/other.xml.gz
./repodata/primary.xml.gz
./repodata/repomd.xml
./repodata/filelists.xml.gz
root@fedora.mdcc.cx:~# vi /etc/yum.repos.d/ticc.repo
root@fedora.mdcc.cx:~# cat /etc/yum.repos.d/ticc.repo
# http://yum.baseurl.org/wiki/RepoCreate
[ticc]
name = TiCC Software
baseurl = file:///home/joostvb/repo
joostvb@fedora.mdcc.cx:~/repo% mkdir -p SRPMS 14/RPMS
joostvb@fedora.mdcc.cx:~% cp -a rpmbuild/RPMS/x86_64 repo/14/RPMS
joostvb@fedora.mdcc.cx:~% cp -a rpmbuild/SRPMS/* repo/SRPMS
joostvb@fedora.mdcc.cx:~/repo% createrepo --update `pwd`
joostvb@fedora.mdcc.cx:~/repo/repodata% zgrep location primary.xml.gz
joostvb@fedora.mdcc.cx:~% rpm -q mbtserver
le paquetage mbtserver n'est pas installé
root@fedora.mdcc.cx:~# yum --nogpgcheck install mbtserver
Configuration du processus d'installation
Résolution des dépendances
--> Lancement de la transaction de test
---> Paquet mbtserver.x86_64 0:0.1-1 marqué pour être installé
--> Résolution des dépendances terminée
Dépendances résolues
================================================================================
Paquet Architecture Version Dépôt Taille
================================================================================
Installation:
mbtserver x86_64 0.1-1 ticc 313 k
Résumé de la transaction
================================================================================
Install 1 Package(s)
Taille totale des téléchargement : 313 k
Installed size: 1.0 M
Est-ce correct [o/N] :
Téléchargement des paquets :
Lancement de rpm_check_debug
Lancement de la transaction de test
Transaction de test réussie
Lancement de la transaction
Installation : mbtserver-0.1-1.x86_64 1/1
Installé:
mbtserver.x86_64 0:0.1-1
Terminé !
joostvb@fedora.mdcc.cx:~% rpm -q mbtserver
mbtserver-0.1-1.x86_64
obsolete:
joostvb@bruhat:~% ssh fedora tar zc repo | ssh popper "cat > repo.tar.gz"
joostvb@popper:~% ssh zeus tar zvx < repo.tar.gz && rm repo.tar.gz
repo/SRPMS/mbt-3.2.4-1.src.rpm
repo/14/RPMS/x86_64/timbl-6.3.2-1.fc14.x86_64.rpm
repo/repodata/repomd.xml
joostvb@popper:~% ssh zeus tar zc /var/www/ilk/packages/repo > repo12.tar.gz
joostvb@bruhat:~% ssh popper cat repo12.tar.gz | ssh fedora tar zvx
joostvb@fedora.mdcc.cx:~/repo% createrepo --update `pwd`
joostvb@zeus.uvt.nl:/var...ar/www/ilk/packages/repo% rm -rf *
joostvb@bruhat:~% ssh fedora tar zc repo | ssh popper "cat > repo.tar.gz"
joostvb@popper:~% ssh zeus tar zvx < repo.tar.gz && rm repo.tar.gz
repo/
joostvb@zeus.uvt.nl:/var...ar/www/ilk/packages/repo% mv ~/repo/* .
joostvb@zeus.uvt.nl:/var/www/ilk/packages% ln -s repo fedora-core-14-yum-rpm
yum repos and gpg signing
-------------------------
root@fedora.mdcc.cx:~# yum install mbtserver
[...]
Le paquet mbtserver-0.1-1.fc14.x86_64.rpm n'est pas signé
yum(8)
--nogpgcheck
Run with GPG signature checking disabled.
Configuration Option: gpgcheck
yum.conf(5)
[repository] OPTIONS
The repository section(s) take the following form:
gpgcheck Either `1' or `0'. This tells yum whether or not it
should perform a GPG signature check on the packages gotten from
this repository.
repo_gpgcheck Either `1' or `0'. This tells yum whether or not
it should perform a GPG signature check on the repodata from
this repository.
gpgkey A URL pointing to the ASCII-armored GPG key file for the
repository. This option is used if yum needs a public key to
verify a package and the required key hasn't been imported into
the RPM database. If this option is set, yum will automatically
import the key from the specified URL. You will be prompted
before the key is installed unless the assumeyes option is set.
using our yum repo
------------------
root@fedora.mdcc.cx:~# cat /etc/yum.repos.d/ticc.repo
[ticc]
name = TiCC Software
baseurl = http://ilk.uvt.nl/packages/fedora-core-14-yum-rpm
root@fedora.mdcc.cx:~# rpm -q timblserver
le paquetage timblserver n'est pas installé
root@fedora.mdcc.cx:~# yum --nogpgcheck install timblserver
================================================================================
Paquet Architecture Version Dépôt Taille
================================================================================
Installation:
timblserver x86_64 1.1-1 ticc 395 k
Résumé de la transaction
================================================================================
Install 1 Package(s)
Taille totale des téléchargement : 395 k
Installed size: 1.4 M
Installé:
timblserver.x86_64 0:1.1-1
Terminé !
Fink
====
rebuilding a new timbl fink package
-----------------------------------
joostvb@bruhat:~/sv...lk.uvt.nl/Timbl-pkg/fink% vi timbl.info
( update version, sha1sum )
joostvb@bruhat:~/sv...lk.uvt.nl/Timbl-pkg/fink% scp timbl.info dhcp-23:/sw/fink/10.4/local/main/finkinfo/
check:
joostvb@dhcp-23.mdcc.cx:~% fink list timbl
joostvb@dhcp-23.mdcc.cx:~% fink validate /sw/fink/dists/local/main/finkinfo/timbl.info
build:
joostvb@dhcp-23.mdcc.cx:~% fink -m --build-as-nobody rebuild timbl
[...]
dpkg-deb: building package `timbl' in `/sw/fink/10.5/local/main/binary-darwin-powerpc/timbl_6.2.0-1_darwin-powerpc.deb'.
test:
joostvb@dhcp-23.mdcc.cx:~% dpkg-deb -c /sw/fink/dists/local/main/binary-darwin-powerpc/timbl_6.2.0-1_darwin-powerpc.deb | less
joostvb@dhcp-23.mdcc.cx:~% fink install timbl
[more tests]
upload:
joostvb@bruhat:/var/tmp% scp dhcp-23:/sw/fink/dists/local/main/binary-darwin-powerpc/timbl_6.2.0-1_darwin-powerpc.deb .
joostvb@bruhat:/var/tmp% scp timbl_6.2.0-1_darwin-powerpc.deb rhea.uvt.nl:/tmp/
joostvb@bruhat:/var/tmp% scp ~/svn/svn.ilk.uvt.nl/Timbl-pkg/fink/timbl.info rhea.uvt.nl:/tmp/timbl_6.2.1-1_darwin_Mac_OS_X.info
joostvb@rhea:~$ scp /tmp/timbl_6.2.0-1_darwin-powerpc.deb zeus:/var/www/ilk/packages/
See also
https://svn.ilk.uvt.nl/svn/trunk/sources/Timbl-pkg/fink/log and
https://svn.ilk.uvt.nl/svn/trunk/doc/build-instructions-for-software-packages/get-install.html