$Id: build.txt 14449 2012-03-13 07:47:09Z joostvb $ $URL: https://svn.ilk.uvt.nl/svn/trunk/doc/build-instructions-for-software-packages/build.txt $ Introduction ============ This documents describes how to create Debian packages, Fedora RPM packages and Fink packages for Mac OSX from the software produced at ILK/TiCC, Tilburg University. It describes procedures appicable for the TiMBL, Mbt, Dimbl and Frog software, a.o. This document is mainly useful for ILK/TiCC software developers. This document is still in DRAFT status. A possibly up to date copy might be available from http://ilk.uvt.nl/software-packages/build.txt . Debian ====== You need a system running Debian unstable to create a package suitable for uploading to the Debian archive. If you want to upload to the ILK apt repo, Debian stable or testing is likely more suitable. Set up build infrastructure --------------------------- If not yet installed: $ sudo apt-get install svn-buildpackage devscripts Get the Debian packaging files: $ svn co svn+ssh://svn.debian.org/svn/debian-science/packages/timbl (use "svn co svn://svn.debian.org/svn/debian-science/packages/timbl" if you don't have write access, use "timblserver", "mbt", "mbtserver", "dimbl", "ucto" or "frog" instead of "timbl" for the other packages, of course.) Fetch upstream -------------- Fetch the current upstream tarball: $ mkdir timbl/tarballs $ cd timbl/trunk See which tarballs are available: $ uscan --destdir=../tarballs --download-current-version --verbose (This uses information from debian/watch .) If you're trying to build a newer upstream version than the latest one mentioned in debian/changelog, adjust this file to reflect that: $ dch -i --release-heuristic=changelog (Fix the version number.) Then run again: $ uscan --destdir=../tarballs --download-current-version --verbose Build and commit ---------------- Check if the package properly builds: $ cd svn/debian-science/package/timbl/trunk $ DH_VERBOSE=1 svn-buildpackage -uc -us -rfakeroot --svn-ignore --svn-reuse --svn-move --svn-lintian --svn-builder=debuild (If you don't have write-access to the svn repo, run $ cd ../tarballs $ tar zxf timbl_0.4.2.orig.tar.gz $ cd timbl-0.4.2 $ svn export svn://svn.debian.org/svn/debian-science/packages/ucto/trunk/debian $ DH_VERBOSE=1 debuild -uc -us -rfakeroot instead.) Check the result: $ ls .. Inspect all files found there. Viewing a .diff.gz is easy. If however you don't use old-style .diff.gz, but your package generates .debian.tar.gz, you might like to view it's contents by running e.g.: tar -vOzxf timbl_10.8-1.debian.tar.gz 2>&1 | less Inspect content of binary packages by running e.g. $ debc ../mbt_3.2.7-2_amd64.changes | less If lintian complains, run $ lintian --info ../*.dsc | less and $ lintian --info ../*.deb | less and fix ALL the problems and record the fixes. Now, let's do some real work: $ vi debian/control Check your changes: $ svn diff Record your changes in debian/changelog (make sure $DEBEMAIL is set to your email address as listed in debian/control!): $ dch -i --release-heuristic=changelog (Or use plain $ vi debian/changelog .) Commit your change to SVN, using your debian/changelog entry as commit message: $ debcommit --diff; debcommit --confirm Now build as indicated above, and check the quality. Intermezzo: updating to new 3.0 dpkg source format -------------------------------------------------- If your package lacks a debian/source directory, create it: $ svn mkdir debian/source $ echo '3.0 (quilt)' > debian/source/format $ svn add debian/source/format $ dch -i --release-heuristic=changelog 'updated package to use 3.0 source format' $ debcommit --diff; debcommit --confirm See also http://wiki.debian.org/Projects/DebSrc3.0 and dpkg-source(1). Get ready for Release --------------------- Once the package is suitable for release: $ svn-buildpackage --svn-tag -rfakeroot --svn-reuse --svn-move --svn-builder=debuild (This assumes you have gpg setup correctly, and your key is suitable for signing the release. If not, run "svn-buildpackage --svn-tag -rfakeroot -uc -us" instead.) NB! Never modify files in the tag directory. The svn-buildpackage run ends in something like: dch -D UNRELEASED -i NOT RELEASED YET which means you'll have a pending commit in your working directory. _Do_ commit it! E.g.: $ svn commit -m 'start next release' Envoi trunk/debian/changelog . Upload to Debian ---------------- If you have upload rights: upload the package: $ cd build-area $ dupload --to anonymous-ftp-master *changes Uploading to ILK apt archive and rebuilding the index file ---------------------------------------------------------- If you don't want to wait for propagation of the packages through Debian's mirrors and Ubuntu, upload to the ILK apt archive: Make sure the upload dir is empty: $ ssh zeus $ cd /var/www/apt.ticc.uvt.nl/upload $ rm -rf * Copy generated files *.{changes,deb,diff.gz,dsc,orig.tar.gz} (or e.g. timbl_6.2.0.pre1*) to upload dir at zeus:/var/www/apt.ticc.uvt.nl/upload/. Afterwards, on zeus run $ cd /var/www/apt.ticc.uvt.nl $ make dists/lenny/Release PGP-sign the new Release file. You will need the secret key and add it to your keyring. Also a password is needed ;) $ cd /var/www/apt.ticc.uvt.nl/dists/lenny $ gpg --no-random-seed-file -bao Release.gpg Release . Rebuilding from unchanged sources --------------------------------- In order to link against wheezy libs: joostvb@bruhat:~% gpg --armor --export-secret-keys 'Joost van Baal-Ilić' | ssh virbalas gpg --import Build only source, don't sign: joostvb@bruhat:~/sv...ence/packages/ucto/trunk% svn-buildpackage -S -uc -us --svn-tag -rfakeroot --svn-reuse --svn-move --svn-builder=debuild joostvb@bruhat:~/sv...ence/packages/ucto/trunk% scp ../ucto_0.4.9-4.d* virbalas:build/ On testing/unstable host, build binary: joostvb@virbalas:~/build% dget http://apt.ticc.uvt.nl/t/libfolia_0.5-2.dsc of sloot@zeus:~/a$ dget http://ftp.debian.org/debian/pool/main/m/mbt/mbt_3.2.7-3.dsc joostvb@virbalas:~/build% ls libfolia_0.5-2.debian.tar.gz libfolia_0.5-2.dsc libfolia_0.5.orig.tar.gz joostvb@virbalas:~/build% dpkg-source -x libfolia_0.5-2.dsc Installeer eventueel nodige build-dependencies: root@virbalas:~# apt-get -V install cdbs pkg-config libicu-dev libxml2-dev Maak n nieuwe versie zoals bv mbt (3.2.7-3lenny1) unstable; urgency=low; daarvoor pas je debian/changelog aan: sloot@zeus:~/a/mbt-3.2.7$ dch -lilk --release-heuristic=changelog 'Rebuild on squeeze' (of, met t handje: sloot@zeus:~/a/mbt-3.2.7$ vi ./debian/changelog ) Daarna bouw je de .deb: joostvb@virbalas:~/build/libfolia-0.5% debuild -sa -k0B86B067 of, als je niet wilt signen, en wel de upstream tarball in je upload wilt: sloot@zeus:~/a/mbt-3.2.7$ debuild -sa -uc -us Als je verder wilt bouwen, doe je b.v. root@virbalas:~# dpkg -i ~joostvb/build/libfolia1-dev_0.5-2_amd64.deb ~joostvb/build/libfolia1_0.5-2_amd64.deb en daarna: joostvb@virbalas:~/build/ucto-0.4.9% debuild -sa -k0B86B067 joostvb@bruhat:~/sv...nce/packages/ucto/wheezy% scp virbalas:build/\*ucto\* . joostvb@bruhat:~/sv...cience/packages/libfolia% scp virbalas:build/libfolia\* . root@virbalas:~# apt-get remove libfolia1 libfolia1-dev cdbs pkg-config libicu-dev libxml2-dev root@virbalas:~# apt-get autoremove Rebuilding from unchanged sources: already tagged release from svn ------------------------------------------------------------------ joostvb@bruhat:~/sv...e/packages/libfolia/tags% ln -s ../tarballs joostvb@bruhat:~/sv...ages/libfolia/tags/0.6-1% uscan --destdir=../tarballs --download-current-version --verbose joostvb@bruhat:~/sv...ages/libfolia/tags/0.6-1% svn-buildpackage -rfakeroot --svn-reuse --svn-move --svn-builder=debuild , and, if you want to force inclusion of upstream source, use -sa: $ svn-buildpackage -sa -rfakeroot --svn-reuse --svn-move --svn-builder=debuild , and, if you need to include more, use -v3.2.5-3 (changelog >> 3.2.5-3) Signing after the fact ---------------------- $ echo DEBSIGN_KEYID=0B86B067 >> ~/.devscripts $ debsign ../ticcutils_0.4-5_amd64.changes Alternative: syncing ILK apt archive with Debian archive -------------------------------------------------------- On zeus: cd /var/www/apt.ticc.uvt.nl/upload dget http://ftp.debian.nl/debian/pool/main/t/timblserver/timblserver_1.2-4.dsc wget http://ftp.debian.nl/debian/pool/main/t/timblserver/timblserver_1.2-4_amd64.deb Be sure to copy the relevant .changes file to zeus too. Joost likely has a copy in bruhat:svn/svn.debian.org/debian-science/packages/timblserver/. More information ---------------- Generic official Debian packaging documentation: http://www.debian.org/doc/manuals/maint-guide/ http://www.debian.org/doc/manuals/developers-reference/ Official svn-buildpackage documentation: http://svn-bp.alioth.debian.org/ Tips for specific ways of working with svn-buildpackage: http://pkg-perl.alioth.debian.org/subversion.html http://workaround.org/debian-subversion-pbuilder https://trac.cyconet.org/debian/wiki/SvnBuildPackage Inofficial tutorials: Building Tutorial by Margarita Manterola e.a. at http://wiki.debian.org/BuildingTutorial "Debian Packaging Tutorial" by Lucas Nussbaum at http://www.lucas-nussbaum.net/blog/?p=676 See also svn+ssh://svn.debian.org/svn/debian-science/packages/timbl/trunk/debian/README.devel RPM === See also https://svn.ilk.uvt.nl/svn/trunk/sources/Timbl-pkg/rpm/log . (FIXME: merge that one) Building {,src}rpm from spec and .tar.gz ---------------------------------------- joostvb@bruhat:~/svn.ilk.uvt.nl/Frog/trunk% vi frog.spec joostvb@bruhat:~/svn.ilk.uvt.nl/Frog/trunk% svn commit -m 'new upstream' frog.spec joostvb@bruhat:~/svn.ilk.uvt.nl/ucto/trunk% scp ucto.spec fedora:rpmbuild/SPECS/ joostvb@fedora.mdcc.cx:~/rpmbuild/SOURCES% egrep -i sourc\|vers ../SPECS/ucto.spec joostvb@fedora.mdcc.cx:~/rpmbuild/SOURCES% wget http://software.ticc.uvt.nl/ucto-0.4.1.tar.gz (FIXME: find way to automate download. tried this: joostvb@fedora.mdcc.cx:~/rpmbuild/SOURCES% rpm -q --specfile ../SPECS/ucto.spec ucto-0.4.1-1.x86_64 joostvb@fedora.mdcc.cx:~/rpmbuild/SOURCES% rpm -q --qf %{description} --specfile ../SPECS/ucto.spec Ucto can tokenize UTF-8 encoded text files (i.e. separate words from [...] use to you. % joostvb@fedora.mdcc.cx:~/rpmbuild/SOURCES% rpm -q --qf %{source} --specfile ../SPECS/ucto.spec (none) rpm -q -i --specfile ../SPECS/ucto.spec % ) joostvb@centos-5.3.mdcc.cx:~/rpmbuild/SPECS% rpmbuild -ba timbl.spec root@centos-5.3.mdcc.cx:~# rpm -i ~joostvb/rpmbuild/RPMS/i386/timbl-6.3.2-1.i386.rpm root@fedora.mdcc.cx:~# rpm -U ~joostvb/rpmbuild/RPMS/x86_64/ucto-0.4.1-1.x86_64.rpm $ rpm -q -l timbl joostvb@bruhat:~/tmp% scp fedora:rpmbuild/SRPMS/ucto-0.4.1-1.src.rpm . joostvb@bruhat:~/tmp% scp fedora:rpmbuild/RPMS/x86_64/ucto-0.4.1-1.x86_64.rpm . joostvb@bruhat:~/tmp% scp ucto-0.4.1-1.* popper: joostvb@popper:~% scp ucto-0.4.1-1.src.rpm zeus:/var/www/ilk/packages/fedora-core-14-yum-rpm/SRPMS/ joostvb@popper:~% scp ucto-0.4.1-1.x86_64.rpm zeus:/var/www/ilk/packages/fedora-core-14-yum-rpm/14/RPMS/x86_64/ joostvb@zeus.uvt.nl:/var...s/fedora-core-14-yum-rpm% zgrep timblserver repodata/primary.xml.gz | grep location joostvb@zeus.uvt.nl:/var/www/ilk/packages/fedora-core-14-yum-rpm% createrepo --update `pwd` 25/25 * 14/RPMS/x86_64/mbtserver-0.1-1.fc14.x86_64.rpm Saving Primary metadata Saving file lists metadata Saving other metadata See also -------- Date: Thu, 12 Nov 2009 16:22:56 +0100 From: Joost van Baal To: Antal van den Bosch Cc: Ko van der Sloot Subject: timbl-6.2.0-1.fc11.x86_64.rpm (was: Re: timbl_6.2.0-1_darwin-powerpc.deb) Message-ID: <20091112152256.GM31021@bruhat.mdcc.cx> Creating binary RPM from SRPM ----------------------------- http://ilk.uvt.nl/packages/timbl-6.2.0-1.fc11.x86_64.rpm is er nu, voor Fedora 11 op x68-64. Daarnaast is er timbl-6.2.0-1.src.rpm; dat zou, met rpmbuild, voor iedere Red Hat-achtige een binary .rpm moeten kunnen opleveren. Voer daarvoor uit, als een normale gebruiker: mkdir -p ~/rpmbuild/{SOURCES,SPECS,SRPMS,RPMS} cd ~/rpmbuild/RPMS wget http://ilk.uvt.nl/packages/timbl-6.2.0-1.src.rpm rpm -i timbl-6.2.0-1.src.rpm cd ~/rpmbuild/SPECS rpmbuild -ba timbl*spec Constructing a yum repo ----------------------- joostvb@fedora.mdcc.cx:~% cat /etc/fedora-release Fedora release 14 (Laughlin) joostvb@fedora.mdcc.cx:~% rpm -q createrepo le paquetage createrepo n'est pas installé root@fedora.mdcc.cx:~# yum install createrepo ================================================================================ Paquet Architecture Version Dépôt Taille ================================================================================ Installation: createrepo noarch 0.9.8-5.fc14 fedora 81 k Installation pour dépendance: deltarpm x86_64 3.6-0.2.20100708git.fc14 fedora 74 k libxml2-python x86_64 2.7.7-2.fc14 fedora 316 k python-deltarpm x86_64 3.6-0.2.20100708git.fc14 fedora 28 k Résumé de la transaction ================================================================================ Install 4 Package(s) Taille totale des téléchargement : 498 k Installed size: 2.4 M Est-ce correct [o/N] : joostvb@fedora.mdcc.cx:~% mkdir repo && cd repo joostvb@fedora.mdcc.cx:~/repo% createrepo `pwd` joostvb@fedora.mdcc.cx:~/repo% find . . ./repodata ./repodata/other.xml.gz ./repodata/primary.xml.gz ./repodata/repomd.xml ./repodata/filelists.xml.gz root@fedora.mdcc.cx:~# vi /etc/yum.repos.d/ticc.repo root@fedora.mdcc.cx:~# cat /etc/yum.repos.d/ticc.repo # http://yum.baseurl.org/wiki/RepoCreate [ticc] name = TiCC Software baseurl = file:///home/joostvb/repo joostvb@fedora.mdcc.cx:~/repo% mkdir -p SRPMS 14/RPMS joostvb@fedora.mdcc.cx:~% cp -a rpmbuild/RPMS/x86_64 repo/14/RPMS joostvb@fedora.mdcc.cx:~% cp -a rpmbuild/SRPMS/* repo/SRPMS joostvb@fedora.mdcc.cx:~/repo% createrepo --update `pwd` joostvb@fedora.mdcc.cx:~/repo/repodata% zgrep location primary.xml.gz joostvb@fedora.mdcc.cx:~% rpm -q mbtserver le paquetage mbtserver n'est pas installé root@fedora.mdcc.cx:~# yum --nogpgcheck install mbtserver Configuration du processus d'installation Résolution des dépendances --> Lancement de la transaction de test ---> Paquet mbtserver.x86_64 0:0.1-1 marqué pour être installé --> Résolution des dépendances terminée Dépendances résolues ================================================================================ Paquet Architecture Version Dépôt Taille ================================================================================ Installation: mbtserver x86_64 0.1-1 ticc 313 k Résumé de la transaction ================================================================================ Install 1 Package(s) Taille totale des téléchargement : 313 k Installed size: 1.0 M Est-ce correct [o/N] : Téléchargement des paquets : Lancement de rpm_check_debug Lancement de la transaction de test Transaction de test réussie Lancement de la transaction Installation : mbtserver-0.1-1.x86_64 1/1 Installé: mbtserver.x86_64 0:0.1-1 Terminé ! joostvb@fedora.mdcc.cx:~% rpm -q mbtserver mbtserver-0.1-1.x86_64 obsolete: joostvb@bruhat:~% ssh fedora tar zc repo | ssh popper "cat > repo.tar.gz" joostvb@popper:~% ssh zeus tar zvx < repo.tar.gz && rm repo.tar.gz repo/SRPMS/mbt-3.2.4-1.src.rpm repo/14/RPMS/x86_64/timbl-6.3.2-1.fc14.x86_64.rpm repo/repodata/repomd.xml joostvb@popper:~% ssh zeus tar zc /var/www/ilk/packages/repo > repo12.tar.gz joostvb@bruhat:~% ssh popper cat repo12.tar.gz | ssh fedora tar zvx joostvb@fedora.mdcc.cx:~/repo% createrepo --update `pwd` joostvb@zeus.uvt.nl:/var...ar/www/ilk/packages/repo% rm -rf * joostvb@bruhat:~% ssh fedora tar zc repo | ssh popper "cat > repo.tar.gz" joostvb@popper:~% ssh zeus tar zvx < repo.tar.gz && rm repo.tar.gz repo/ joostvb@zeus.uvt.nl:/var...ar/www/ilk/packages/repo% mv ~/repo/* . joostvb@zeus.uvt.nl:/var/www/ilk/packages% ln -s repo fedora-core-14-yum-rpm yum repos and gpg signing ------------------------- root@fedora.mdcc.cx:~# yum install mbtserver [...] Le paquet mbtserver-0.1-1.fc14.x86_64.rpm n'est pas signé yum(8) --nogpgcheck Run with GPG signature checking disabled. Configuration Option: gpgcheck yum.conf(5) [repository] OPTIONS The repository section(s) take the following form: gpgcheck Either `1' or `0'. This tells yum whether or not it should perform a GPG signature check on the packages gotten from this repository. repo_gpgcheck Either `1' or `0'. This tells yum whether or not it should perform a GPG signature check on the repodata from this repository. gpgkey A URL pointing to the ASCII-armored GPG key file for the repository. This option is used if yum needs a public key to verify a package and the required key hasn't been imported into the RPM database. If this option is set, yum will automatically import the key from the specified URL. You will be prompted before the key is installed unless the assumeyes option is set. using our yum repo ------------------ root@fedora.mdcc.cx:~# cat /etc/yum.repos.d/ticc.repo [ticc] name = TiCC Software baseurl = http://ilk.uvt.nl/packages/fedora-core-14-yum-rpm root@fedora.mdcc.cx:~# rpm -q timblserver le paquetage timblserver n'est pas installé root@fedora.mdcc.cx:~# yum --nogpgcheck install timblserver ================================================================================ Paquet Architecture Version Dépôt Taille ================================================================================ Installation: timblserver x86_64 1.1-1 ticc 395 k Résumé de la transaction ================================================================================ Install 1 Package(s) Taille totale des téléchargement : 395 k Installed size: 1.4 M Installé: timblserver.x86_64 0:1.1-1 Terminé ! Fink ==== rebuilding a new timbl fink package ----------------------------------- joostvb@bruhat:~/sv...lk.uvt.nl/Timbl-pkg/fink% vi timbl.info ( update version, sha1sum ) joostvb@bruhat:~/sv...lk.uvt.nl/Timbl-pkg/fink% scp timbl.info dhcp-23:/sw/fink/10.4/local/main/finkinfo/ check: joostvb@dhcp-23.mdcc.cx:~% fink list timbl joostvb@dhcp-23.mdcc.cx:~% fink validate /sw/fink/dists/local/main/finkinfo/timbl.info build: joostvb@dhcp-23.mdcc.cx:~% fink -m --build-as-nobody rebuild timbl [...] dpkg-deb: building package `timbl' in `/sw/fink/10.5/local/main/binary-darwin-powerpc/timbl_6.2.0-1_darwin-powerpc.deb'. test: joostvb@dhcp-23.mdcc.cx:~% dpkg-deb -c /sw/fink/dists/local/main/binary-darwin-powerpc/timbl_6.2.0-1_darwin-powerpc.deb | less joostvb@dhcp-23.mdcc.cx:~% fink install timbl [more tests] upload: joostvb@bruhat:/var/tmp% scp dhcp-23:/sw/fink/dists/local/main/binary-darwin-powerpc/timbl_6.2.0-1_darwin-powerpc.deb . joostvb@bruhat:/var/tmp% scp timbl_6.2.0-1_darwin-powerpc.deb rhea.uvt.nl:/tmp/ joostvb@bruhat:/var/tmp% scp ~/svn/svn.ilk.uvt.nl/Timbl-pkg/fink/timbl.info rhea.uvt.nl:/tmp/timbl_6.2.1-1_darwin_Mac_OS_X.info joostvb@rhea:~$ scp /tmp/timbl_6.2.0-1_darwin-powerpc.deb zeus:/var/www/ilk/packages/ See also https://svn.ilk.uvt.nl/svn/trunk/sources/Timbl-pkg/fink/log and https://svn.ilk.uvt.nl/svn/trunk/doc/build-instructions-for-software-packages/get-install.html