Htcondor Native Packages

Related Contents

Tickets

Ticket #1166 : Add native Linux distro packaging into the build process
Ticket #1080 : Handle * packaging using CPack for cmake branch

Wikis

Background

Old HTCondor rpm is basically a HTCondor tar ball packaged inside RPM. It will install HTCondor into /opt/condor-{version} and runs condor_configure during post install. Below is the list of HTCondor packages provided by 3rd parties.

Fedora - Current HTCondor package is based on this one.
http://cvs.fedoraproject.org/viewvc/rpms/condor/
Debian
http://www.dcl.hpi.uni-potsdam.de/debian/binary/
Ubuntu - by Ian
http://packages.ubuntu.com/lucid/condor

Design Ideas

  1. Use dynamic tar file as a source, HTCondor native package just relocate stuff in tar file to comply with FHS.
  2. During upgrade, there is no need to explicitly restart HTCondor because the Master daemon will detect file changes and gracefully restart the daemons by itself.
  3. Minimize post installation logic by using condor_config parameters than match the standard configuration of each distribution. This will allow RPM to detect configuration file changes. However, when using relocate feature, condor_config will be modified during post install.
  4. In multi-instance installation scenario, the second instance will be installed without touching the first instance. Thus, the proper upgrade path is to remove the second instance and then upgrade the first instance.
  5. Debian have a very detailed packaging policies. HTCondor's Debian package tries to follows these policies as much as possible.

Implementation

Package Layout

Please see the following link for more detail on file layout within HTCondor Packages.
http://www.cs.wisc.edu/condor/yum/index.html#layout
http://www.cs.wisc.edu/condor/debian/index.html#layout

RPM Installation Behavior

pre (pre-install)

  1. Add condor user/group if not exists

postinst

  1. (If relocated) Update condor_config, /etc/sysconfig/condor to match new location
  2. (If relocated) Install init script only if this is the first instance.
  3. (If relocated) Create soft link to man folder in same level as bin/sbin folder so man can automatically find manpages.
  4. Add HTCondor service (chkconfig --add)
  5. Run ldconfig (Might not be necessary because we do not put any lib directly into /usr/lib)
  6. Modify SElinux policy of condor_startd binary (Copy from Fedora package)

preun

  1. Call init script to stop HTCondor only if we are removing the last instance of HTCondor (not include upgrading). If the init script fails, abort the uninstall
  2. Remove HTCondor service (chkconfig --del)
  3. (If relocated) remove init script

postun

  1. Run ldconfig (Might not be necessary because we do not put any lib directly into /usr/lib)

Debian Installation Behavior

preinst

  1. Add condor user/group if not exists

postinst

  1. Add HTCondor service (update-rc.d)
  2. Change owner of some folders to condor

prerm

  1. (If remove) Stop HTCondor

postrm

  1. Remove HTCondor service
  2. (If purge) remove files created HTCondor daemons

Package Building (Imake)

Related Files

File Description
src/condor_scripts/native/SPECS/condor.spec Spec file
src/condor_scripts/native/SPECS/filelist.txt Template for file list in %file section
src/condor_scripts/native/DEBIAN/* Metadata files for Debian package
src/condor_scripts/make_native_packages.pl Main building script
src/condor_scripts/make_native_rpm.sh RPM-specific script
src/condor_scripts/make_native_debian.sh Debian-specific script
src/condor_examples/condor.boot.vdt Used for /etc/init.d/condor
src/condor_examples/condor_config.generic.debian.patch Patch for default parameters on Debian
src/condor_examples/condor_config.generic.rpm.patch Patch for default parameters on RHEL
src/condor_examples/condor_config.local.generic Used for /etc/condor/condor.local

Build Process

During make public , make_native_packages.pl will be called. This is a main script that will detect OS distribution by looking at release file ( /etc/debian_version | /etc/redhat-release ). After deciding which type of package to build ( Deb | RPM ), it will populate a build area ( native folder ) with necessary files and invoke script specific to each type of package. Both of them will generate a package in native folder and the main script will move the package into the public folder.

make_native_rpm.sh

This script will update the spec file (version, release and date) then invoke rpmbuild (Imake detect this command and pass it into the script) to create RPM. Finally, it will modify RPM to include OS name. Key notes are:
  1. As part of the SPEC's %install section, it will generate a list of files from bin and sbin folder into filelist.txt . Thus, the final list of files in %file section is a combined list of static entries in the %file section and the generated entries in filelist.txt .
  2. condor_config, condor_config.local, /etc/sysconfig/condor and /etc/init.d/condor are marked as config(no_replace)

make_native_deb.sh

This script contains build logic similar to make_native_rpm.sh + SPEC file + rpmbuild. Key notes are:
  1. Some HTCondor libraries are not striped so we have to strip it manually. The strip program is invoked with similar argument used by dh_strip
  2. Debian helper programs (such as dpkg-shlibdeps ) read and generate metadata files in debian folder, but the final metadata folder that must be presented in the package is DEBIAN .
  3. Debian package store its sysconfig file in /etc/default/condor. Thus, this script will modify init script to match the Debian convention.

Limitations

RPM

  1. Preferred method of installation is to install into default location. This also provide the cleanest method for installing/updating/uninstalling because RPM logic will take care of everything
  2. There is not standard guide lines on how RPM should behave when installing multiple version and/or relocating the RPM. Thus, we should not encourage user to use these features extensively because users may expect different installation behavior.

Debian

  1. Many Debian packages use defconf to allow users to specify some configuration parameters during post install. For example, user may specify types of installation: central manager or personal HTCondor. This feature is out of the scope of the current package.
  2. Lintian report of the HTCondor package still contains warnings such as no manpage associated with certain commands. However, any warnings that can be fixed at the packing process have been solved.

HTCondor Packages in the New Build Process (Cmake)

  1. RPM and Debian package logic will be same. However, some new features have not been ported to the CMake's RPM.
  2. Patched version of CMake (CPack) is required to build these packages. Currently the patches are submitted to CMake developers for review.