Testing Thoughts

Changes to release process
  • How long do we need to run on CSL/NMI/CHTC?
  • How much notice can we give to CSL/NMI/CHTC?
  • We need release notes ahead of beta testers upgrade
  • Can we change the HTCondor release version to include the run id?

Test pool?

  • Should we run a test pool that we upgrade before the pre-release to NMI/CHTC?
    • Should NMI have a shadow pool that functions as the test pool?
  • Maybe NMI should not be running the development series?
  • Can we submit VMs to CHTC? The VMs can have any version of HTCondor installed

Testing of tarballs:

  • Still working on integrating the tarball tests in
  • RPM/Deb tests up next
  • Should we do extra manual tests before release checking for things that are hard to automate? Examples:
    • Difference in tarball size between this and last version
    • Difference in number of files between this and last version

Testing cross version

  • Should we run a test that validates compatibility between HTCondor releases, esp. point release?
    • Could we do this in NMI using "parallel" ability?
    • If not automated in NMI what about just doing a manual test before release?

Platform list

  • Meet with NMI and HTCondor to get platforms to drop
  • Likely candidates include AIX, HPUX

Improvements to build status page:

  • Change tasks to be sorted by the order they run (rather than alpha order)
  • Change colors so that tests which have not yet run are not green (make them gray?)

Improvements to builds:

  • Stop running NMI ports tests on anything except appropriate branch (it currently does not need to run on master or V7_4)