HTCondor Standalone ClassAd Catalog
The ClassAd Catalog is a simple but powerful system for tracking and discovering resources in a distributed system. It is a standalone system based on HTCondor technology. We use this system to keep track of distributed resources such as Kangaroo nodes. You are welcome to use it for your own purposes. This software is released under the GNU General Public License.
OverviewA catalog system is centered around a well-known catalog server. Providers of resources periodically send UDP update messages describing themselves to the server. Updates are kept for a short time, typically fifteen minutes, and then discarded. To remain in the catalog, a provider must continue to send updates. Consumers may contact the server via TCP to query for available resources. Once found, a consumer may contact a provider directly. We use the catalog to keep track of disks and CPUs, but you could use it to keep track of almost anything you can think of.
(HTCondor experts should recognize that this system is very similar to the HTCondor collector daemon and status utilities.)
The catalog uses the ClassAd language, which is the lingua franca of the HTCondor Project. This language allows one to express arbitrary structured data (much like XML), but with a type and expression system that allows one to compose complex queries. You can learn all about the details of ClassAds on another page.
ExampleLet's suppose that you have started a catalog server on a machine called "catalog.project.org" like so:
Now, suppose that a storage resource has described itself in the classad language as follows:
[ Name = "Barney"; UpdateAddress = "www.xxx.yyy.zzz"; DiskFree = 2048; AllowsAnonymous = true; Owner = "thain"; SpeaksFTP = true; LoadAverage = 0.75; ]If this classad is stored in the file "info.ad", then it could be sent to the catalog server as follows:
% catalog_update -c catalog.project.org < info.adThen, any of these queries could be used to read the ad back:
% catalog_query -c catalog.project.org -q '(DiskFree>1000)' % catalog_query -c catalog.project.org -q '(AllowsAnonymous && Owner!="firstname.lastname@example.org" )' % catalog_query -c catalog.project.org -q '(SpeaksFTP && LoadAverage<5.0 )'
DownloadsYou may download the catalog software in binary form:
- classads-0.9.2-special.tar.gz (A slightly modified ClassAds library)
- catalog-0.9.1.tar.gz (The catalog software itself)