Jeff's Stork Responsibilities:
Brain dump meeting Monday June 5, 3:30pm notes:
*query Gnats category stork
*fix security bugs first !!!!!!!
query for category==Stork and sub-category==condor-security
*Jaime's nanoHUB bug reports, feature requests
complete, and merge V6_7-storkd-rewrite-branch (module=V6_7_CODE).
This branch is a complete rewrite of stork server daemon. The goal of this
effort was to take the original research-quality codebase up to a modern,
production quality daemon. As of 2006-06-09, this rewrite is incomplete, and
approx 60% done.
general improvements in V6_7-storkd-rewrite-branch:
*remove hundreds of fixed length buffers
*improved fault protection
*consolidation of many redundant code fragments into new Stork class
*consolidated multiple URL parsers into single new URL parser class.
new features in V6_7-storkd-rewrite-branch:
*added requirements and rank to job scheduling. Old stork scheduling was
previously a simple FIFO.
*checks all job attributes in submit file at submit time, not execution time
*per-job run time sandbox directories
*added throttle for starting new jobs
*increased used of [new] ClassAd views [for better or worse ;-) ] to
factor out many redundant static queries.
*consolidated all stork job ad attributes into single file stork_job_ad.h
*consolidated all stork config params into single file stork_config.h
Gnats bug fixes:
570 hung job detection uses job submit timestamp
573 eliminate fixed length strings
574 convert Stork MyString implementations to std::string
575 move Stork job queue log to SPOOL directory
672 user jobs run in LOG directory
*SC05 stork matchmaker, tag V6_7-SC2005_Stork_Demo_3-branch (See Nick)
*work with Tevfik and grad student Sirish Tummata at LSU to add storage
contacts: Tevfik Kosar <email@example.com>,
Sirish Tummala <firstname.lastname@example.org>
*support stork-announce, stork-discuss lists
*start transfers to FNAL tape, from Business School GLOW storage (old)
contact: Bill Taylor
See Gnats PR 602 for a description of this documentation request, and the
required Stork installation steps.
Configuring, Using Stork + HTCondor Credd [Carey's credd, not GreqQ's credd]:
Note: all references below are to the "old" condor_credd, aka the Stork
credential management daemon. The fact that we now have 2 distinct credential
management daemons is a huge source of confusion.
Configure a personal stork+credd:
# condor_configure --install --make-personal-stork
# condor_config_val DAEMON_LIST
DAEMON_LIST: COLLECTOR, CREDD, MASTER, NEGOTIATOR, SCHEDD, STARTD, STORK
Start HTCondor, including Stork, CredD:
Using Stork: See tutorial:
Using Stork and [Carey's Stork] Credential Manager daemon:
Store a X.509 credential in the [Stork] Credential Manager:
# stork_store_cred -t x509 -f /tmp/x509up_u19100 -N weber
Credential submitted successfully
list credentials managed by the [Stork] Credential Manager:
Direct Stork to use a credential managed by Credential Manager. Add to stork
cred_name = "weber";
[DO NOT use the x509proxy directive in the stork submit file, when using the
cred_name directive. These are exclusive. And no, I haven't yet looked up how
a conflict is resolved.
Using [Stork] CredD with MyProxy:
[see http://grid.ncsa.uiuc.edu/myproxy/credmgmt.html for MyProxy credential
[note: the whole condor_credd/myproxy interaction has a working unit test in
src/condor_credd/test_credd and src/stork/test_harness_common.sh. run the
-myproxy test with the -nomemory and -noleak options.]
First, start up a MyProxy server. See
The best way to install MyProxy is via the VDT. See Alain. Here's some VDT
test script to configure/run the MyProxy server, for testing:
--- start myproxy config ---
# first be sure to initialize shell with VDT, GLOBUS shell config!
# this works best as root if credential dirs are restricted to root
mkdir -p $TEMP_DIR
# Start MyProxy
echo "*** Starting MyProxy"
cat > $TEMP_DIR/myproxy.conf <<EOF
mkdir -p $TEMP_DIR/myproxy_repository
chmod 0700 $TEMP_DIR/myproxy_repository
$VDT_LOCATION/globus/sbin/myproxy-server -p $MYPROXY_PORT -c $TEMP_DIR/myproxy.conf -s $TEMP_DIR/myproxy_repository
--- end myproxy config ---
store proxy in myproxy server using myproxy-init:
Optionally: review contents of myproxy server, using myproxy-info
Store a credential named 'weber' in the stork credd,
from file /tmp/x509up_u19100
which is automatically refreshed from
the MyProxy Server username@host:port using the MyProxy distinguished name
`grid-cert-info -subject` :
# stork_store_cred -t x509 -f /tmp/x509up_u19100 -N weber -m username@host:port -D `grid-cert-info -subject`
Now, the condor_credd will check its managed credential every
CRED_CHECK_INTERVAL seconds [default=60] and refresh the credential from the
copy in the MyProxy server, if needed.
The next obvious question is "then why are there _two_ copies of the
credential: in both the MyProxy server, and the condor_credd?". There are 2
1) Carey did it. ;-) This code and reasoning predated Jeff.
2). Jeff can infer that the condor_credd was intended to be a HTCondor daemoncore
generic credential management daemon, and was intended to manage a wide variety
of credentials, including X.509 proxies, and using the daemoncore API. HTCondor
daemoncore clients to the condor_credd only had to retrieve credentials from
the [Stork] condor_credd, and did not have to know or care that the
condor_credd used MyProxy behind the scenes to refresh X.509 credentials.