Condor Week 2007

From Ben's Writing

Contents 1 Supplemental 2 Day 1 2.1 Tutorial: Using Condor 2.2 Tutorial: Administrating Condor 2.3 Tutorial: Condor on Windows 2.4 Tutorial: Configuring Quill 2.5 Developing in the Condor Environment 3 Day 2 3.1 Welcome 3.2 Self-configuring Condor Virtual Machine Appliances for Ad-Hoc Grids 3.3 Unclipped Condor in Windows via coLinux 3.4 Virtual Machines in Condor 3.5 CondorDB 3.6 Pegasus and DAGMan: From Concept to Execution Mapping Scientific Work-flows onto the National Cyberinfrastructure 3.7 A Managed Object Placement Service (MOPS) using Nest and GridFTP 3.8 Condor In Flight at The Hartford 3.9 High Throughput Computing on Blue Gene 3.10 Integrating Linux Technologies with Condor 4 Day 3 4.1 Interactive MPI on Demand 4.2 Solving Hard Integer Programs with MW 4.3 Glidein Factories 4.4 Challenges in Dyanmic Deployment of Condor Across Distributed Environments 4.5 Harnessing the Power of Condor for Human Genetics 4.6 Windows Condor Pool at Clemson 4.7 Lessons learned in the Purdue TeraGrid Condor Pools 4.8 OSG Operation 4.9 FermiGrid — A Condor-based Heterogeneous Campus Grid 4.10 Beauties of BLAH 4.11 Open Science Grid Security 4.12 Security Assessment of Grid Software 4.13 Privilege Separation in Condor 4.14 Signed ClassAds and Restricted Delegation 4.15 Condor Gotchas II 4.16 What's new in Condor? What's coming up? 4.17 Closing Remarks 5 Day 4 5.1 BOF: Managing Large Condor Pools 5.2 BOF: Virtualization 5.3 BOF: Windows Specific Aspects of Condor 5.3.1 Active Directory 5.3.2 Random Tips

Supplemental

Condor Developer Site

Day 1

Tutorial: Using Condor

Media:nleroy_user_tutorial.ppt

Tutorial: Administrating Condor

Media:admin_tutorial.ppt [[Media:admin_tutorial.html]

Tutorial: Condor on Windows

Media:zmiller_security_tutorial.ppt

NTSSPI - NT/Windows level authentication GSI - Public-key cryptography system

Tutorial: Configuring Quill

Media:quinn_windows.ppt

If you want to scale a Condor system, use Quill: Takes job queue and puts it into a relational database. This way queries can be run on the DB rather than on schedd | drastically decreasing the load on it.

Developing in the Condor Environment

Windows:

make.bat
dorelease.bat c:\condor

Error handling: error stacks

All client communication to Condor should be done via SOAP.

Day 2

Welcome

Media:miron_05-07-Annual.ppt

Self-configuring Condor Virtual Machine Appliances for Ad-Hoc Grids

Media:figueiredo_CondorWeek07.ppt

Grid Appliance

Unclipped Condor in Windows via coLinux

Media:neeman_condorweek2007_ou_unl_colinux_20070430.ppt

Get checkpointing on Windows. coLinux == Cooperative Linux (runs IN Windows).

Virtual Machines in Condor

Media:yoon_vmuniverse.ppt

CondorDB

Media:cieslak_madisontalk.ppt

Pegasus and DAGMan: From Concept to Execution Mapping Scientific Work-flows onto the National Cyberinfrastructure

Media:deelman_condor-week07.ppt

A Managed Object Placement Service (MOPS) using Nest and GridFTP

[[Media::fraser_cweek2.ppt]]

Condor In Flight at The Hartford

Media:nordlund_Condor_In_Flight.ppt

High Throughput Computing on Blue Gene

Media:peters_HTC_BlueGene_CondorWeek.ppt

Integrating Linux Technologies with Condor

Media:peters_HTC_BlueGene_CondorWeek.ppt

Day 3

Interactive MPI on Demand

Solving Hard Integer Programs with MW

The MW Homepage

Glidein Factories

Challenges in Dyanmic Deployment of Condor Across Distributed Environments

Harnessing the Power of Condor for Human Genetics

Windows Condor Pool at Clemson

Lessons learned in the Purdue TeraGrid Condor Pools

OSG Operation

FermiGrid — A Condor-based Heterogeneous Campus Grid

Beauties of BLAH

Open Science Grid Security

Security Assessment of Grid Software

Privilege Separation in Condor

Signed ClassAds and Restricted Delegation

Condor Gotchas II

What's new in Condor? What's coming up?

Look into callgrind and kcachgrind.

Closing Remarks

Day 4

BOF: Managing Large Condor Pools

Tricks & Tips for Managing Large Condor Pools

If you are using Quill 1, turn off history because it you will always be going to your database for the history rather than the file anyway.

Win32 machines only support so many processes, so either switch to Win64 or run less processes. This is due to some of static data structures in the kernel that only allow for a certain number of processes to be run. (Can we fix, or work around this?)

Answer: Yes, just increase the desktop heap for non-interactive processes. See Google for details.

BOF: Virtualization

BOF: Windows Specific Aspects of Condor

Active Directory

Condor config: user Greg and Todd can submit, but no other, how do we do this? Use the Condor *ALLOW_ macros: WRITEALLOW_CONFIG, for instance.

Random Tips

From the auto-install: Add INSTALLDIR_NTS, and STARTSERVICE="Y".

Run as local system account:

at 3:45 /interactive cmd

To see task lists:

schtask

that is, "schedule tasks."