Table of Contents Executive Summary 3

3.2Middleware/Software

As in all major software distributions, significant effort must be given to ongoing support. OSG 1.0 was released in August 2008 and was extensively documented in last year’s annual report. Subsequently, there have been 24 incremental updates to OSG 1.0, which demonstrates that OSG 1.0 was successful in one of our main goals: being able to support incremental updates to the software stack, something that has been traditionally challenging in the OSG software stack. That said, there were still significant challenges to being able support incremental updates using our distribution mechanisms (packaging softaresoftware called Pacman), so in early 2009, we developed OSG 1.2. Our goal was to release OSG 1.2 before the restart of the LHC so that sites could install the new version. We had a pre-release in June 2009, and it was formally released in July 2009, which gave sites sufficient time to upgrade if they chose to do so, and roughly 60% of the sites in OSG have done so; since the initial release in June we have released 11 software updates to OSG 1.2.

Most of the software updates to the OSG software stack were “standard” updates featuring numerous bug fixes, security fixes, and occasional minor feature upgrades. It should be noted that this general maintenance consumes roughly 50% of the effort of the OSG Software effort.

There have been several software updates and events in the last year that are worthy of deeper discussion. As background, the OSG software stack is based on the VDT grid software distribution. The VDT is grid-agnostic and used by several grid projects including OSG, TeraGrid, and WLCG. The OSG software stack is the VDT with the addition of OSG-specific configuration.

1. 
   VDT 1.10.1v (released in April 2009) was a significant new update that stressed our ability to supply a major incremental upgrade without requiring complete re-installations. To do this, we supplied a new update program that assists site administrators with the updating process and ensures that it is done correctly. This updater will be used for all future updates provided by the VDT. The update provided a new version of Globus, an update to our authorization infrastructure, and an update to our information infrastructure. It underwent significant testing both internally and by VOs in our integration testbed.
   
2. 
   OSG 1.2 was released in July 2009. Not only did it significantly improve our ability to provide updates to users, but it also added support for a new operating system (Debian 5), which is required by LIGO. This differed from VDT 1.10.1v in that we tackled a new source of packaging difficulty which prevented us from easily adding new platform support without disrupting our ability to provide incremental updates for other users.
   
3. 
   Since the summer of 2009, we have been focusing on the needs of the upcoming ATLAS and CMS Tier-3 sites. In particular, we have focused on Tier-3 support, particularly with respect to new storage solutions. We have improved our packaging, testing, and releasing of BeStMan, XRootd, and Hadoop, which are a large part of our set of storage solutions.
   
4. 
   We have emphasized improving our storage solutions in OSG. This is partly for the Tier-3 effort mention in item #3, but is also for broader use in OSG. For example, we have created new testbeds for Xrootd and Hadoop and expanded our test suite to ensure that the storage software we support and release are well tested and understood internally. We have started regular meetings with the XRootd developers and ATLAS to make sure that we understand how development is proceeding and what changes are needed. We have also provided new tools to help users query our information system for discovering information about deployed storage systems, which has traditionally been hard in OSG. We expect these tools to be particularly useful to LIGO and SCEC, though other VOs will likely benefit as well. We also conducted an in-person storage forum in June 2009 at Fermnilab, to help us better understand the needs of our users and to directly connect them with storage experts.
   

The VDT continues to be used by external collaborators. EGEE/WLCG uses portions of VDT (particularly Condor, Globus, UberFTP, and MyProxy). The VDT team maintains close contact with EGEE/WLCG due to the OSG Software Coordinator's (Alain Roy's) weekly attendance at the EGEE Engineering Management Team's phone call. TeraGrid and OSG continue to maintain a base level of interoperability by sharing a code base for Globus, which is a release of Globus, patched for OSG and TeraGrid’s needs. The VDT software and storage coordinators (Alain Roy and Tanya Levshina) are members of the WLCG Technical Forum whichForum, which is addressing ongoing lacks, needs and evolution of the WLCG infrastructure in the face of data taking.