The Software Maintainer's Guide

Computing Services
Carnegie Mellon University
October 9, 1996
Edition 1.0
This guide is a reference for any individual who develops or maintains software on the Andrew System at Carnegie Mellon University. It provides an overview of the Software Management Structure and explains the procedures for maintaining software under this structure. It is also a guide to sources of additional information, such as online help and technical reports. This guide contains the following sections:

CHAPTER 1
Software Management on Andrew

1.1 Overview

In order to keep the large amount of software on Andrew organized and easily accessible, Computing Services enforces a Software Management Structure. When software is released, it is released to an environment. There are two environments on Andrew - Gamma and Beta. Only one version of an application may be released to each environment.

1.2 Software Development Areas

AFS volumes, areas of self- contained disk space, are provided to software developers for software development. Once created, a volume is mounted as a directory in the Andrew File System (AFS). To users, volumes look like any other directory, however, volumes have a set quota of storage space, and volumes may be accessed by more than one pathname. An example of a volume is an Andrew user's home directory. Their home directory is a volume mounted on the system as their userID.

Several volumes are available to software developers and maintainers for their software - source, dest, common, and object. By default, each volume has five megabytes of quota. Archive volumes are created only for software that is of a critical nature to the operation of the system(e.g. OS source, file system source).

1.2.1 The Source Area

Source volumes contain the version of the software that is to be installed. A source volume is created for every software package which is to be installed or evaluated. This guarantees that the software, whether it is of binary or source distribution, is uniformly backed-up and available through a common directory structure.

The mountpoint for a source volume is:

For example: The contents of the source volume may be in any order as long as everything contained in the source volume is related to the collection.

1.2.2 The Dest Area

The dest area is where the destination volumes are kept. These are also referred to as release volumes. These volumes hold the software that is released to the various environments. There is a single destination volume for each version of software that is compiled for a system type. When the sources are compiled, the software should be installed into the destination volume. For more information on how this is done, see Chapter 4, Compiling and Installing Software. The mountpoints for destination volumes are:

For example:

1.2.3 The Common Area

If a significant amount of data (roughly more than one megabyte) could be shared between the system types, a common volume may be used. A common volume is a single volume that contains files for software that is not system dependent, for example, help files.

This volume appears as the mountpoint common within the dest volume. For example:

Before deciding to use a common volume, please read Section 6.1, How Depot Organizes Software.

1.2.4 The Object Area

The object area is where the objects and executables are built when software is compiled. By default, this area will be located on the local disk of the work station on which the software is compiled, in:

For example:

AFS object space may be requested if there are compelling reasons to do so. Such reasons may include the need to retain unstripped binaries for debugging purposes, or because the application is so large and complex that it would be difficult and time-consuming to recreate all objects to test small changes.

The mountpoint for object volumes on AFS is:

For example:

1.2.5 Archive Volumes

The archive tree provides an online reference of unaltered vendor software. Archive volumes are created if the software is of a critical nature to the system and the reading of tapes would be time consuming (e.g. OS source, File system source).A read-only archive tree is rooted at:

Currently the vendor's name is used as the top level under archive. For example, a "dec" directory could contain "ultrix.v.2.2." It would be located in:

In cases where the name of the vendor does not provide an intuitive understanding of the software archived there, other naming conventions may be used. Archive volumes are intended to provide an area where vendor tapes can be unloaded, stored, and easily referenced.

1.3 Access Protection

Access to each of these development areas is controlled through pts groups. A pts group is used to assign access privileges to several users at once. For more information on pts groups, see the online help for pts.

1.3.1 The Commanders group

This group contains the userIDs of all of the people who will be part of the development team for a particular application. Each collection is automatically given a pts group named:

For example:

Members of the commanders group can add or delete userIDs from the commander's list.

1.3.2 The Visitors group

In cases where the source files should not be publicly available, a visitors group should be created.This group contains the userIDs of all the people who have been given access to read the source files.

The visitors group is named:

For example:

1.3.3 Checking the members of a pts group

To display which users are within a protection group, at the system prompt, type:

For example:

1.4 Environment Maintenance Tool

Environment Maintenance Tool, or EMT, is an application used to manage the collections on AFS automatically. Any operation to create, delete, or release software is processed by EMT. This ensures that everything conforms to the software management structure and that the volumes and symbolic links are constructed properly.

The Gatekeeper (who can be one or more individuals employed by Computing Services), has access to EMT's full functionality within a given tree. Through the carpe program, requests are sent to the Gatekeeper to assign and set up collections, delete collections, or release software to Beta or Gamma. All of these requests are posted to a set of electronic bboards. When requests have been received and processed, they will be posted to the following appropriate bboards.

For example,

CHAPTER 2
Communicating with the Gatekeeper: Using the carpe Program

2.1 About carpe

Carpe is an automated way to send requests to the Gatekeeper to create, delete, or release collections.

Carpe is a command line program. All of the necessary information may be included in the first line, or carpe will prompt the user until it has all the information it needs. All carpe commands use the same syntax. To use carpe, at the system prompt, type:

2.2 Creating a Collection

To request that the Gatekeeper create a collection, the following information must be provided: The local tree maintainer can create a collection with EMT and may also set the quota of the collection.

For information on how to use EMT, see the online help file for EMT.

2.2.1 Using the carpe interface

To create a volume, at the system prompt type:

Carpe will prompt the user with a series of questions about their collection. To use the default answer, simply press <return> after the question.

Example 1 Using carpe to create a collection

% carpe create
Tree name (default is local):<return>
--------------
Collection name: test
--------------
Enter a list of systems separated by spaces
(default is sun4413 pmax_42):sun4413 <return>
--------------
What version would you like to create (default is 001):<return>
--------------
Would you like source volumes created (default is yes)?<return>
--------------
Would you like AFS object volumes created (default is no)?<return>
--------------
Would you like a common volume created (default is no)?<return>
--------------
This is the first version of a collection
No clone can be made
--------------
How much quota would you like for the source volume (default is 5000)? <return>
How much quota would you like for the dest volumes (default is 5000)? <return>
--------------
What users should be added to the commanders list (default is the user)? kb3t sk4i ez00 mf3s mw5e <return>
--------------
Please enter any additional comments about this request.
End your input with a ^D
this is a test
^D
Carpe will then send the information to the Gatekeeper. The Gatekeeper will use EMT to create the collection and will notify the user when the set-up is complete. For collections in the local tree, it usually takes a day or two. For contributed collections, it takes about a week. If the release of a program is to be coordinated with another event such as an announcement, the Gatekeeper should be contacted directly. To contact the gatekeeper, post a message on org.acs.asg.request.gatekeeper.

2.2.2 Using the carpe command line flags

All of the relevant information to create a collection may be included in the command line as flags.

To do this, at the system prompt type:

Table 1 Command Line Flags for the Create Command
Flag Use
-tree <tree > Designates the tree where the collection will reside.
-collection <collection > Designates the name of the collection.
-version <version> Designates the version number of the request. Version numbers start at 001 and are incremented by one for each new version.
-systems <systems> Specifies the systems the collection can be used under. Separate the system names by spaces.
-C <version > Specifies which existing version number to clone, or copy the source files from, to create the new version.
-nC Specifies that a version should not be cloned. Instead, you will create the source files from scratch.
-ns Specifies that a source volume should not be created.
-mkcom Creates a common volume for a collection.
-obj <AFS or local> Specifies where the object volume will be created, either on AFS or your local disk.
-Qsrc <quota> Requests a new quota for the source volume.
-Qdest <quota> Requests a new quota for the dest volume.
-Qcomm <quota> Requests a new quota for the common volume.
-Qobj <quota> Requests a new quota for the object volume.
-users <users> Creates a list of users with access to the volumes.
-U Specifies that the request is urgent. You must list the reason in the comments section. This option is only available as a command line flag.
For example:

Carpe will prompt the user for any information that was not specified with a flag.

2.3 Deleting a Collection

To delete a volume, at the system prompt type:

Carpe will ask for the version numbers and the system types that are to be deleted and then ask to confirm each deletion. More than one version type may be deleted with one request.

The appropriate command line flags may also be used.

Table 2 Command Line Flags for the Delete Command
Flag Use
-tree <tree> Designates the tree where the collection will reside.
-collection<collection > Designates the name of the collection.
-version <version> Designates the version number of the request. Version numbers start at 001 and are incremented by one for each new version.
-systems <systems> Specifies the systems the collection can be used under. Separate the system names by spaces.

2.4 Releasing a Collection

For information on releasing a collection with carpe, see Chapter 5, Releasing Software

CHAPTER 3
Documenting Software

All releases must include some kind of documentation, which can consist of help files, man pages, or both. When the software is released, this documentation will become a part of the system documentation database, accessible from the help command. All help files should be written in EZ using the.help template provided. To access this template, create a file named <file name>.help and open it in EZ.

Man pages should be in troff format. This is sometimes referred to as man format.

Any changes made in subsequent releases of the application that affect the behavior of the application must be reflected in the help documents.

Tips for writing Help Files

CHAPTER 4
Compiling and Installing Software

4.1 Modifying the Makefile

The Makefile is a file which controls the process of compiling, linking, and installing software packages. The MakeFile also controls the creation of appropriate subdirectories in the destination volume and the subsequent vms allocation of appropriate files. Man pages go into the appropriate man subdirectory, help files into a help subdirectory, libraries into lib, and the executable image into bin. For any other file, create an appropriate directory in the dest area and install it there.

In order for the software to install correctly in the destination volumes, the variable DESTDIR should be used to point to the destination volume.

A Makefile excerpt follows:

Example 2 A sample Makefile excerpt

install: myprogram myprogram.man libmy.a myprogram.help
-mkdir $(DESTDIR)
-mkdir $(DESTDIR)/man
-mkdir $(DESTDIR)/man/man1
-mkdir $(DESTDIR)/bin
-mkdir $(DESTDIR)/lib
-mkdir $(DESTDIR)/help
install -s myprogram $(DESTDIR)/bin
install <lib>.a $(DESTDIR)/lib
install -m 644 myprogram.man $(DESTDIR/man/man1/ myprogram.1
install -m 644 myprogram.help (DESTDIR)/help

4.1.1 Installing Binaries

When installing binaries, the -s switch should be used with install. See the Makefile excerpt above for an example of this switch. This strips the files as they are installed and saves space on the system. If a debugger is to be used on the program as part of testing, do not use this switch.

4.1.2 Installing Non-executable files

When installing non-executable files such as text, help, or man files, the -m 644 switch sets the protection bits appropriately during the installation. See the Makefile excerpt on the previous page for an example of this switch.

4.1.3 Defining DESTDIR explicitly

DESTDIR must be defined explicitly if the sources are not located in the top level of the source volume.

DESTDIR may be defined explicitly in the Makefile, or a top-level Makefile may be created to define DESTDIR, that looks like this:

Example 3 A top-level Makefile to define DESTDIR

SUBDIRS= src docs

all install clean:

for i in $(SUBDIRS); do (cd $$i;$(MAKE)$(MFLAGS)
DESTDIR=$(DESTDIR) $@); done

This Makefile would pass the correct value for DESTDIR to the Makefiles in the lower-level directories, and they in turn would be able to use DESTDIR to install files in the correct location.

For more information, see the online help files for install.

4.2 Building a Program

4.2.1 Build

Build is a tool that is available to developers for compiling their programs. Build must be run on a machine which is running the system type for which the software is being compiled.Washing machines are provided and maintainted by Computing Services for the purpose of building software. Please see the following section for more information about washing machines.

To run build, at the system prompt type:

Build should be run on the Makefile instead of make.Build does precisely the same things as make with two exceptions. First, it causes the compile to take place in the object area. Second, build automatically sets several variables that can be used in the Makefile, such as the DESTDIR variable. Standard Makefiles can be converted to files which build can use in a distributed environment.

Before the software is compiled, make sure that all the pathnames referenced in the software are the pathnames to the files as they will be released and not the pathnames to the development areas.

Please study the online help on build before installing files

4.2.2 Washing Machines

Washing machines are provided and maintained by Computing Services for the purpose of compiling software. There is one washing machine for each supported system type. Access to the washing machines is restricted via / etc/user.permits. If a software maintainer cannot access these machines, then the washing machine maintainer should be notified. A post on org.acs.asg.request.washconfig will notify the maintainer. By default, washing machines are linked to the beta versions of /usr/local and /usr/contributed. If custom linkage to a washing machine is needed, a post with an explanation should be sent to org.acs.asg.request.washconfig.

4.3 Using Mklink

If vendor supplied Makefiles are nested and complex it can be difficult to render the conversion from make to build. A technique that is used to resolve this is to run a program called mklink. The mklink program is installed in /usr/local/bin and is used to create masses of symbolic links from the system-specific object directories to a common source tree. Standard make can then be run in the object tree.

To use mklink, telnet to a washing machine and make a directory for the application to be built under /usr/obj. Then, cd to that directory on the washing machine and run mklink with the long pathname to the source volume as the only argument along with an appropriate set of flags. Typically, mklink is run with the -RrvF flags. Mklink does not have to be rn on a washing machine. It will work on any machine given sufficient disk space, or in AFS. Running /usr/local/bin/mklink -x will output the usage message.

4.4 Testing a Program

Software that is to be installed to a beta or gamma environment needs to be tested before it is released. Local disk depot (LDD) provides an environment for testing software. LDD allows a software developer to recreate an environment on the local disk of a workstation that is, with the exception of the component to be tested, identical to the environment to which the component is to be released.

In order to set up LDD, one must have root access to a workstation. After root access has been obtained, follow the instructions below:

To test new software, edit /usr/local/depot/custom.depot.proto so that there is a path command that points to the location of the software to be tested. For example, to test wsadmin 040 on a pmax, the following line would be added to custom.depot.proto.

More information on depot and local disk depot can be found in the technical paper on depot. To retrieve the technical paper, anonymously FTP to export.acs.cmu.edu.The file is in /pub/depot.

CHAPTER 5
Releasing Software

5.1 Release Environments

Once the software has been compiled and tested, it may be released to a release environment. There are two such environments on Andrew.

5.2 Sending a Release Request with carpe

To release a version of software, a request must be sent to the Gatekeeper through carpe. The following information must be provided to the Gatekeeper:

To request the release of a collection, type:

Carpe will then prompt the user for all of the necessary information. This information will include details of any bug fixes and changes to the program. Lack of detail will be considered sufficient reason to reject the request.

Command line flags may also be used with the release command.

Table 3 Command Line Flags for the Release Command
Flag Use
-tree <tree name> Designates the tree where the collection will reside.
-collection <collection name> Designates the name of the collection.
-version <version number> Designates the version number of the request. Version numbers start at 001 and are incremented by one for each new version.
-systems <systems> Specifies the systems the collection can be used under. Separate the system names by spaces.
-environment <environment> Specifies which environment a collection should be released to.
-U Specifies that the request is urgent. You must list the reason in the comments section.
The Gatekeeper will process requests with EMT as soon as possible. For local software, Beta releases usually take one work day, Gamma releases usually take three work days. Local software must be in Beta for at least three days before it can be released to Gamma. For contributed software, releases can take several days. User Services reserves the right to defer the release of any software beyond the original release date. Any information about a delay will be posted on:

To coordinate a release with another activity such as an announcement, contact the gatekeeper directly. To contact the gatekeeper post a message on org.acs.asg.request.gatekeeper.

Releases which are defined as routine system maintenance releases are not subject to the review process. Routine system maintenance releases are defined as such by agreement between the System Manager, the Gatekeeper, and the User Services Release Coordinator. An example of a routine system maintenance release would be the release of the help file indices.

5.3 Backing Out Software

Sometimes, particularly in the Beta environment, The most recently released version software must be removed in order to correct serious bugs. To do this, request the release of an earlier version of the software. However, unless the currently-released version is unusable or damaged, a new version should be requested, the software fixed, and the new version released.

CHAPTER 6
The Release Process

When EMT releases software to an environment, it: The following sections describe how Depot makes the symbolic links to the software, and how to tell Depot how to find the software.

6.1 How Depot Organizes Software

Depot controls the mapping of software so that all of the parts that make up a collection appear in a single target location.

On workstations, /usr/contributed program binaries are found under the directory /usr/contributed/bin. Similarly, libraries are in /usr/contributed/lib, man pages in /usr/contributed/man, and so on.

However, no real files exist in these directories. Instead, these directories contain symbolic links to the appropriate files underneath /usr/contributed/depot/<collection>. This special subdirectory contains the bin, lib, man, and help directories for that specific collection.

For example, the contributed program myprog will actually sit in the directory:

but users will find it in:

For more information, read the online help files on depot and the technical paper on depot. To retrieve the technical paper, anonymously FTP to export.acs.cmu.edu. The file is in /pub/depot.

6.2 How Depot Finds Software

Files and directories will be linked to the /usr/contributed area as they appear at the top level of the destination volume. This linking is done by Depot. For example, if the top level of a destination volume contained: Depot would link the files beneath bin into /usr/contributed/bin, the files beneath lib into /usr/contributed/lib, and the man subdirectories into /usr/contributed/man.

However there may be a common volume that contains all of the help files for an application. In this case, the default depot links will not work.

To link a common volume to the system:

For more information, read the online help for depot.conf.