package test; /** Base class of tests.

Tests are created by subclassing Test and overriding the perform method. The test author may optionally also override the clean and initialize methods. Tests should have a public no-argument constructor (such as the default constructor). Optionally, tests may include two static fields, notice and prerequisites, which are used by the testing library for printing progress, and for deciding the order in which tests are run, respectively. When a test has finished, it should call success or failure to indicate the outcome. A minimal test class therefore is:

    public class MinimalTest extends test.Test
    {
        protected void perform()
        {
            success();
        }
    }

A more complex example:

    public class ComplexTest extends test.Test
    {
        public static final String  notice = "checking property X";
        public static final Class[] prerequisites =
            new Class[] {OtherTest.class, AnotherTest.class};

        // Declarations of members go here, including system resources.

        protected void perform()
        {
            // Test body here. The test may start threads.

            success();
        }

        protected void initialize()
        {
            // Initialize system resources.
        }

        protected void clean()
        {
            // Clean up system resources and terminate test threads.
        }
    }

Exceptions thrown from initialize and perform are equivalent to calls to failure. Exceptions thrown from clean are equivalent to calls to cleanupFailure. Returning from perform or clean is equivalent to calling success and cleanupSuccess, respectively.

Tests are run as part of a test series. This is captured by {@link Series} objects.

The perform method runs the bulk of the test. This method, or a method in another thread, should eventually call success or failure to indicate the result of the test. If neither the perform method nor a method run by one of the threads started by perform calls either success or failure within a certain time period, the test will be terminated by a timer started by the testing library. The timer will itself call failure.

The first call to success or failure determines the outcome of the test. Subsequent calls to either method have no effect.

After the test is stopped, whether by an explicit call to success or failure, or due to timeout, the clean method is called. The purpose of the clean method is to stop test threads and to release system resources acquired by the test. The clean method must likewise eventually cause either cleanupSuccess or cleanupFailure to be called. If the clean method fails to do so, the cleanup process will time out, and will be considered to have failed.

Because of the test timer, and because perform might start many threads, a test may, in principle, terminate at any point after perform is started, including while it is still running, and therefore clean may start at any time while perform is still running. The intended coding style for perform and clean then is that clean will set references to system objects to null, so that perform will receive NullPointerExceptions when it attempts to use them. clean should also close sockets, streams, and other I/O objects that perform or other test threads may be blocked on, to ensure their speedy termination. The exceptions caused by these disruptions will lead to calls to failure by perform and the other threads. These calls, however, will be safely ignored, as the outcome of the test will already have been determined by the time clean is started.

This coding style complicates initialization. If a system object reference is being initialized inside perform, there is a race condition with clean: clean may run first, find that the reference is still null, and do nothing to release the object. perform, which has not yet been terminated by an exception, may then set the reference to a new system object, and continue using it. To prevent this scenario, a third method initialize is provided to the test author. initialize is as perform, the difference being that it is guaranteed that clean will only be called after initialize has completed. It is therefore safe to initialize system objects within initialize without taking additional safety precautions.

Tests should perform no system object initialization within their constructors. In fact, the intent is that the default constructor be used for the majority of tests. A derived class should generally not have an explicit written constructor. The reason is that if initialization fails while partially complete in initialize, the test author may simply call failure (or allow an exception to escape), and rely on the subsequent call to clean to clean up the objects that were created. If, instead, initialization fails while partially complete in the constructor, then the testing library cannot call clean - there is no object yet to call clean on.

The testing library expects every test to have a public no-argument constructor. The default constructor suffices for this purpose. If the author wishes to write constructors, one must be provided which is public and takes no arguments. Constructors written by the author may throw exceptions, and these will be caught by the testing library. Generally, the rules for constructor exceptions are the same as for exceptions thrown from initialize and perform.

Failure of a test during the initialize or perform methods is considered to be regular test failure, and the testing library may try to run more tests. Failure of a test during cleanup is considered a fatal error, and the testing library will not attempt any more tests. This is because the test that failed to clean up after itself may be holding exclusive resources, which may cause subsequent tests to fail when they are unable to acquire them.

As a side effect, timeout during initialization will likely lead to a fatal error. This is because the clean method does not attempt to terminate initialize, waiting instead for it to complete. If the initialize method does not stop on its own soon after it times out, the clean method will also time out and fail.

A test may make assumptions about the correctness of certain features, which are tested by another test. In this case, it is highly desirable for the other test to be run first. The test author may indicate this dependency to the testing library by creating a public, static, final field in the test class with the name prerequisites, as shown in the example above.

Another static field, notice, may be used to give the message that should be printed when the test is run by the Series.run method, if printing is turned on.

Each test object has an optional task message, settable by the author. The task message describes the current activity of the test. If a test failure occurs while there is a task message set, the exception representing the failure will include the message. Later, when the failure report is printed, the message will be printed as part of the description of the failure. This is especially useful when a test performs long-running operations which may time out. If such an operation times out with no task message set, the report printed will simply state that there was a timeout, leaving the reader with no hint of what the test was doing when the timeout occurred. In this case, the test author should set the task message before starting the operation, and clear it after the operation completes.

Calls to failure by the testing library itself, described above, may instead bypass failure and directly call a library-private implementation method. It is therefore not, in general, possible to override failure and expect to capture every attempt to stop the test. In particular, the timeout threads deliberately do not call failure, and instead call the underlying implementation method. */ public abstract class Test { /** Test state reference. See {@link Series.TestState}.

This reference is set by a thread in the Series object immediately after the test object is constructor, but is not available while the constructor is still running. */ Series.TestState state = null; /** Called to initialize the test object.

This method is called before perform. It is guaranteed that the clean method will not be called until initialize has exited. The initialize method, or threads started by initialize, may call success or failure to terminate the test. If this occurs, clean will be called afterwards. The test may also be terminated by timeout. Therefore, clean should be prepared to handle a partially-initialized test object. Code in clean and initialize should be written in a way that allows partially-initialized objects to be reliably cleaned up.

Throwing any exception from initialize results in a call to failure. If the exception thrown is assignable to TestFailed, the reason argument given to failure is the exception. Otherwise, the exception is wrapped in a TestFailed object. Test authors overriding initialize are strongly encouraged to mark the subclass implementation as throwing only TestFailed, and throw only that exception. @throws Throwable Upon test failure. The Throwable is wrapped in a TestFailed object, if it is not already one, and passed to failure. */ protected void initialize() throws Throwable { } /** Called to perform the test.

This method is called after initialize. The clean method may be called in another thread while this method is running. perform should therefore be ready to handle the effects of the actions done by clean. Generally, clean will set object references to null and close I/O streams that perform or threads started by perform are using. It is acceptable for perform to throw exceptions in response to these actions. These exception will not affect the outcome of the test, as the outcome has been determined by the time clean has started running. The purpose of these exceptions is to terminate perform and any threads started by it as quickly as possible. The test author must take care, however, that the exceptions do not adversely affect state internal to the test itself, and do not damage system objects.

perform or one of the threads started by it should call success or failure once the outcome of the test is known. perform is run within its own thread by the testing library. There is an additional parallel thread started by the testing library, which will call failure after a timeout interval if perform fails to terminate the test.

Throwing any exception from perform results in a call to failure. Remarks regarding exceptions thrown from perform are the same as for exceptions thrown from initialize. In particular, the test author is strongly encouraged to mark subclass implementations as throwing only TestFailed.

Returning from perform causes success to be called. If this is not desired, the test author should have perform wait until some condition internal to the test is satisfied. @throws Throwable Upon test failure. The Throwable is wrapped in a TestFailed object, if it is not already one, and passed to failure. */ protected abstract void perform() throws Throwable; /** Called to terminate test threads and release system resources.

This method is called after success or failure has been called, and the test has been terminated. Note that in addition to the code written by the test author, there is also a timeout thread in the testing library which will call failure after the timeout interval. clean is guaranteed to be called only after initialize has completed, but may be called during perform.

clean, or another thread started by it, should call cleanupSuccess or cleanupFailure. Throwing any exception from clean is equivalent to a call to cleanupFailure. A thread is started in parallel with clean. If the cleanup process fails to terminate before a timeout interval, this other thread calls cleanupFailure.

Cleanup failure is a fatal error. If a test is unable to release system resources, subsequent tests, which may demand them, will be unable to acquire them. This can lead to potentially misleading test failure messages. To prevent this, the testing library does not run any more tests after a test fails in the cleanup process.

Returning from clean is considered to be an implicit call to cleanupSuccess. @throws Throwable If cleanup cannot be completed. This results in a fatal error. */ protected void clean() throws Throwable { } /** Terminates the test as successful.

This method should be called from initialize or perform, or from one of the threads started by them. If success or failure have not yet been called, calling this method results in a later call to clean. Otherwise, calling this method has no effect. @param reason Reason for the test failure. The test author is strongly encouraged to make the reason a descriptive TestFailed object which explains the situation that lead to the underlying exception, if there is one. However, for convenience, it is possible to call failure with any Throwable as the argument. @throws IllegalStateException If the test object is still being constructed, and therefore neither initialize not perform have been called. */ public void failure(Throwable reason) { if(state == null) { throw new IllegalStateException("test failure method called " + "while test is being constructed"); } state.stop(reason); } /** Indicates that cleanup has been successfully completed.

This method should be called from clean or from one of the threads started by it. If cleanupSuccess or cleanupFailure have not yet been called, this terminates the cleanup process. Otherwise, calling this method has no effect.

Calling this method before clean has been called results in a call to failure, and is a fatal error. @throws IllegalStateException If the test object is still being constructed, and therefore clean has not been called. */ public void cleanupSuccess() { if(state == null) { throw new IllegalStateException("test cleanupSuccess method " + "called while test is being " + "constructed"); } state.stopCleanup(null); } /** Indicates that cleanup has failed.

Calling this method before clean has been called results in a call to failure, and is a fatal error. @param reason Reason for the cleanup failure. @throws IllegalStateException If the test object is still being constructed, and therefore clean has not been called. */ public void cleanupFailure(Throwable reason) { if(state == null) { throw new IllegalStateException("test cleanupFailure method " + "called while test is being " + "constructed"); } state.stopCleanup(new FatalError("cleanup failed with explicit call " + "to cleanupFailure", reason)); } /** Sets the test task message.

Any existing task message is replaced. @param description Description of the current task. @throws IllegalStateException If the test object is still being constructed. */ public void task(String description) { if(state == null) { throw new IllegalStateException("test task method called while " + "test is being constructed"); } state.task(description); } /** Clears the task message. @throws IllegalStateException If the test object is still being constructed. */ public void task() { if(state == null) { throw new IllegalStateException("test task method called while " + "test is being constructed"); } state.task(null); } }