@RepeatedTest

From here, three chapters look at running one test method more than once. This one covers the simplest case.

@RepeatedTest runs the same test, with the same input, several times.

@RepeatedTest(value = 5)   // run this test 5 times
void repeatedTest() {
    System.out.println("Running test...");
}

@RepeatedTest replaces @Test; do not put both.

It is useful for:

• Detecting intermittent issues: code that works sometimes and fails sometimes.
• Testing idempotency: the same call repeated gives the same result.
• Stress testing: running code repeatedly to check that it behaves properly under heavy, prolonged load.

Each repetition is a separate test

From the lifecycle chapter: with PER_METHOD (the default) a new instance is created for each test; with PER_CLASS, one instance serves all tests. For @RepeatedTest, each repetition counts as an independent test: with PER_METHOD it gets a new instance and its own @BeforeEach/@AfterEach.

public class MyServiceTest {

    public MyServiceTest() {
        System.out.println("instance created");
    }

    @BeforeAll
    public static void beforeAllMethod() {
        System.out.println("inside before all method");
    }

    @BeforeEach
    public void beforeEachMethod() {
        System.out.println("inside before each method");
    }

    @Test
    void testMethod() {
        System.out.println("inside testMethod");
    }

    @RepeatedTest(value = 2)
    void testRepeatedMethod() {
        System.out.println("inside testRepeatedMethod");
    }

    @AfterEach
    public void afterEachMethod() {
        System.out.println("inside after each method");
    }

    @AfterAll
    public static void afterAllMethod() {
        System.out.println("inside after all method");
    }
}

inside before all method

instance created
inside before each method
inside testMethod
inside after each method

instance created
inside before each method
inside testRepeatedMethod
inside after each method

instance created
inside before each method
inside testRepeatedMethod
inside after each method

inside after all method

RepetitionInfo

A repeated test can take a RepetitionInfo parameter, which tells it which repetition is running. JUnit supplies it through a built-in parameter resolver (resolvers are covered in the extensions chapter), so nothing needs configuring.

@RepeatedTest(value = 2)
void testRepeatedMethod(RepetitionInfo repetitionInfo) {
    System.out.println("Running Test : currentRepetition: " + repetitionInfo.getCurrentRepetition()
            + ", totalRepetitions: " + repetitionInfo.getTotalRepetitions()
            + ", failureCount: " + repetitionInfo.getFailureCount()
            + ", failureThreshold: " + repetitionInfo.getFailureThreshold());
}

Running Test : currentRepetition: 1, totalRepetitions: 2, failureCount: 0, failureThreshold: 2147483647
Running Test : currentRepetition: 2, totalRepetitions: 2, failureCount: 0, failureThreshold: 2147483647

Method	Meaning
getCurrentRepetition()	which repetition is running (1, 2, 3, ...)
getTotalRepetitions()	the total configured repetitions
getFailureCount()	how many earlier repetitions failed
getFailureThreshold()	how many failures are tolerated

The failure threshold

failureThreshold says how many failures are tolerated; once that many repetitions have failed, the remaining repetitions are skipped.

@RepeatedTest(value = 10, failureThreshold = 3)
void testWithThreshold() {
    throw new RuntimeException("boom");
}

repetition 1 of 10   ✘ failed
repetition 2 of 10   ✘ failed
repetition 3 of 10   ✘ failed
repetition 4 of 10   ↷ skipped: Failure threshold [3] exceeded
...
repetition 10 of 10  ↷ skipped: Failure threshold [3] exceeded

The default is Integer.MAX_VALUE (the 2147483647 printed above): every repetition runs, however many fail.

Names

@DisplayName

A general annotation that gives a test class or method a human-readable name in the results, instead of the method name:

@Test
@DisplayName("Dummy test to learn JUnit")
void testMethod() {
    System.out.println("inside test method");
}

Naming repetitions

@DisplayName names the repeated test as a whole; each repetition gets its name from the name attribute of @RepeatedTest. A fixed text would give every repetition the same name, so use the placeholders:

• {displayName}: the test's display name
• {currentRepetition}: the current repetition number
• {totalRepetitions}: the total

and two ready-made formats:

@RepeatedTest(value = 5, name = RepeatedTest.LONG_DISPLAY_NAME)
@DisplayName("Dummy test")
void testMethod() { }
// Dummy test :: repetition 1 of 5, Dummy test :: repetition 2 of 5, ...

@RepeatedTest(value = 5, name = RepeatedTest.SHORT_DISPLAY_NAME)   // the default
@DisplayName("Dummy test")
void testMethod() { }
// repetition 1 of 5, repetition 2 of 5, ...

@RepeatedTest(value = 5, name = "{displayName}, repetition no {currentRepetition}/{totalRepetitions}")
@DisplayName("Dummy test")
void testMethod() { }
// Dummy test, repetition no 1/5, Dummy test, repetition no 2/5, ...

@RepeatedTest as a meta-annotation

@RepeatedTest may be placed on other annotations (its @Target includes annotation types), so you can build your own (custom annotations are covered in the Java series):

@Target({ ElementType.METHOD })
@Retention(RetentionPolicy.RUNTIME)
@RepeatedTest(10)
public @interface TenIterations {
}

@TenIterations
void testMethod() {
    System.out.println("inside test method");   // runs 10 times
}

Next: Repeating a test with the same input is rare. Chapter 8 runs a test with different inputs each time: parameterized tests.

Check yourself

1. What does @RepeatedTest(5) do? Name three uses.
2. With PER_METHOD, how many instances and @BeforeEach calls does a class with one @Test and one @RepeatedTest(2) produce?
3. How does a test find out which repetition is running?
4. @RepeatedTest(value = 10, failureThreshold = 3) fails every time. What happens to repetitions 4 to 10?
5. What is the default failure threshold, and what does it mean?
6. How do you name each repetition "Dummy test, repetition no 1/5"?
7. How can you create a @TenIterations annotation?

Answers

1. Runs the same test five times. Detecting intermittent issues, testing idempotency, stress testing.
2. Three instances and three @BeforeEach calls: each repetition is a separate test.
3. With a RepetitionInfo parameter (getCurrentRepetition()).
4. They are skipped: "Failure threshold [3] exceeded".
5. Integer.MAX_VALUE: all repetitions run, whatever the failures.
6. @RepeatedTest(value = 5, name = "{displayName}, repetition no {currentRepetition}/{totalRepetitions}") with @DisplayName("Dummy test").
7. Annotate a runtime-retained annotation interface with @RepeatedTest(10).