The minimal experimental matrix, in the order run, is shown below. For each searcher, the order of the systems and the order of the topics for each system are shown.
| Searcher | Block #1 | Block #2 |
| 1 | System 1: 1-3 | System 2: 2-4 |
| 2 | System 2: 1-3 | System 1: 2-4 |
| 3 | System 1: 3-1 | System 2: 4-2 |
| 4 | System 2: 3-1 | System 1: 4-2 |
Participants can be added in groups of 4 using the design above. If eight searchers are used, the second group of 4 searchers should use the following experiment matrix:
| Searcher | Block #1 | Block #2 |
| 5 | System 1: 1-3 | System 2: 4-2 |
| 6 | System 2: 1-3 | System 1: 4-2 |
| 7 | System 1: 3-1 | System 2: 2-4 |
| 8 | System 2: 3-1 | System 1: 2-4 |
Any integer multiple of eight searchers can be accommodated by replicating both matrices as needed. Any other integer multiple of four searchers can be accommodated by replicating only the first matrix (e.g., for 12 searchers, replicate the first matrix three times and do not use the second matrix).
Fernando López Ostenero - Webmaster |
i CLEF ' 2005