Requirements for New Diagnostic Tool

We needed new tool which would:

• Apply principle of bisectioning to reduce number of commits to be run

• Focus on differences in entire test output rather than just test result (PASS/FAIL)

  • Has output of lib/locale.t at commit N changed from commit N - 1?

  • Identifying "First FAIL" not sufficient

  • We know lib/locale.t failed in at least 2 different ways

• Need to identify each commit at which test output changed

  • After we identify first change in test output, must repeatedly bisect