Sane use of test_expect_failure
Originally, test_expect_failure was designed to be the opposite
of test_expect_success, but this was a bad decision. Most tests
run a series of commands that leads to the single command that
needs to be tested, like this:
test_expect_{success,failure} 'test title' '
setup1 &&
setup2 &&
setup3 &&
what is to be tested
'
And expecting a failure exit from the whole sequence misses the
point of writing tests. Your setup$N that are supposed to
succeed may have failed without even reaching what you are
trying to test. The only valid use of test_expect_failure is to
check a trivial single command that is expected to fail, which
is a minority in tests of Porcelain-ish commands.
This large-ish patch rewrites all uses of test_expect_failure to
use test_expect_success and rewrites the condition of what is
tested, like this:
test_expect_success 'test title' '
setup1 &&
setup2 &&
setup3 &&
! this command should fail
'
test_expect_failure is redefined to serve as a reminder that
that test *should* succeed but due to a known breakage in git it
currently does not pass. So if git-foo command should create a
file 'bar' but you discovered a bug that it doesn't, you can
write a test like this:
test_expect_failure 'git-foo should create bar' '
rm -f bar &&
git foo &&
test -f bar
'
This construct acts similar to test_expect_success, but instead
of reporting "ok/FAIL" like test_expect_success does, the
outcome is reported as "FIXED/still broken".
Signed-off-by: Junio C Hamano <gitster@pobox.com>
diff --git a/t/t5302-pack-index.sh b/t/t5302-pack-index.sh
index 2a2878b..67b9a7b 100755
--- a/t/t5302-pack-index.sh
+++ b/t/t5302-pack-index.sh
@@ -42,9 +42,9 @@
'both packs should be identical' \
'cmp "test-1-${pack1}.pack" "test-2-${pack2}.pack"'
-test_expect_failure \
+test_expect_success \
'index v1 and index v2 should be different' \
- 'cmp "test-1-${pack1}.idx" "test-2-${pack2}.idx"'
+ '! cmp "test-1-${pack1}.idx" "test-2-${pack2}.idx"'
test_expect_success \
'index-pack with index version 1' \
@@ -78,9 +78,9 @@
'git verify-pack -v "test-3-${pack3}.pack"'
test "$have_64bits" &&
-test_expect_failure \
+test_expect_success \
'64-bit offsets: should be different from previous index v2 results' \
- 'cmp "test-2-${pack2}.idx" "test-3-${pack3}.idx"'
+ '! cmp "test-2-${pack2}.idx" "test-3-${pack3}.idx"'
test "$have_64bits" &&
test_expect_success \
@@ -112,22 +112,22 @@
bs=1 count=20 conv=notrunc &&
git cat-file blob "$delta_sha1" > blob_2 )'
-test_expect_failure \
+test_expect_success \
'[index v1] 3) corrupted delta happily returned wrong data' \
- 'cmp blob_1 blob_2'
+ '! cmp blob_1 blob_2'
-test_expect_failure \
+test_expect_success \
'[index v1] 4) confirm that the pack is actually corrupted' \
- 'git fsck --full $commit'
+ '! git fsck --full $commit'
test_expect_success \
'[index v1] 5) pack-objects happily reuses corrupted data' \
'pack4=$(git pack-objects test-4 <obj-list) &&
test -f "test-4-${pack1}.pack"'
-test_expect_failure \
+test_expect_success \
'[index v1] 6) newly created pack is BAD !' \
- 'git verify-pack -v "test-4-${pack1}.pack"'
+ '! git verify-pack -v "test-4-${pack1}.pack"'
test_expect_success \
'[index v2] 1) stream pack to repository' \
@@ -150,16 +150,16 @@
bs=1 count=20 conv=notrunc &&
git cat-file blob "$delta_sha1" > blob_4 )'
-test_expect_failure \
+test_expect_success \
'[index v2] 3) corrupted delta happily returned wrong data' \
- 'cmp blob_3 blob_4'
+ '! cmp blob_3 blob_4'
-test_expect_failure \
+test_expect_success \
'[index v2] 4) confirm that the pack is actually corrupted' \
- 'git fsck --full $commit'
+ '! git fsck --full $commit'
-test_expect_failure \
+test_expect_success \
'[index v2] 5) pack-objects refuses to reuse corrupted data' \
- 'git pack-objects test-5 <obj-list'
+ '! git pack-objects test-5 <obj-list'
test_done
