| Core GIT Tests |
| ============== |
| |
| This directory holds many test scripts for core GIT tools. The |
| first part of this short document describes how to run the tests |
| and read their output. |
| |
| When fixing the tools or adding enhancements, you are strongly |
| encouraged to add tests in this directory to cover what you are |
| trying to fix or enhance. The later part of this short document |
| describes how your test scripts should be organized. |
| |
| |
| Running Tests |
| ------------- |
| |
| The easiest way to run tests is to say "make". This runs all |
| the tests. |
| |
| *** t0000-basic.sh *** |
| * ok 1: .git/objects should be empty after git-init in an empty repo. |
| * ok 2: .git/objects should have 256 subdirectories. |
| * ok 3: git-update-index without --add should fail adding. |
| ... |
| * ok 23: no diff after checkout and git-update-index --refresh. |
| * passed all 23 test(s) |
| *** t0100-environment-names.sh *** |
| * ok 1: using old names should issue warnings. |
| * ok 2: using old names but having new names should not issue warnings. |
| ... |
| |
| Or you can run each test individually from command line, like |
| this: |
| |
| $ sh ./t3001-ls-files-killed.sh |
| * ok 1: git-update-index --add to add various paths. |
| * ok 2: git-ls-files -k to show killed files. |
| * ok 3: validate git-ls-files -k output. |
| * passed all 3 test(s) |
| |
| You can pass --verbose (or -v), --debug (or -d), and --immediate |
| (or -i) command line argument to the test, or by setting GIT_TEST_OPTS |
| appropriately before running "make". |
| |
| --verbose:: |
| This makes the test more verbose. Specifically, the |
| command being run and their output if any are also |
| output. |
| |
| --debug:: |
| This may help the person who is developing a new test. |
| It causes the command defined with test_debug to run. |
| |
| --immediate:: |
| This causes the test to immediately exit upon the first |
| failed test. |
| |
| --long-tests:: |
| This causes additional long-running tests to be run (where |
| available), for more exhaustive testing. |
| |
| --valgrind:: |
| Execute all Git binaries with valgrind and exit with status |
| 126 on errors (just like regular tests, this will only stop |
| the test script when running under -i). Valgrind errors |
| go to stderr, so you might want to pass the -v option, too. |
| |
| Since it makes no sense to run the tests with --valgrind and |
| not see any output, this option implies --verbose. For |
| convenience, it also implies --tee. |
| |
| --tee:: |
| In addition to printing the test output to the terminal, |
| write it to files named 't/test-results/$TEST_NAME.out'. |
| As the names depend on the tests' file names, it is safe to |
| run the tests with this option in parallel. |
| |
| --with-dashes:: |
| By default tests are run without dashed forms of |
| commands (like git-commit) in the PATH (it only uses |
| wrappers from ../bin-wrappers). Use this option to include |
| the build directory (..) in the PATH, which contains all |
| the dashed forms of commands. This option is currently |
| implied by other options like --valgrind and |
| GIT_TEST_INSTALLED. |
| |
| --root=<directory>:: |
| Create "trash" directories used to store all temporary data during |
| testing under <directory>, instead of the t/ directory. |
| Using this option with a RAM-based filesystem (such as tmpfs) |
| can massively speed up the test suite. |
| |
| You can also set the GIT_TEST_INSTALLED environment variable to |
| the bindir of an existing git installation to test that installation. |
| You still need to have built this git sandbox, from which various |
| test-* support programs, templates, and perl libraries are used. |
| If your installed git is incomplete, it will silently test parts of |
| your built version instead. |
| |
| When using GIT_TEST_INSTALLED, you can also set GIT_TEST_EXEC_PATH to |
| override the location of the dashed-form subcommands (what |
| GIT_EXEC_PATH would be used for during normal operation). |
| GIT_TEST_EXEC_PATH defaults to `$GIT_TEST_INSTALLED/git --exec-path`. |
| |
| |
| Skipping Tests |
| -------------- |
| |
| In some environments, certain tests have no way of succeeding |
| due to platform limitation, such as lack of 'unzip' program, or |
| filesystem that do not allow arbitrary sequence of non-NUL bytes |
| as pathnames. |
| |
| You should be able to say something like |
| |
| $ GIT_SKIP_TESTS=t9200.8 sh ./t9200-git-cvsexport-commit.sh |
| |
| and even: |
| |
| $ GIT_SKIP_TESTS='t[0-4]??? t91?? t9200.8' make |
| |
| to omit such tests. The value of the environment variable is a |
| SP separated list of patterns that tells which tests to skip, |
| and either can match the "t[0-9]{4}" part to skip the whole |
| test, or t[0-9]{4} followed by ".$number" to say which |
| particular test to skip. |
| |
| Note that some tests in the existing test suite rely on previous |
| test item, so you cannot arbitrarily disable one and expect the |
| remainder of test to check what the test originally was intended |
| to check. |
| |
| |
| Naming Tests |
| ------------ |
| |
| The test files are named as: |
| |
| tNNNN-commandname-details.sh |
| |
| where N is a decimal digit. |
| |
| First digit tells the family: |
| |
| 0 - the absolute basics and global stuff |
| 1 - the basic commands concerning database |
| 2 - the basic commands concerning the working tree |
| 3 - the other basic commands (e.g. ls-files) |
| 4 - the diff commands |
| 5 - the pull and exporting commands |
| 6 - the revision tree commands (even e.g. merge-base) |
| 7 - the porcelainish commands concerning the working tree |
| 8 - the porcelainish commands concerning forensics |
| 9 - the git tools |
| |
| Second digit tells the particular command we are testing. |
| |
| Third digit (optionally) tells the particular switch or group of switches |
| we are testing. |
| |
| If you create files under t/ directory (i.e. here) that is not |
| the top-level test script, never name the file to match the above |
| pattern. The Makefile here considers all such files as the |
| top-level test script and tries to run all of them. A care is |
| especially needed if you are creating a common test library |
| file, similar to test-lib.sh, because such a library file may |
| not be suitable for standalone execution. |
| |
| |
| Writing Tests |
| ------------- |
| |
| The test script is written as a shell script. It should start |
| with the standard "#!/bin/sh" with copyright notices, and an |
| assignment to variable 'test_description', like this: |
| |
| #!/bin/sh |
| # |
| # Copyright (c) 2005 Junio C Hamano |
| # |
| |
| test_description='xxx test (option --frotz) |
| |
| This test registers the following structure in the cache |
| and tries to run git-ls-files with option --frotz.' |
| |
| |
| Source 'test-lib.sh' |
| -------------------- |
| |
| After assigning test_description, the test script should source |
| test-lib.sh like this: |
| |
| . ./test-lib.sh |
| |
| This test harness library does the following things: |
| |
| - If the script is invoked with command line argument --help |
| (or -h), it shows the test_description and exits. |
| |
| - Creates an empty test directory with an empty .git/objects |
| database and chdir(2) into it. This directory is 't/trash directory' |
| if you must know, but I do not think you care. |
| |
| - Defines standard test helper functions for your scripts to |
| use. These functions are designed to make all scripts behave |
| consistently when command line arguments --verbose (or -v), |
| --debug (or -d), and --immediate (or -i) is given. |
| |
| |
| End with test_done |
| ------------------ |
| |
| Your script will be a sequence of tests, using helper functions |
| from the test harness library. At the end of the script, call |
| 'test_done'. |
| |
| |
| Test harness library |
| -------------------- |
| |
| There are a handful helper functions defined in the test harness |
| library for your script to use. |
| |
| - test_expect_success <message> <script> |
| |
| This takes two strings as parameter, and evaluates the |
| <script>. If it yields success, test is considered |
| successful. <message> should state what it is testing. |
| |
| Example: |
| |
| test_expect_success \ |
| 'git-write-tree should be able to write an empty tree.' \ |
| 'tree=$(git-write-tree)' |
| |
| - test_expect_failure <message> <script> |
| |
| This is NOT the opposite of test_expect_success, but is used |
| to mark a test that demonstrates a known breakage. Unlike |
| the usual test_expect_success tests, which say "ok" on |
| success and "FAIL" on failure, this will say "FIXED" on |
| success and "still broken" on failure. Failures from these |
| tests won't cause -i (immediate) to stop. |
| |
| - test_debug <script> |
| |
| This takes a single argument, <script>, and evaluates it only |
| when the test script is started with --debug command line |
| argument. This is primarily meant for use during the |
| development of a new test script. |
| |
| - test_done |
| |
| Your test script must have test_done at the end. Its purpose |
| is to summarize successes and failures in the test script and |
| exit with an appropriate error code. |
| |
| - test_tick |
| |
| Make commit and tag names consistent by setting the author and |
| committer times to defined stated. Subsequent calls will |
| advance the times by a fixed amount. |
| |
| - test_commit <message> [<filename> [<contents>]] |
| |
| Creates a commit with the given message, committing the given |
| file with the given contents (default for both is to reuse the |
| message string), and adds a tag (again reusing the message |
| string as name). Calls test_tick to make the SHA-1s |
| reproducible. |
| |
| - test_merge <message> <commit-or-tag> |
| |
| Merges the given rev using the given message. Like test_commit, |
| creates a tag and calls test_tick before committing. |
| |
| Tips for Writing Tests |
| ---------------------- |
| |
| As with any programming projects, existing programs are the best |
| source of the information. However, do _not_ emulate |
| t0000-basic.sh when writing your tests. The test is special in |
| that it tries to validate the very core of GIT. For example, it |
| knows that there will be 256 subdirectories under .git/objects/, |
| and it knows that the object ID of an empty tree is a certain |
| 40-byte string. This is deliberately done so in t0000-basic.sh |
| because the things the very basic core test tries to achieve is |
| to serve as a basis for people who are changing the GIT internal |
| drastically. For these people, after making certain changes, |
| not seeing failures from the basic test _is_ a failure. And |
| such drastic changes to the core GIT that even changes these |
| otherwise supposedly stable object IDs should be accompanied by |
| an update to t0000-basic.sh. |
| |
| However, other tests that simply rely on basic parts of the core |
| GIT working properly should not have that level of intimate |
| knowledge of the core GIT internals. If all the test scripts |
| hardcoded the object IDs like t0000-basic.sh does, that defeats |
| the purpose of t0000-basic.sh, which is to isolate that level of |
| validation in one place. Your test also ends up needing |
| updating when such a change to the internal happens, so do _not_ |
| do it and leave the low level of validation to t0000-basic.sh. |