ci: parallelize testing on Windows

The fact that Git's test suite is implemented in Unix shell script that
is as portable as we can muster, combined with the fact that Unix shell
scripting is foreign to Windows (and therefore has to be emulated),
results in pretty abysmal speed of the test suite on that platform, for
pretty much no other reason than that language choice.

For comparison: while the Linux build & test is typically done within
about 8 minutes, the Windows build & test typically lasts about 80
minutes in Azure Pipelines.

To help with that, let's use the Azure Pipeline feature where you can
parallelize jobs, make jobs depend on each other, and pass artifacts
between them.

The tests are distributed using the following heuristic: listing all
test scripts ordered by size in descending order (as a cheap way to
estimate the overall run time), every Nth script is run (where N is the
total number of parallel jobs), starting at the index corresponding to
the parallel job. This slicing is performed by a new function that is
added to the `test-tool`.

To optimize the overall runtime of the entire Pipeline, we need to move
the Windows jobs to the beginning (otherwise there would be a very
decent chance for the Pipeline to be run only the Windows build, while
all the parallel Windows test jobs wait for this single one).

We use Azure Pipelines Artifacts for both the minimal Git for Windows
SDK as well as the built executables, as deduplication and caching close
to the agents makes that really fast. For comparison: while downloading
and unpacking the minimal Git for Windows SDK via PowerShell takes only
one minute (down from anywhere between 2.5 to 7 when using a shallow
clone), uploading it as Pipeline Artifact takes less than 30s and
downloading and unpacking less than 20s (sometimes even as little as
only twelve seconds).

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
5 files changed
tree: aef39b932e1dcf935abb6e38a87ceafe22ea075e
  1. .github/
  2. block-sha1/
  3. builtin/
  4. ci/
  5. compat/
  6. contrib/
  7. Documentation/
  8. ewah/
  9. git-gui/
  10. gitk-git/
  11. gitweb/
  12. mergetools/
  13. negotiator/
  14. perl/
  15. po/
  16. ppc/
  17. refs/
  18. sha1dc/
  19. t/
  20. templates/
  21. vcs-svn/
  22. xdiff/
  23. .clang-format
  24. .editorconfig
  25. .gitattributes
  26. .gitignore
  27. .gitmodules
  28. .mailmap
  29. .travis.yml
  30. .tsan-suppressions
  31. abspath.c
  32. aclocal.m4
  33. advice.c
  34. advice.h
  35. alias.c
  36. alias.h
  37. alloc.c
  38. alloc.h
  39. apply.c
  40. apply.h
  41. archive-tar.c
  42. archive-zip.c
  43. archive.c
  44. archive.h
  45. argv-array.c
  46. argv-array.h
  47. attr.c
  48. attr.h
  49. azure-pipelines.yml
  50. banned.h
  51. base85.c
  52. bisect.c
  53. bisect.h
  54. blame.c
  55. blame.h
  56. blob.c
  57. blob.h
  58. branch.c
  59. branch.h
  60. builtin.h
  61. bulk-checkin.c
  62. bulk-checkin.h
  63. bundle.c
  64. bundle.h
  65. cache-tree.c
  66. cache-tree.h
  67. cache.h
  68. chdir-notify.c
  69. chdir-notify.h
  70. check-builtins.sh
  71. check-racy.c
  72. check_bindir
  73. checkout.c
  74. checkout.h
  75. color.c
  76. color.h
  77. column.c
  78. column.h
  79. combine-diff.c
  80. command-list.txt
  81. commit-graph.c
  82. commit-graph.h
  83. commit-reach.c
  84. commit-reach.h
  85. commit-slab-decl.h
  86. commit-slab-impl.h
  87. commit-slab.h
  88. commit.c
  89. commit.h
  90. common-main.c
  91. config.c
  92. config.h
  93. config.mak.dev
  94. config.mak.in
  95. config.mak.uname
  96. configure.ac
  97. connect.c
  98. connect.h
  99. connected.c
  100. connected.h
  101. convert.c
  102. convert.h
  103. copy.c
  104. COPYING
  105. credential-cache--daemon.c
  106. credential-cache.c
  107. credential-store.c
  108. credential.c
  109. credential.h
  110. csum-file.c
  111. csum-file.h
  112. ctype.c
  113. daemon.c
  114. date.c
  115. decorate.c
  116. decorate.h
  117. delta-islands.c
  118. delta-islands.h
  119. delta.h
  120. detect-compiler
  121. diff-delta.c
  122. diff-lib.c
  123. diff-no-index.c
  124. diff.c
  125. diff.h
  126. diffcore-break.c
  127. diffcore-delta.c
  128. diffcore-order.c
  129. diffcore-pickaxe.c
  130. diffcore-rename.c
  131. diffcore.h
  132. dir-iterator.c
  133. dir-iterator.h
  134. dir.c
  135. dir.h
  136. editor.c
  137. entry.c
  138. environment.c
  139. exec-cmd.c
  140. exec-cmd.h
  141. fast-import.c
  142. fetch-negotiator.c
  143. fetch-negotiator.h
  144. fetch-object.c
  145. fetch-object.h
  146. fetch-pack.c
  147. fetch-pack.h
  148. fmt-merge-msg.h
  149. fsck.c
  150. fsck.h
  151. fsmonitor.c
  152. fsmonitor.h
  153. fuzz-pack-headers.c
  154. fuzz-pack-idx.c
  155. generate-cmdlist.sh
  156. gettext.c
  157. gettext.h
  158. git-add--interactive.perl
  159. git-archimport.perl
  160. git-bisect.sh
  161. git-compat-util.h
  162. git-cvsexportcommit.perl
  163. git-cvsimport.perl
  164. git-cvsserver.perl
  165. git-difftool--helper.sh
  166. git-filter-branch.sh
  167. git-instaweb.sh
  168. git-legacy-rebase.sh
  169. git-merge-octopus.sh
  170. git-merge-one-file.sh
  171. git-merge-resolve.sh
  172. git-mergetool--lib.sh
  173. git-mergetool.sh
  174. git-p4.py
  175. git-parse-remote.sh
  176. git-quiltimport.sh
  177. git-rebase--am.sh
  178. git-rebase--common.sh
  179. git-rebase--merge.sh
  180. git-rebase--preserve-merges.sh
  181. git-remote-testgit.sh
  182. git-request-pull.sh
  183. git-send-email.perl
  184. git-sh-i18n.sh
  185. git-sh-setup.sh
  186. git-stash.sh
  187. git-submodule.sh
  188. git-svn.perl
  189. GIT-VERSION-GEN
  190. git-web--browse.sh
  191. git.c
  192. git.rc
  193. gpg-interface.c
  194. gpg-interface.h
  195. graph.c
  196. graph.h
  197. grep.c
  198. grep.h
  199. hash.h
  200. hashmap.c
  201. hashmap.h
  202. help.c
  203. help.h
  204. hex.c
  205. http-backend.c
  206. http-fetch.c
  207. http-push.c
  208. http-walker.c
  209. http.c
  210. http.h
  211. ident.c
  212. imap-send.c
  213. INSTALL
  214. interdiff.c
  215. interdiff.h
  216. iterator.h
  217. json-writer.c
  218. json-writer.h
  219. khash.h
  220. kwset.c
  221. kwset.h
  222. levenshtein.c
  223. levenshtein.h
  224. LGPL-2.1
  225. line-log.c
  226. line-log.h
  227. line-range.c
  228. line-range.h
  229. linear-assignment.c
  230. linear-assignment.h
  231. list-objects-filter-options.c
  232. list-objects-filter-options.h
  233. list-objects-filter.c
  234. list-objects-filter.h
  235. list-objects.c
  236. list-objects.h
  237. list.h
  238. ll-merge.c
  239. ll-merge.h
  240. lockfile.c
  241. lockfile.h
  242. log-tree.c
  243. log-tree.h
  244. ls-refs.c
  245. ls-refs.h
  246. mailinfo.c
  247. mailinfo.h
  248. mailmap.c
  249. mailmap.h
  250. Makefile
  251. match-trees.c
  252. mem-pool.c
  253. mem-pool.h
  254. merge-blobs.c
  255. merge-blobs.h
  256. merge-recursive.c
  257. merge-recursive.h
  258. merge.c
  259. mergesort.c
  260. mergesort.h
  261. midx.c
  262. midx.h
  263. name-hash.c
  264. notes-cache.c
  265. notes-cache.h
  266. notes-merge.c
  267. notes-merge.h
  268. notes-utils.c
  269. notes-utils.h
  270. notes.c
  271. notes.h
  272. object-store.h
  273. object.c
  274. object.h
  275. oidmap.c
  276. oidmap.h
  277. oidset.c
  278. oidset.h
  279. pack-bitmap-write.c
  280. pack-bitmap.c
  281. pack-bitmap.h
  282. pack-check.c
  283. pack-objects.c
  284. pack-objects.h
  285. pack-revindex.c
  286. pack-revindex.h
  287. pack-write.c
  288. pack.h
  289. packfile.c
  290. packfile.h
  291. pager.c
  292. parse-options-cb.c
  293. parse-options.c
  294. parse-options.h
  295. patch-delta.c
  296. patch-ids.c
  297. patch-ids.h
  298. path.c
  299. path.h
  300. pathspec.c
  301. pathspec.h
  302. pkt-line.c
  303. pkt-line.h
  304. preload-index.c
  305. pretty.c
  306. pretty.h
  307. prio-queue.c
  308. prio-queue.h
  309. progress.c
  310. progress.h
  311. prompt.c
  312. prompt.h
  313. protocol.c
  314. protocol.h
  315. quote.c
  316. quote.h
  317. range-diff.c
  318. range-diff.h
  319. reachable.c
  320. reachable.h
  321. read-cache.c
  322. README.md
  323. rebase-interactive.c
  324. rebase-interactive.h
  325. ref-filter.c
  326. ref-filter.h
  327. reflog-walk.c
  328. reflog-walk.h
  329. refs.c
  330. refs.h
  331. refspec.c
  332. refspec.h
  333. remote-curl.c
  334. remote-testsvn.c
  335. remote.c
  336. remote.h
  337. replace-object.c
  338. replace-object.h
  339. repository.c
  340. repository.h
  341. rerere.c
  342. rerere.h
  343. resolve-undo.c
  344. resolve-undo.h
  345. revision.c
  346. revision.h
  347. run-command.c
  348. run-command.h
  349. send-pack.c
  350. send-pack.h
  351. sequencer.c
  352. sequencer.h
  353. serve.c
  354. serve.h
  355. server-info.c
  356. setup.c
  357. sh-i18n--envsubst.c
  358. sha1-array.c
  359. sha1-array.h
  360. sha1-file.c
  361. sha1-lookup.c
  362. sha1-lookup.h
  363. sha1-name.c
  364. sha1dc_git.c
  365. sha1dc_git.h
  366. shallow.c
  367. shell.c
  368. shortlog.h
  369. sideband.c
  370. sideband.h
  371. sigchain.c
  372. sigchain.h
  373. split-index.c
  374. split-index.h
  375. strbuf.c
  376. strbuf.h
  377. streaming.c
  378. streaming.h
  379. string-list.c
  380. string-list.h
  381. sub-process.c
  382. sub-process.h
  383. submodule-config.c
  384. submodule-config.h
  385. submodule.c
  386. submodule.h
  387. symlinks.c
  388. tag.c
  389. tag.h
  390. tar.h
  391. tempfile.c
  392. tempfile.h
  393. thread-utils.c
  394. thread-utils.h
  395. tmp-objdir.c
  396. tmp-objdir.h
  397. trace.c
  398. trace.h
  399. trailer.c
  400. trailer.h
  401. transport-helper.c
  402. transport-internal.h
  403. transport.c
  404. transport.h
  405. tree-diff.c
  406. tree-walk.c
  407. tree-walk.h
  408. tree.c
  409. tree.h
  410. unicode-width.h
  411. unimplemented.sh
  412. unix-socket.c
  413. unix-socket.h
  414. unpack-trees.c
  415. unpack-trees.h
  416. upload-pack.c
  417. upload-pack.h
  418. url.c
  419. url.h
  420. urlmatch.c
  421. urlmatch.h
  422. usage.c
  423. userdiff.c
  424. userdiff.h
  425. utf8.c
  426. utf8.h
  427. varint.c
  428. varint.h
  429. version.c
  430. version.h
  431. versioncmp.c
  432. walker.c
  433. walker.h
  434. wildmatch.c
  435. wildmatch.h
  436. worktree.c
  437. worktree.h
  438. wrap-for-bin.sh
  439. wrapper.c
  440. write-or-die.c
  441. ws.c
  442. wt-status.c
  443. wt-status.h
  444. xdiff-interface.c
  445. xdiff-interface.h
  446. zlib.c
README.md

Build Status

Git - fast, scalable, distributed revision control system

Git is a fast, scalable, distributed revision control system with an unusually rich command set that provides both high-level operations and full access to internals.

Git is an Open Source project covered by the GNU General Public License version 2 (some parts of it are under different licenses, compatible with the GPLv2). It was originally written by Linus Torvalds with help of a group of hackers around the net.

Please read the file INSTALL for installation instructions.

Many Git online resources are accessible from https://git-scm.com/ including full documentation and Git related tools.

See Documentation/gittutorial.txt to get started, then see Documentation/giteveryday.txt for a useful minimum set of commands, and Documentation/git-.txt for documentation of each command. If git has been correctly installed, then the tutorial can also be read with man gittutorial or git help tutorial, and the documentation of each command with man git-<commandname> or git help <commandname>.

CVS users may also want to read Documentation/gitcvs-migration.txt (man gitcvs-migration or git help cvs-migration if git is installed).

The user discussion and development of Git take place on the Git mailing list -- everyone is welcome to post bug reports, feature requests, comments and patches to git@vger.kernel.org (read Documentation/SubmittingPatches for instructions on patch submission). To subscribe to the list, send an email with just “subscribe git” in the body to majordomo@vger.kernel.org. The mailing list archives are available at https://public-inbox.org/git/, http://marc.info/?l=git and other archival sites.

Issues which are security relevant should be disclosed privately to the Git Security mailing list git-security@googlegroups.com.

The maintainer frequently sends the “What's cooking” reports that list the current status of various development topics to the mailing list. The discussion following them give a good reference for project status, development direction and remaining tasks.

The name “git” was given by Linus Torvalds when he wrote the very first version. He described the tool as “the stupid content tracker” and the name as (depending on your mood):

  • random three-letter combination that is pronounceable, and not actually used by any common UNIX command. The fact that it is a mispronunciation of “get” may or may not be relevant.
  • stupid. contemptible and despicable. simple. Take your pick from the dictionary of slang.
  • “global information tracker”: you're in a good mood, and it actually works for you. Angels sing, and a light suddenly fills the room.
  • “goddamn idiotic truckload of sh*t”: when it breaks