[{"data":1,"prerenderedAt":286},["ShallowReactive",2],{"doc-progmasoft-roadmap":3},{"id":4,"title":5,"body":6,"description":16,"extension":277,"meta":278,"navigation":279,"path":280,"seo":281,"slug":282,"source":283,"stem":284,"__hash__":285},"docs\u002Fdocs\u002Fprogmasoft-roadmap.md","Catch3 roadmap",{"type":7,"value":8,"toc":267},"minimark",[9,13,17,51,54,166,171,174,206,210,216,220,223,227,230,234,237,241,244,264],[10,11,5],"h1",{"id":12},"catch3-roadmap",[14,15,16],"p",{},"Catch3 grows from a Catch2-compatible test framework into a testing toolkit:\nunit testing, property testing, mocking, coverage, mutation testing, test\ngeneration and test selection, driven by a runner of its own. Two rules hold\nfor every step:",[18,19,20,45],"ul",{},[21,22,23,27,28,32,33,36,37,40,41,44],"li",{},[24,25,26],"strong",{},"Catch2's test-writing API stays compatible."," The macros and the ",[29,30,31],"code",{},"Catch","\nnamespace, with its matchers, generators and listeners, keep working\nunchanged. Everything else is Catch3's own: the runner replaced Catch2's\nsession, command line and reporters, the headers are under\n",[29,34,35],{},"Progmasoft\u002FCatch3\u002F"," and the build names are ",[29,38,39],{},"Catch3",". New capabilities\nlive in the ",[29,42,43],{},"Progmasoft::Catch3"," namespace.",[21,46,47,50],{},[24,48,49],{},"Nothing is claimed before it is tested."," A feature is listed as done when\nits tests run in CI on Windows, Linux and macOS.",[14,52,53],{},"The steps are ordered by what each one builds on.",[55,56,57,73],"table",{},[58,59,60],"thead",{},[61,62,63,67,70],"tr",{},[64,65,66],"th",{},"Step",[64,68,69],{},"Capability",[64,71,72],{},"State",[74,75,76,93,106,119,135,146,156],"tbody",{},[61,77,78,82,85],{},[79,80,81],"td",{},"1",[79,83,84],{},"Native property testing: choice-sequence shrinking, combinators, state machines, statistics, targeted search, example database, differential and metamorphic oracles",[79,86,87,88],{},"done; see ",[89,90,92],"a",{"href":91},"\u002Fdocs\u002Fprogmasoft-properties\u002F","Native property testing",[61,94,95,98,101],{},[79,96,97],{},"2",[79,99,100],{},"The Catch3 runner",[79,102,87,103],{},[89,104,100],{"href":105},"\u002Fdocs\u002Fprogmasoft-runner\u002F",[61,107,108,111,114],{},[79,109,110],{},"3",[79,112,113],{},"Mocking",[79,115,87,116],{},[89,117,113],{"href":118},"\u002Fdocs\u002Fprogmasoft-mocking\u002F",[61,120,121,124,127],{},[79,122,123],{},"4",[79,125,126],{},"Per-test code coverage",[79,128,129,130,134],{},"in progress: Linux and macOS, with Clang and GCC; see ",[89,131,133],{"href":132},"\u002Fdocs\u002Fprogmasoft-runner\u002F#per-test-coverage","Per-test coverage",". Windows with clang-cl is next",[61,136,137,140,143],{},[79,138,139],{},"5",[79,141,142],{},"Change-impact test selection and test prioritization",[79,144,145],{},"planned",[61,147,148,151,154],{},[79,149,150],{},"6",[79,152,153],{},"Mutation testing",[79,155,145],{},[61,157,158,161,164],{},[79,159,160],{},"7",[79,162,163],{},"Coverage-, mutation- and history-guided test generation",[79,165,145],{},[167,168,170],"h2",{"id":169},"_2-the-catch3-runner","2. The Catch3 runner",[14,172,173],{},"Catch2's runner executed the tests of one binary in one process. The\nfeatures that follow need more: they run each test in isolation, under\ncoverage or mutation, and decide which tests to run and in which order. The\nCatch3 runner replaced Catch2's in every test binary:",[18,175,176,179,182,185,188,191,194,197,200,203],{},[21,177,178],{},"tests run in parallel worker processes that take the next test when they\nare free; a crash or hang ends one test, not the run;",[21,180,181],{},"failures are retried alone, which tells deterministic, flaky and\norder-dependent failures apart, and the test that leaves state behind is\nfound by bisection;",[21,183,184],{},"a history of verdicts and durations drives quarantine, adaptive time\nlimits, predictive ordering, incremental runs, replay and analytics;",[21,186,187],{},"tests are selected by change, by past failure or by prediction, and\nsharded by a stable hash or by duration;",[21,189,190],{},"tags declare dependencies, shared resources, serial tests, time limits and\nretries;",[21,192,193],{},"every failure gets a category, a signature and an analysis;",[21,195,196],{},"reports are console, JUnit XML with flaky and rerun elements, JSON and a\ntrace; the discovery listing has a fixed, versioned format;",[21,198,199],{},"reports of other frameworks are imported, so their tests get the same\nhistory, flaky detection, quarantine and analytics;",[21,201,202],{},"every test gets a temporary directory of its own;",[21,204,205],{},"its command line is parsed by a hand-written, typed parser, like the other\nProgmasoft tools.",[167,207,209],{"id":208},"_3-mocking","3. Mocking",[14,211,212,213,215],{},"Mocks report through Catch2's assertion path: expectations with call counts,\nargument matchers built on Catch2's matchers, sequences, and actions, for\nvirtual interfaces and for free functions behind a seam. Unexpected calls and\nunmet expectations fail the test case that owns the mock. See\n",[89,214,113],{"href":118},".",[167,217,219],{"id":218},"_4-per-test-code-coverage","4. Per-test code coverage",[14,221,222],{},"The runner runs each test case of an instrumented binary on its own and keeps\none coverage profile per test: LLVM source-based coverage with Clang and GCC's\ngcov format with GCC. The result is a map from each test to the lines and\nbranches it reaches, plus the usual combined report.",[167,224,226],{"id":225},"_5-change-impact-selection-and-prioritization","5. Change-impact selection and prioritization",[14,228,229],{},"With the per-test map, the runner selects the tests whose covered lines a\nchange touches, taking the change from Git, and runs those first. Ordering\nuses failure history, duration and coverage of the change, so that a failing\nrun fails early. A full run remains one option away, and the selection is\nreported, never silent.",[167,231,233],{"id":232},"_6-mutation-testing","6. Mutation testing",[14,235,236],{},"The runner makes small changes to the code under test, such as a flipped\ncomparison or a removed call, rebuilds, and runs the tests that cover the\nchanged line. A mutation that no test detects points at a missing assertion.\nThe per-test coverage map keeps this affordable: only covering tests run for\neach mutant.",[167,238,240],{"id":239},"_7-test-generation","7. Test generation",[14,242,243],{},"The property engine already records choice sequences. Test generation feeds\nthose sequences from three sources:",[18,245,246,252,258],{},[21,247,248,251],{},[24,249,250],{},"coverage-guided:"," choice sequences that reach new coverage are kept and\nmutated further, as a fuzzer keeps inputs, so property tests explore code\nthat random inputs miss;",[21,253,254,257],{},[24,255,256],{},"mutation-guided:"," inputs are searched for that tell a surviving mutant\nfrom the original program, and become new regression examples;",[21,259,260,263],{},[24,261,262],{},"incremental:"," examples found in earlier runs, kept in the example\ndatabase, seed the next run, so the search continues where it stopped.",[14,265,266],{},"Generated examples are written as ordinary Catch2 test cases or stored replay\ntokens, so they run without the generator.",{"title":268,"searchDepth":269,"depth":269,"links":270},"",2,[271,272,273,274,275,276],{"id":169,"depth":269,"text":170},{"id":208,"depth":269,"text":209},{"id":218,"depth":269,"text":219},{"id":225,"depth":269,"text":226},{"id":232,"depth":269,"text":233},{"id":239,"depth":269,"text":240},"md",{},true,"\u002Fdocs\u002Fprogmasoft-roadmap",{"title":5,"description":16},"progmasoft-roadmap","docs\u002Fprogmasoft-roadmap.md","docs\u002Fprogmasoft-roadmap","CAiRbHv-5cEtCS-B-xD_3acL5l0xr6BuLpqoTp8If8A",1794961594512]