Skip to content

Commit e17668e

Browse files
committed
Add additional notes about writing good tests.
Refs: #439
1 parent a6520db commit e17668e

File tree

1 file changed

+15
-1
lines changed

1 file changed

+15
-1
lines changed

CONTRIBUTING.md

Lines changed: 15 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -57,7 +57,21 @@ The definition of "trivial" is intentionally slightly ambiguous, and intended to
5757
An example of a trivial change is fixing a typo in the README, or bumping a version of a dependency used by the continuous integration suite.
5858
If another contributor takes issue with a change merged in this fashion, simply commenting politely that they have concerns about the change (either in an issue or directly) is the right remedy.
5959

60-
## Test-Specific Changes
60+
## Writing Good Tests
61+
62+
Be familiar with the test structure and assumptions documented in the [README](README.md).
63+
64+
Test cases should include both valid and invalid instances which exercise the test case schema whenever possible.
65+
Exceptions of course include schemas where only one result is ever possible (such as the `false` schema, or ones using keywords which only produce annotations).
66+
67+
Schemas should be *minimal*, by which we mean that they should contain only those keywords which are being tested by the specific test case, and should not contain complex values when simpler ones would do.
68+
The same applies to instances -- prefer simpler instances to more complex ones, and when testing string instances, consider using ones which are self-descriptive whenever it aids readability.
69+
70+
Comments can and should be used to explain tests which are unclear or complex.
71+
The `comment` field is present both for test cases and individual tests for this purpose.
72+
Links to the relevant specification sections are also encouraged, though they can be tedious to maintain from one version to the next.
73+
74+
When adding test cases, they should be added to all past (and future) versions of the specification which they apply to, potentially with minor modifications (e.g. changing `$id` to `id` or accounting for `$ref` not allowing siblings on older drafts).
6175

6276
Changing the schema used in a particular test case should be done with extra caution, though it is not formally discouraged if the change simplifies the schema.
6377
Contributors should not generally append *additional* behavior to existing test case schemas, unless doing so has specific justification.

0 commit comments

Comments
 (0)