Skip to content

Commit bd8ef97

Browse files
OcramiusOcramius
committed
Rewrote acceptance test suite to use behat/behat as a main runner
While `psalm/codeception-psalm-module` is a good idea in theory, in practice, Gherkin scenario steps as a composer dependency are really (REALLY) hard to distribute and maintain together with a test suite. This move replaces the external dependency with a local one, with the test runner mostly staying out of the way, and test logic all implemented in the `Context` class we wrote ourselves. The main aim is to improve atomicity of changes on the test suite itself. Ref: psalm/codeception-psalm-module#54
1 parent 409c587 commit bd8ef97

File tree

9 files changed

+1844
-1918
lines changed

9 files changed

+1844
-1918
lines changed

‎.github/workflows/ci.yml

Lines changed: 28 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -88,6 +88,30 @@ jobs:
8888
- name: Run PHPCodeSniffer
8989
run: vendor/bin/phpcs --report=checkstyle -q --parallel=1 | cs2pr
9090

91+
test-static-analysis:
92+
name: Run static analysis on the tests themselves
93+
runs-on: ubuntu-latest
94+
95+
steps:
96+
- name: Checkout
97+
uses: actions/checkout@v3
98+
99+
- name: Set up PHP
100+
uses: shivammathur/setup-php@v2
101+
with:
102+
php-version: 8.2
103+
tools: composer:v2
104+
coverage: none
105+
extensions: intl, mbstring, bcmath, sodium
106+
env:
107+
fail-fast: true
108+
109+
- name: Install composer dependencies (high deps)
110+
run: cd tools/behat && composer install
111+
112+
- name: Static analysis
113+
run: cd tools/behat && vendor/bin/psalm
114+
91115
tests:
92116
name: Test on ${{matrix.php}} - ${{matrix.deps}} deps
93117
runs-on: ubuntu-20.04
@@ -136,7 +160,7 @@ jobs:
136160
COMPOSER_ROOT_VERSION: dev-master
137161

138162
- name: Install composer tool dependencies (high deps)
139-
run: cd tools/codeception && composer update --prefer-dist --no-interaction
163+
run: cd tools/behat && composer update --prefer-dist --no-interaction
140164
if: ${{matrix.deps == 'high'}}
141165
env:
142166
COMPOSER_ROOT_VERSION: dev-master
@@ -148,7 +172,7 @@ jobs:
148172
COMPOSER_ROOT_VERSION: dev-master
149173

150174
- name: Install composer tool dependencies (low deps)
151-
run: cd tools/codeception && composer update --prefer-dist --no-interaction --prefer-stable --prefer-lowest
175+
run: cd tools/behat && composer update --prefer-dist --no-interaction --prefer-stable --prefer-lowest
152176
if: ${{matrix.deps == 'low'}}
153177
env:
154178
COMPOSER_ROOT_VERSION: dev-master
@@ -160,7 +184,7 @@ jobs:
160184
COMPOSER_ROOT_VERSION: dev-master
161185

162186
- name: Install composer tool dependencies (stable deps)
163-
run: cd tools/codeception && composer update --prefer-dist --no-interaction --prefer-stable
187+
run: cd tools/behat && composer update --prefer-dist --no-interaction --prefer-stable
164188
if: ${{matrix.deps == 'stable'}}
165189
env:
166190
COMPOSER_ROOT_VERSION: dev-master
@@ -169,4 +193,4 @@ jobs:
169193
run: vendor/bin/psalm --version
170194

171195
- name: Run tests
172-
run: tools/codeception/vendor/bin/codecept run -v
196+
run: cd tools/behat && vendor/bin/behat run -vvv

‎tests/acceptance/TestCase.feature

Lines changed: 41 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -646,7 +646,6 @@ Feature: TestCase
646646
}
647647
}
648648
"""
649-
And I have Psalm newer than "4.99" (because of "sealed shapes")
650649
When I run Psalm
651650
Then I see no errors
652651

@@ -1055,7 +1054,8 @@ Feature: TestCase
10551054
When I run Psalm
10561055
Then I see these errors
10571056
| Type | Message |
1058-
| InvalidDocblock | %@psalm-ignore% |
1057+
| InvalidDocblock | Unrecognised annotation @psalm-ignore |
1058+
| InvalidDocblock | Unrecognised annotation @psalm-ignore in docblock for NS\\MyTestCase |
10591059

10601060
Scenario: Invalid psalm annotation on an before initializer does not crash psalm
10611061
Given I have the following code
@@ -1072,7 +1072,7 @@ Feature: TestCase
10721072
When I run Psalm
10731073
Then I see these errors
10741074
| Type | Message |
1075-
| InvalidDocblock | %@psalm-rm-Rf-slash% |
1075+
| InvalidDocblock | Unrecognised annotation @psalm-rm-Rf-slash |
10761076

10771077
Scenario: Invalid psalm annotation on a test does not crash psalm
10781078
Given I have the following code
@@ -1089,7 +1089,7 @@ Feature: TestCase
10891089
When I run Psalm
10901090
Then I see these errors
10911091
| Type | Message |
1092-
| InvalidDocblock | %@psalm-force-push-master% |
1092+
| InvalidDocblock | Unrecognised annotation @psalm-force-push-master |
10931093

10941094
Scenario: Missing param type on a test with tuple data provider does not crash psalm
10951095
Given I have the following code
@@ -1249,10 +1249,43 @@ Feature: TestCase
12491249
}
12501250
}
12511251
"""
1252-
And I have the following classmap
1253-
| Class | File |
1254-
| NS\MyTestCase | test.php |
1255-
| NS\External | ext.php |
1252+
And I have the following code in "autoload.php"
1253+
"""
1254+
<?php
1255+
spl_autoload_register(function(string $class) {
1256+
/** @var ?array<string,string> $classes */
1257+
static $classes = null;
1258+
1259+
if (null === $classes) {
1260+
$classes = [
1261+
'NS\\MyTestCase' => 'test.php',
1262+
'NS\\External' => 'ext.php',
1263+
];
1264+
}
1265+
1266+
if (array_key_exists($class, $classes)) {
1267+
/** @psalm-suppress UnresolvableInclude */
1268+
include $classes[$class];
1269+
}
1270+
});
1271+
"""
1272+
And I have the following config
1273+
"""
1274+
<?xml version="1.0"?>
1275+
<psalm errorLevel="1" findUnusedCode="false" autoloader="autoload.php">
1276+
<projectFiles>
1277+
<directory name="."/>
1278+
<ignoreFiles> <directory name="../../vendor"/> </ignoreFiles>
1279+
</projectFiles>
1280+
<plugins>
1281+
<pluginClass class="Psalm\PhpUnitPlugin\Plugin"/>
1282+
</plugins>
1283+
<issueHandlers>
1284+
<MissingClassConstType errorLevel="suppress" />
1285+
<DeprecatedMethod errorLevel="suppress" />
1286+
</issueHandlers>
1287+
</psalm>
1288+
"""
12561289
When I run Psalm on "test.php"
12571290
Then I see no errors
12581291

‎tools/behat/behat.yml

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,7 @@
1+
default:
2+
suites:
3+
default:
4+
contexts:
5+
- PsalmTest\PhpUnitPlugin\Context
6+
paths:
7+
- '../../tests/acceptance/'

‎tools/behat/composer.json

Lines changed: 23 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,23 @@
1+
{
2+
"description": "Psalm plugin for PHPUnit - tools/behat locked dependencies - used to run tests only!",
3+
"autoload": {
4+
"psr-4": {
5+
"PsalmTest\\PhpUnitPlugin\\": "src"
6+
}
7+
},
8+
"require": {
9+
"php": "~8.1.0 || ~8.2.0 || ~8.3.0 || ~8.4.0",
10+
"azjezz/psl": "^2.9.1 || ^3.2.0",
11+
"behat/behat": "^3.18.1",
12+
"php-standard-library/psalm-plugin": "^2.3",
13+
"vimeo/psalm": "^6"
14+
},
15+
"scripts": {
16+
"test": "codecept run -v"
17+
},
18+
"config": {
19+
"platform": {
20+
"php": "8.1.99"
21+
}
22+
}
23+
}

0 commit comments

Comments
 (0)