Skip to content

Run tests in Gitlab #8656

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 130 commits into from
May 9, 2025
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
130 commits
Select commit Hold shift + click to select a range
10ecbb9
first pass checks
randomanderson Feb 27, 2025
26a7adb
add some debug
randomanderson Feb 27, 2025
f8448c9
Revert "add some debug"
randomanderson Feb 28, 2025
0b8643a
Normalize node index and node total
randomanderson Feb 28, 2025
65d41ac
fix normalization
randomanderson Feb 28, 2025
661749d
actually use the normalized values
randomanderson Mar 3, 2025
7e21e7b
Merge remote-tracking branch 'origin/master' into landerson/gitlab-ch…
randomanderson Mar 3, 2025
70c494d
first pass: migrate tests to Gitlab
randomanderson Mar 3, 2025
f3ca3e1
Merge branch 'master' into landerson/migrate-tests
randomanderson Mar 6, 2025
2ae3cf3
less scenarios for now
randomanderson Mar 6, 2025
c3ebdc0
bump memory and workers
randomanderson Mar 10, 2025
2d78199
actually use task partitions
randomanderson Mar 10, 2025
aba4f20
Add section start/end
randomanderson Mar 11, 2025
87bffbf
Allow individual tests to control gradle memory
randomanderson Mar 11, 2025
6b6d4e5
remove a little of the reportspam
randomanderson Mar 11, 2025
b2340e0
use concrete docker image tags
randomanderson Mar 12, 2025
42784ee
configure test containers to use mirror
randomanderson Mar 14, 2025
3bed067
Merge remote-tracking branch 'origin/master' into landerson/migrate-t…
randomanderson Mar 14, 2025
e781c3b
fix limit
randomanderson Mar 14, 2025
ffb90db
fix missing container versions
randomanderson Mar 17, 2025
c18b491
Better gitlab naming with smarter split
randomanderson Mar 17, 2025
a09c626
update caches with test dep cache, save test results
randomanderson Mar 17, 2025
97d5bc6
Cleanup repositories {} blocks
randomanderson Mar 19, 2025
2b37e3b
update proxy url to use internal routing
randomanderson Mar 19, 2025
7092cd0
no "s"
randomanderson Mar 19, 2025
391d0c3
allow insecure everywhere
randomanderson Mar 19, 2025
e225757
include apache with confluence
randomanderson Mar 19, 2025
340aaa1
Merge branch 'landerson/replace-maven-central' into landerson/migrate…
randomanderson Mar 20, 2025
7019854
remove intentional fail
randomanderson Mar 20, 2025
08a85b0
use gradlePluginCache, mavenCentral after pluginportal
randomanderson Mar 20, 2025
525031a
proxy mule as well
randomanderson Mar 20, 2025
7671cfb
Merge branch 'landerson/replace-maven-central' into landerson/migrate…
randomanderson Mar 21, 2025
0063e65
Merge remote-tracking branch 'origin/master' into landerson/migrate-t…
randomanderson Mar 31, 2025
7847b01
change BUILD_CACHE_TYPE to CACHE_TYPE since its used for both
randomanderson Mar 31, 2025
5085ef9
split cache like with circleci. Cache became too big
randomanderson Apr 1, 2025
0c400ca
Some caches are too big to be saved
randomanderson Apr 1, 2025
5a65b3a
test fixes
randomanderson Apr 1, 2025
7adf224
more test fixes
randomanderson Apr 1, 2025
c708e92
disable some tests for now
randomanderson Apr 1, 2025
57b86e9
more fixes
randomanderson Apr 2, 2025
b3b2a14
Use maven proxy with openliberty smoketests
randomanderson Apr 2, 2025
1e01c54
missed one more mongodb case
randomanderson Apr 2, 2025
de3fc5b
enable some more of the matrix
randomanderson Apr 2, 2025
38aa293
Fix lettuce tests
randomanderson Apr 2, 2025
9c8cd5a
Fix lettuce tests
randomanderson Apr 2, 2025
d7592b8
cleanup imports
randomanderson Apr 2, 2025
7dad598
create asyncCommands
randomanderson Apr 2, 2025
305b97e
codenarc
randomanderson Apr 2, 2025
5a4913b
Cleanup aggregator tests to try to reduce flakiness
randomanderson Apr 2, 2025
88da84a
Merge branch 'landerson/fix-lettuce-tests' into landerson/migrate-tests
randomanderson Apr 2, 2025
c5c158f
debug maven logs for now
randomanderson Apr 2, 2025
afdc208
Merge remote-tracking branch 'origin/master' into landerson/migrate-t…
randomanderson Apr 3, 2025
4dee866
try non-ssh url
randomanderson Apr 3, 2025
c23dcde
Revert "debug maven logs for now"
randomanderson Apr 3, 2025
6d6dc39
add some debug logging
randomanderson Apr 3, 2025
93fdb13
codenarc
randomanderson Apr 3, 2025
aa6a29d
try fixed value of parallelism
randomanderson Apr 3, 2025
58e4a01
enable debug logs for smoketest building only
randomanderson Apr 3, 2025
28cf042
limit number of reserved threads
randomanderson Apr 3, 2025
99593da
some debug for why the smoketest is not running
randomanderson Apr 3, 2025
c038257
more debug for gradle launcher failure
randomanderson Apr 3, 2025
6dd342a
try fixing graal test
randomanderson Apr 3, 2025
854ba80
try to force the logging
randomanderson Apr 3, 2025
5e0c67a
codenarc
randomanderson Apr 3, 2025
b7d5427
set available processors for jetty
randomanderson Apr 3, 2025
cc2af16
codenarc
randomanderson Apr 4, 2025
9d43248
bluntly bump max threads
randomanderson Apr 4, 2025
cc3744c
some more debug
randomanderson Apr 4, 2025
b3770bc
remove verbose logging
randomanderson Apr 4, 2025
5630d86
try memcached test with new location
randomanderson Apr 4, 2025
cdfb060
opt out of split by parallel
randomanderson Apr 4, 2025
b836b35
Try with 3 gradle workers
smola Apr 7, 2025
96ea71c
try decreasing reporting interval
randomanderson Apr 7, 2025
1594733
Refractor CrashTrackingSmokeTest to use OutputThreads
randomanderson Apr 7, 2025
b8adb1e
Simple forked memory test
randomanderson Apr 7, 2025
12dd18d
better ergonomics
randomanderson Apr 7, 2025
1290b77
try increasing timeout
randomanderson Apr 7, 2025
c9ba32a
extend agent test runner
randomanderson Apr 8, 2025
9973da8
don't override forkedMinHeapSize and forkedMaxHeapSize
randomanderson Apr 8, 2025
296bc5a
print out requests
randomanderson Apr 8, 2025
277c391
remove logging statement
randomanderson Apr 8, 2025
fc3a9ae
Better crash tracking
randomanderson Apr 8, 2025
3631181
remove intentionally failing test
randomanderson Apr 8, 2025
9f51ad4
Merge remote-tracking branch 'origin/master' into landerson/migrate-t…
randomanderson Apr 8, 2025
0975adc
use a test UDP server to record oom events
randomanderson Apr 9, 2025
561f192
Make crash tracking smoke test more resiliant
randomanderson Apr 9, 2025
551f88e
loop through messages until the correct event
randomanderson Apr 9, 2025
3df5bbf
some debug
randomanderson Apr 9, 2025
e4b8c9e
configurable dogstatsd port
randomanderson Apr 10, 2025
8794264
enable everything
randomanderson Apr 10, 2025
b5a1003
Merge branch 'landerson/dd-dogstatsd-port' into landerson/fix-crashtr…
randomanderson Apr 10, 2025
d6250d5
use sleep instead of park nanos
randomanderson Apr 10, 2025
0ecc8bb
save body data ahead of time
randomanderson Apr 10, 2025
38a35ad
retry once for system failures
randomanderson Apr 10, 2025
2e0d7ba
spotless
randomanderson Apr 10, 2025
ac863f0
cleanup
randomanderson Apr 11, 2025
0383ef9
Merge remote-tracking branch 'origin/master' into landerson/fix-crash…
randomanderson Apr 14, 2025
b378de5
Merge branch 'landerson/fix-crashtrackingsmoke' into landerson/migrat…
randomanderson Apr 14, 2025
da9a29c
spotless/merge issues
randomanderson Apr 14, 2025
0f26adf
Skip ExceptionHistogramTest on J9
randomanderson Apr 15, 2025
edeebc2
add a print statement around the body for debugging
randomanderson Apr 15, 2025
078a3c7
comment out all but 8, 17, and 21 for now
randomanderson Apr 15, 2025
a546529
fix spring 6 forwarding headers
randomanderson Apr 15, 2025
f16a6af
revert logging statements
randomanderson Apr 15, 2025
a1e0bb1
needs to be set in 3.1 for latestDepTest
randomanderson Apr 15, 2025
3f62a18
faster OOM
randomanderson Apr 16, 2025
014057c
property was set in incorrect location
randomanderson Apr 16, 2025
3c28009
ssi_smoke, flaky, and esoteric jvms on master only
randomanderson Apr 16, 2025
5c2c161
use normal runner when TestContainers isn't needed
randomanderson Apr 16, 2025
67a45fe
skip forwarding header filter here too
randomanderson Apr 16, 2025
96cfd44
tweaks
randomanderson Apr 16, 2025
69dd021
limit executor size and fix IPV4 assumptions
randomanderson Apr 17, 2025
0a21371
try more workers with less memory
randomanderson Apr 17, 2025
848a328
set max memory for all crashtracking tests
randomanderson Apr 17, 2025
196e96c
double retry
randomanderson Apr 17, 2025
4b14d00
scale up memory, less jobs
randomanderson Apr 17, 2025
f9d56d8
revert memory bump, increase forked tests
randomanderson Apr 17, 2025
b99ac8d
tweaks
randomanderson Apr 18, 2025
daaa1a5
Merge remote-tracking branch 'origin/master' into landerson/migrate-t…
randomanderson Apr 18, 2025
19ac6b4
add flag for nondefault jvms and flaky tests
randomanderson Apr 18, 2025
3fc0b97
Merge remote-tracking branch 'origin/master' into landerson/migrate-t…
randomanderson Apr 24, 2025
c3fd666
correct description
randomanderson May 6, 2025
6e5f1a9
Externalize runtime processors override
randomanderson May 6, 2025
d0b313d
Merge remote-tracking branch 'origin/master' into landerson/migrate-t…
randomanderson May 6, 2025
2b5d9ac
ExceptionHistogramTest exclusions already handled by master
randomanderson May 6, 2025
0df2344
increase spring native build memory
randomanderson May 7, 2025
8d74d0e
bump gradle build timeout
randomanderson May 7, 2025
e77ec82
Merge remote-tracking branch 'origin/master' into landerson/migrate-t…
randomanderson May 7, 2025
545cf9a
Merge remote-tracking branch 'origin/master' into landerson/migrate-t…
randomanderson May 9, 2025
99c67a3
bump graalvm compile memory
randomanderson May 9, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .gitlab-ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -112,6 +112,7 @@ default:
KUBERNETES_MEMORY_REQUEST: 8Gi
KUBERNETES_MEMORY_LIMIT: 8Gi
CACHE_TYPE: lib #default
RUNTIME_AVAILABLE_PROCESSORS_OVERRIDE: 4 # Runtime.getRuntime().availableProcessors() returns incorrect or very high values in Kubernetes
cache:
- key: '$CI_SERVER_VERSION-$CACHE_TYPE' # Dependencies cache. Reset the cache every time gitlab is upgraded. ~Every couple months
paths:
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -566,8 +566,7 @@ abstract class HttpServerTest<SERVER> extends WithHttpServer<SERVER> {
def request = request(SUCCESS, method, body).build()
if (testParallelRequest()) {
// Limit pool size. Too many threads overwhelm the server and starve the host
// availableProcessors() in CI can be very high and incorrect depending on JDK version
def poolSize = Math.min(4, Runtime.getRuntime().availableProcessors())
def poolSize = System.getenv().getOrDefault("RUNTIME_AVAILABLE_PROCESSORS_OVERRIDE", Runtime.getRuntime().availableProcessors())
def executor = Executors.newFixedThreadPool(poolSize)
def completionService = new ExecutorCompletionService(executor)
(1..count).each {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -4,8 +4,8 @@ runner {

// Runtime.getRuntime().availableProcessors() is used to scale the parallelism by default
// but it returns weird values in Gitlab/kubernetes so fix the parallelism to a specific value
if (System.getenv("GITLAB_CI") != null) {
fixed(4)
if (System.getenv("RUNTIME_AVAILABLE_PROCESSORS_OVERRIDE") != null) {
fixed(Integer.valueOf(System.getenv("RUNTIME_AVAILABLE_PROCESSORS_OVERRIDE")))
}
}
}