CASSANALYTICS144: Split testing pipelines out by jmckenzie-dev · Pull Request #198 · apache/cassandra-analytics

jmckenzie-dev · 2026-04-17T19:18:41Z

We need to de-parameterize testing, split out to a representative pre-commit smoke test, and ideally randomize our test ordering reproducibly based on the SHA of the commit we're testing. Oh, and probably tune our resource utilization for integration and unit tests.

skoppu22 · 2026-04-21T15:24:08Z

+          # then shard across runners via round-robin on the shuffled order.
+          CLASSNAMES=$(find . -name '*Test.java' | cut -c 3- | sed 's@/@.@g' | sed 's/.\{5\}$//' \
+            | python3 -c "import random,sys; lines=sys.stdin.read().splitlines(); random.seed('$GITHUB_SHA'); random.shuffle(lines); print('\n'.join(lines))" \
+            | awk 'NR % 5 == ${{ matrix.job_index }}')


[minor] can we have matrix.job_total in-place of hardcoded 5 ?

Yep - promoted that up.

skoppu22 · 2026-04-21T15:30:06Z

LGTM, can you please share Circle CI pipeline link

Patch by Josh McKenzie, reviewed by TBD for CASSANALYTICS-144 Breaks out tests to parameterize based on specific cassandra versions. Gradle files will be coupled with CassandraVersion.java but both are commented to indicate their interdependency with one another. Also adds some convenience configurations in intellij config so you can select a specific cassandra version to run tests against at runtime or just choose "test" or "testSequential" to run against all supported major cassandra versions.

…A of commit

…elism

MultipleTokens test classes use assumeThat(>= 4.1) in @BeforeAll to skip on Cassandra 4.0. When CI runs these individually via --tests, the skipped @BeforeAll leaves zero discovered tests, causing Gradle to fail with "No tests found." Move failOnNoMatchingTests(false) out of the skipContainerTest guard so it applies unconditionally.

…tBase.java" This reverts commit 8cf8b70.

jmckenzie-dev · 2026-04-21T16:38:47Z

Here's the circle link from before the above matrix job total promotion: https://app.circleci.com/pipelines/gh/jmckenzie-dev/cassandra-analytics/51/details?job=caad7d43-6be5-43e3-9228-d1691bc2f18a&buildNumber=386&jobType=build&workflowId=9a16c32f-f7eb-4976-97a6-696300e7708f

jmckenzie-dev · 2026-04-21T18:41:41Z

There are some CI failures @skoppu22 but best I can tell, they're related to individual tests that are flaky not to the structure of the pipelines. @yifan-c - tapping you for 2nd reviewer role.

jmckenzie-dev · 2026-04-21T18:41:58Z

Here's latest circle pipeline: link

jmckenzie-dev · 2026-04-21T20:52:46Z

k; CI looks much cleaner than it has been lately. @yifan-c - waiting on you now.

yifan-c · 2026-04-21T21:55:02Z

-            INTEGRATION_MAX_PARALLEL_FORKS: 1
-            INTEGRATION_MAX_HEAP_SIZE: "1500M"
-            CORE_MAX_PARALLEL_FORKS: 2
+            INTEGRATION_MAX_PARALLEL_FORKS: 2


~~INTEGRATION_MAX_PARALLEL_FORKS has always been 1. Please revert the change. The in-jvm-dtest cannot run in parallel.~~

If updating to 2 works, I am good with 2.

Happy to revert, but also uncertain why the claim they cannot be run in parallel as they're running and passing in both circle and github. There is some flakiness in both env right now but the test failures look to be pre-existing failures or timeouts afaict.

Either way, I'll drop this to 1, bump the heap to 3072 for all integration, and let's see how it runs w/that combo.

I don't think the original statement is true, we've run integration tests in parallel in the past, not sure if anything changed recently that prevents us from running in parallel

yifan-c · 2026-04-22T03:13:40Z

+        config: ['s2.13-c5.0.5', 's2.12-c4.1.4', 's2.12-c4.0.17']
+        job_index: [0, 1, 2, 3, 4]
+        job_total: [5]
+        include:
+          - config: 's2.13-c5.0.5'
+            scala: '2.13'
+            cassandra: '5.0.5'
+          - config: 's2.12-c4.1.4'
+            scala: '2.12'
+            cassandra: '4.1.4'
+          - config: 's2.12-c4.0.17'
+            scala: '2.12'
+            cassandra: '4.0.17'


This is the same as the previous matrix with exclusion. Either one is fine.

I don't understand what you mean here. Are you saying that the previous structure here:

matrix: scala: [ '2.12', '2.13' ] cassandra: [ '4.0.17', '4.1.4', '5.0.5' ] job_index: [ 0, 1, 2, 3, 4 ] job_total: [ 5 ] exclude: - scala: "2.12" cassandra: "5.0.5" - scala: "2.12" cassandra: "4.1.4" - scala: "2.13" cassandra: "4.0.17"

before this patch is identical? I agree, but I find the include approach to be easier to reason about than the exclude. i.e. easier to reason about just what we're running directly than to think about the entire spectrum of combinations and carving out parts.
wdyt?

yifan-c · 2026-04-22T05:11:05Z

+//   - cassandraVersionEnumMap values must match the implemented_versions default
+//   - cassandraFullVersionMap values must match the supported_versions default
+ext.cassandraVersionEnumMap = ["4.0": "FOURZERO", "4.1": "FOURONE", "5.0": "FIVEZERO"]
+ext.cassandraFullVersionMap = ["4.0": "4.0.17", "4.1": "4.1.4", "5.0": "5.0.5"]


can you add a comment to reference to scripts/build-dtest-jars.sh, the script that builds the cassandra dtest jars?

Ugh. Added comments in all 3 places (build.gradle, build-dtest-jars.sh, CassandraVersion.java to all reference each other. I'll take a note for a follow up JIRA to have a single source of truth for this that we pull from at build time; this is way too brittle.

…ew feedback

jmckenzie-dev · 2026-04-22T15:16:45Z

@yifan-c - Circle CI green, and the failure in github is the reusable cluster problems I'm tackling on CASSANALYTICS-146. maxParallelForks at 1 seems to have gotten in and out in comparable time to 2 in Circle; we can always create a follow up ticket to see if we can push to parallel at 2 w/3G heap each, or even 3.5g each and leave 1G for system.

We about ready to merge this one?

jmckenzie-dev · 2026-04-22T15:17:01Z

Circle pipeline: https://app.circleci.com/pipelines/gh/jmckenzie-dev/cassandra-analytics/55/details

skoppu22 reviewed Apr 21, 2026

View reviewed changes

Comment thread .circleci/config.yml

skoppu22 reviewed Apr 21, 2026

View reviewed changes

Comment thread ...ndra-five-zero-bridge/src/main/java/org/apache/cassandra/bridge/CdcBridgeImplementation.java

skoppu22 reviewed Apr 21, 2026

View reviewed changes

jmckenzie-dev added 17 commits April 21, 2026 12:33

Have cdc tasks inherit base JVM params; move to DiskAccessMode mmap

02a158f

slim down to 3 'diagonal' matrixed pipelines for precommit

d29de8d

Move to 3 full combos across all GA; randomize test order based on SH…

a2c846e

…A of commit

Simplify gh job config

2b03133

Comment on 4.1 exclusion; default 2G/3 fork

ec55508

make CDC resource alloc in CI pluggable as well

13313c2

tidy up cdc build.gradle a touch to make heap decl adjacent to parall…

a8f1007

…elism

Add bridge jar mapping when shared across versions

3cddeed

Bump integration to 3G heap

0063349

Always report artifacts in circle even in failure case

6cb8d35

Make 4.0 and 4.1 both use same nb format

1156550

null check guard around s3Mock in SharedClusterSparkIntegrationTetsBase

4b42400

remove full validation requirement in CL < ALL for JoiningTestBase.java

a04afd1

Revert "remove full validation requirement in CL < ALL for JoiningTes…

2d88357

…tBase.java" This reverts commit 8cf8b70.

Make matrix.job_total a top level param

1f6275a

jmckenzie-dev force-pushed the ca144_split_pipelines branch from a3c15e3 to 1f6275a Compare April 21, 2026 16:38

jmckenzie-dev requested a review from yifan-c April 21, 2026 18:41

yifan-c reviewed Apr 22, 2026

View reviewed changes

Revert all integration fork max to 1; change heap to 3072; other revi…

48469b1

…ew feedback

yifan-c approved these changes Apr 22, 2026

View reviewed changes

jmckenzie-dev merged commit daa08d4 into apache:trunk Apr 22, 2026
18 of 19 checks passed

Conversation

jmckenzie-dev commented Apr 17, 2026

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

skoppu22 commented Apr 21, 2026

Uh oh!

jmckenzie-dev commented Apr 21, 2026

Uh oh!

jmckenzie-dev commented Apr 21, 2026

Uh oh!

jmckenzie-dev commented Apr 21, 2026

Uh oh!

jmckenzie-dev commented Apr 21, 2026

Uh oh!

yifan-c Apr 21, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

jmckenzie-dev commented Apr 22, 2026

Uh oh!

jmckenzie-dev commented Apr 22, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

yifan-c Apr 21, 2026 •

edited

Loading