CI red on all PRs since 2025-12: three independent causes (tracking) #1135

Summary

Every PR Build since 2025-12-19 has failed, on all five matrix jobs, regardless of what the pull request changes. A one-line Dependabot version bump (#1132) fails identically to a large feature branch. Nobody can currently get a green check, and a genuine regression would be indistinguishable from the existing noise.

There are three independent causes, now tracked separately as sub-issues. Two of them are bugs in our own code that affect anyone using CathInstallation or EcodInstallation at runtime, not only CI.

Correction to the original diagnosis

This issue originally recorded cause 2 as "EcodInstallationTest.testVersion downloads 657 MB from a slow host", and described it as a judgement call rather than a bug. That was wrong.

The download completes. The file is read. The assertion then fails because ECOD changed its version comment from #ECOD version develop291 to # Version: v295, so getVersion() falls back to the literal string "latest" — exactly what the test asserts against. The column layout changed at the same time, from 15 columns to 23 and then 25, which means getAllDomains() on the current release returns an empty list, silently, for every user of the library.

The 657 MB is real and is still worth addressing — it is pulled once per matrix job, five times per pull request — but it is a cost, not the failure, and making the download faster would not have fixed anything. Full detail in #1139.

Evidence

Logs older than GitHub's 90-day retention have expired, so the cause is only confirmable for recent runs. The failure streak itself is visible in run metadata back to 2025-12-19; whether the earliest ones share this cause cannot now be checked.

Reproducing

mvn verify -pl biojava-integrationtest -Dtest='CathDomainTest+EcodInstallationTest#testVersion'

Keeping this from recurring

Both code causes are the same shape: an upstream service changed something, and we found out months later through a failure that named neither the service nor the change.

• pull_request.yml runs mvn verify with no module exclusion, so every pull request executes biojava-integrationtest, which by design depends on CATH, ECOD, RCSB and EBI being up and fast. master.yml already excludes it. Run integration tests nightly rather than on every pull request #1137 proposes applying the same exclusion to pull requests and running the full suite nightly — an upstream break then reaches us within a day, as a failure whose meaning is unambiguous, instead of as a red check on somebody else's pull request.
• Support the new ECOD distribution format #1141 adds offline unit tests pinning every ECOD file layout the project has ever read. Those run in under a second and would have caught the format change without downloading anything, which is the coverage that was actually missing.