Skip to content

CathInstallation downloads over http, which now redirects to https, and the redirect body is cached as classification data #1138

Description

@aalhossary

Sub-issue of #1135, cause 1 of 3.

The bug

CathInstallation.java:57:

public static final String CATH_DOWNLOAD_URL = "http://download.cathdb.info/cath/releases/";

That host now redirects:

$ curl -sI http://download.cathdb.info/cath/releases/
HTTP/1.1 301 Moved Permanently
Location: https://download.cathdb.info/cath/releases/

$ curl -sI https://download.cathdb.info/cath/releases/
HTTP/1.1 200 OK

HttpURLConnection follows redirects within a protocol but deliberately will not follow one that switches http to https. downloadFileFromRemote read the response with a bare remoteURL.openStream() and no status check, so the body of the 301 was written into the local cache as though it were classification data. Parsing then produced no domains, and the first symptom appeared much later and far from the cause:

CathDomainTest.test:40  NullPointer
  Cannot invoke "org.biojava.nbio.structure.cath.CathDomain.toCanonical()" because "domain" is null

Why the redirect matters more than the URL

Switching to https alone turns the test green, and the upstream change is out of our hands. Accepting a redirect body as data is not. Any download that reads a stream without checking the status can cache an error page or a redirect body that looks like a valid file forever after — and because it is cached, it keeps failing after the service recovers.

Fix

Fixed by commit ed36d3a in #1133, which changes the URL to https and routes the download through the status-checking FileDownloadUtils.downloadFileWithValidation. #1133 closes the same hole in DownloadChemCompProvider, which is not broken today only because the RCSB ligand endpoint still serves plain http.

Verified in CI: before that commit all five matrix jobs failed on CathDomainTest; after it, none do (run 31916362199).

mvn verify -pl biojava-integrationtest -Dtest=CathDomainTest

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions