sparql_test does not cover components

[gouttegd]

The standard Makefile includes a sparql_test target (executed as part of the test target) which runs a bunch of SPARQL-based checks using robot verify:

sparql_test: $(EDIT_PREPROCESSED) | $(REPORTDIR)
        $(ROBOT) verify -i $(EDIT_PREPROCESSED) --queries $(SPARQL_VALIDATION_QUERIES) -O $(REPORTDIR)

Those checks only operate on the contents of the (preprocessed) -edit file; they do not cover any imported ontology (actual ODK imports and components). This may not be explicitly documented in ROBOT’s manual, but a brief look at the code shows this is the normal behaviour of the verify command, and that this behaviour is not configurable:

// Load into dataset without imports
Dataset dataset = QueryOperation.loadOntologyAsDataset(state.getOntology(), false);

I don’t think it is correct for the sparql_test to cover only the contents of the -edit file. Of course it is fine to exclude imports (in the ODK sense) from the SPARQL tests (we may not want those tests to fail because of something coming from an external ontology), but components (and I include under that umbrella the pattern-derived definitions.owl file) are merely parts of the “local” ontology that just happen to be in different files -- I see no reason for excluding them from the SPARQL tests.

Therefore, and since verify will not follow OWL imports declarations, I believe the sparql_test should be executed over the $(SRCMERGED) file (in which components, that is, all files listed as $(OTHER_SRC), have been explicitly merged in) rather than $(EDIT_PREPROCESSED).

[matentzn]

You are right. The default could be changed to SRC_MERGED

[gouttegd]

Note that this may wreak havoc in existing repos when such a change lands in a released version…

Currently, in all existing ODK setups, the contents of components is not checked, so any kind of violations may have crept up in components without ever being caught. All of a sudden, components will be checked, and ontologies that previously passed the sparql_test check may now very well fail it.

[matentzn]

I thought of that as well, but there is no great way around it.

1. Any QC we have configured is in fact designed to apply to components
2. People can manually shut the QC off if they so want to rather than fix stuff

I am fine with the fallout

[gouttegd]

I thought of that as well, but there is no great way around it.

Agree. If any pipeline fails because of that change, it will ultimately be because the components contain (possibly have always contained) errors. The new ODK will merely reveal those problems, not cause them.