You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
After integrating the https://github.com/openaire/iis/blob/marekhorst_1298_refactor_iis_content_importer branch with #1361 pull request we end up with one eu.dnetlib.iis.wf.importer.content.WorkflowTest#testImportContentUrlWorkflowWithHiveBasedAggregationSubsystemAsBackend() test as disabled because it can't be run on CI test cluster due to missing Hive service. It works on the main IIS CDH5 cluster though.
Apart from enabling the test once the hive service is installed on CI test cluster one another issue should be addressed related to the workflow.xml definition (link currently pointing to the branch but it will be merged with IIS master branch eventually) which should be modified, along with the build system configuration in iis-wf/pom.xml file, to avoid setting jdbc-url and hive_metastore_uris to explicit values bound to a particular environment.
The oozie workflow, by design, should remain environment-agnostic and it should be possible to rely on specific placeholders predefined for a given environment in ~/.iis/integration-test.properties file stored in an environment just as it is already done for various env specific properties such as sparkExecutorMemory or oozieServiceLoc.
Currently defined values which are currently pointing to the main hadoop cluster and which should be addressed:
After integrating the https://github.com/openaire/iis/blob/marekhorst_1298_refactor_iis_content_importer branch with #1361 pull request we end up with one
eu.dnetlib.iis.wf.importer.content.WorkflowTest#testImportContentUrlWorkflowWithHiveBasedAggregationSubsystemAsBackend()
test as disabled because it can't be run on CI test cluster due to missing Hive service. It works on the main IIS CDH5 cluster though.Apart from enabling the test once the hive service is installed on CI test cluster one another issue should be addressed related to the
workflow.xml
definition (link currently pointing to the branch but it will be merged with IIS master branch eventually) which should be modified, along with the build system configuration iniis-wf/pom.xml
file, to avoid settingjdbc-url
andhive_metastore_uris
to explicit values bound to a particular environment.The oozie workflow, by design, should remain environment-agnostic and it should be possible to rely on specific placeholders predefined for a given environment in
~/.iis/integration-test.properties
file stored in an environment just as it is already done for various env specific properties such assparkExecutorMemory
oroozieServiceLoc
.Currently defined values which are currently pointing to the main hadoop cluster and which should be addressed:
jdbc-url
:jdbc:hive2://iis-cdh5-test-m3.ocean.icm.edu.pl:10000
hive_metastore_uris
:thrift://iis-cdh5-test-m3.ocean.icm.edu.pl:9083
which should be placed among the connection properties dedicated for each individual properly at: https://git.icm.edu.pl/openaire/iis-deployment/-/tree/master/connection-properties
The text was updated successfully, but these errors were encountered: