Database setup for VCS tests.
The setup is consistent with tools-nuxeo-ftest (org.nuxeo:nuxeo-ftest, nuxeo-ftest.xml) and uses the same environment variables and Maven profiles.
Automatically setup environment variables for testing purpose and persist those variables for using them as well in unit, functional and integration tests.
Here is an overview of the execution timeline with some keypoints.
During the Maven initialize phase, if executed from root org.nuxeo:nuxeo-ecm, the testing parameters are evaluated according to the conventions:
- with the
customdbMaven profile activated- ant-assembly-maven-plugin executes
setup-vcsinstructions fromintegration/vcstests.xmlvcs-tests-setup
Generates the effective database parameters and stores them in${user.home}/nuxeo-test-vcs.propertiesfor later use: typically the functional tests driven by org.nuxeo:nuxeo-ftestprepare-db
Downloads the driver, checks the connection, drops and creates the testing database
- properties-maven-plugin executes
read-project-propertiesinstructions.
Reads the parameters from${user.home}/nuxeo-test-vcs.propertiesand shares them with Java as System properties.
- ant-assembly-maven-plugin executes
During the Maven test phase, regardless where it is executed from:
- maven-surefire-plugin reads the Java System properties
org.nuxeo.ecm.core.test.StorageConfiguration#init()interprets those properties and makes them available to the Nuxeo Testing Framework.- with
<someDB>Maven profile activated
values:default,pgsql,mssql,oracle11g,oracle12c,oracle18c,mysql,mongodb,mariadb...- some database specific properties are optionally set (database kind, default port...)
- the Nuxeo Testing Framework instantiates the right
org.nuxeo.ecm.core.storage.sql.DatabaseHelper
Note that variables in descriptor are resolved when loaded in [DefaultRuntimeContext](org.nuxeo.runtime.model.impl.DefaultRuntimeContext#createRegistrationInfo(org.nuxeo.runtime.model.StreamRef)
Since surefire 2.18 empty properties are propagated which is not expected by some tests that
want to resolve later undefined variables, to do this empty system properties that starts with nuxeo.test.
are removed by the test RuntimeHarness.
The vcstests.xml inherits the properties initialization from nuxeo-ftest.xml then it copies the properties to nuxeo.test.\*. For instance:
nuxeo.db.host=>nuxeo.test.vcs.servernuxeo.db.port=>nuxeo.test.vcs.portnuxeo.mongodb.server=>nuxeo.test.mongodb.server
See https://github.com/nuxeo/tools-nuxeo-ftest#java-parameters for the conventions implemented in org.nuxeo:nuxeo-ftest. Roughly here is the logic:
- the database profile is put in
maven.dbprofileproperty - for a given list of database variables (
HOST,PORT,NAME,USER...),
if the global variableenv.NX_DB_@{dbvar}is not set,
then lookup forenv.NX_${maven.dbprofile}_DB_@{dbvar}specific variable instead.
If no variable is set, then defaults to various behaviors (hardcoded default value, dynamic temporary value, error...).
Here are some extracts from a typical execution of ondemand-testandpush job against MySQL database.
[INFO] --- ant-assembly-maven-plugin:2.1.0:build (setup-vcs) @ nuxeo-ecm ---
[INFO] Active Maven profiles:
qa (source: external)
qa (source: org.nuxeo:nuxeo-ecm:11.1-SNAPSHOT)
customdb (source: org.nuxeo:nuxeo-ecm:11.1-SNAPSHOT)
mysql (source: org.nuxeo:nuxeo-ecm:11.1-SNAPSHOT)
vcs-tests._initdb:
[INFO] [echo] Using NX_MYSQL_DB_PORT fallback for undefined NX_DB_PORT
[INFO] [echo] Using NX_MYSQL_DB_NAME fallback for undefined NX_DB_NAME
[INFO] [echo] Using NX_MYSQL_DB_ADMINNAME fallback for undefined NX_DB_ADMINNAME
[INFO] [echo] Using NX_MYSQL_DB_ADMINUSER fallback for undefined NX_DB_ADMINUSER
[INFO] [echo] Using NX_MYSQL_DB_ADMINPASS fallback for undefined NX_DB_ADMINPASS
vcs-tests.vcs-tests-setup:
[INFO] [propertyfile] Updating property file: /opt/jenkins/nuxeo-test-vcs.properties
vcs-tests.prepare-db:
[INFO] [echo] Prepare MySQL...
vcs-tests.setup-mysql-driver:
vcs-tests.mysql-dbdrop:
[INFO] [sql] Executing commands
[INFO] [sql] 4 of 4 SQL statements executed successfully
vcs-tests.mysql-dbcreate:
[INFO] [sql] Executing commands
[INFO] [sql] 4 of 4 SQL statements executed successfully
[INFO] --- properties-maven-plugin:1.0-alpha-2:read-project-properties (default) @ nuxeo-ecm ---
[INFO] --- maven-surefire-plugin:2.17:test (default-test) @ nuxeo-core-test ---
[INFO] Surefire report directory: (...)
[INFO] Using configured provider org.apache.maven.surefire.junitcore.JUnitCoreProvider
[INFO] parallel='none', perCoreThreadCount=true, threadCount=0, useUnlimitedThreads=false, threadCountSuites=0, threadCountClasses=0, threadCountMethods=0, parallelOptimized=true
-------------------------------------------------------
T E S T S
-------------------------------------------------------
StorageConfiguration: Deploying JDBC using DatabaseMySQL
StorageConfiguration: Deploying a VCS repository
(...)
- Sample Environment setup
export NX_DB_NAME=testvcs
export NX_DB_USER=testvcs
export NX_DB_PASS=testvcs
export NX_MSSQL_DB_HOST=sqlserver.mydomain.com
export NX_MSSQL_DB_PORT=1433
export NX_MSSQL_DB_ADMINNAME=master
export NX_MSSQL_DB_ADMINUSER=sa
export NX_MSSQL_DB_ADMINPASS=supersecretpassw0rd
-
Run your tests with both the
customdbprofile and the profile corresponding to your database:mvn test -Pcustomdb,mssql
Those files are used by the continuous integration. The Groovy files are Jenkins Pipeline scripts.
If you need to know the default environment variables set, then you can issue one of:
docker run --rm dockerpriv.nuxeo.com:443/nuxeo/jenkins-slave /sbin/my_init -- su - jenkins -c "env|sort"
export TESTS_COMMAND="env|sort"
docker-compose -f integration/Jenkinsfiles/docker-compose-mongodb-3.4.yml up --no-color --build --abort-on-container-exit tests db