Ticket #5 (new enhancement)

Opened 1 year ago

run tests, store results in the db, dump overview to stdout

Reported by: akhavr Assigned to: somebody
Priority: major Milestone:
Component: component1 Version:
Keywords: Cc:

Description

Invocation: shell ci-runtests [-k] <tests.d dir> <connect string...>

Event flow:

  1. System users the <connect string...> to connect to the live VPS, fails if it can not.
  2. System executes every script in <tests.d dir> in alpabetical order, collecting the results.
  3. If test script fails and no -k switch (same as in make -k) specified, system stops test run.
  4. System stores the stdout and strerr of test run in the database and dumps the overview (list of scripts run and their result) to stdout for further consumption (e.g. for ci-runtests ... | mail email@report invocation}}}

Timing: less than 1m for simple 'ls' test

Tests:

  1. VPS is not alive.
  2. No tests
  3. Test fails and no -k switch specified
  4. Test fail and -k switch specified
  5. Success test