Firebird automated testing site

Alexey Kovyazin recently shared some interesting news:

Hi All,

FYI – http://firebirdtest.com
This website is a frontend of FBT + OLTP+automated scripts, supported by Pavel Zotov and some IBSurgeon employees.

It is running FBT tests on Windows server (8Gb, 4cores) for SuperServer and Classic for 3.0 and 4.0, and OLTP-EMUL for SuperServer 3.0/4.0.

Tests are triggered automatically, when new build appear at
http://web.firebirdsql.org/download/snapshot_builds/win/3.0/
http://web.firebirdsql.org/download/snapshot_builds/win/4.0/

As a result of test run, the report is generated.
Description
P – passed (ok)
Failed test due to failed assert is marked as F (failed), pink
Failed due to runtime marked as E (error), yellow.
Failed due to Firebird crash – F red/violet

For each error there is a hint (mouseover) with details of the error.

After click on the cell, the new page will be opened with more details and history of runs.

Detailed links:

0. Total results for each kind of outcome and server mode
1. ERRORS MAP: distribution of Firebird errors which occured during runs
2. FRESH FAILS: tests which failed for LAST execution only
3. FRESH ERRORS: tests which raised run-time error for LAST execution only
4. STILL FAILS: tests which fail for at least two last runs
5. Has fails: tests with at least one fail for previous runs
6. Has runtime errors: tests with at least one runtime error for previous runs
7. FRESH PASSED: successful tests with previous outcome = FAIL/ERROR/SKIPPED or absent
8. All tests: outcomes – all results for last 30 runs
9. All tests: A. Skipped: tests which was intentionally excluded from execution

Firebird OLTP Emul results
OLTP Emul simulates high load OLTP aplication, and uses many
sophisticated SQLs in multi-thread environment.

Our plan for the future:
1) Publish sources of automated scripts to GitHub (need to clean them a bit)
2) Perform Linux tests and include report results to the frontend
3) improve web representation – https certificate, design, etc
4) Migrate to the new FBT test framework when its adoption will be completed

We invite all interested to join the effort.
Pavel Zotov is a coordinator of the project – pz@ibase.ru.

Regards,
Alexey Kovyazin
IBSurgeon