Skip to content

Using Test Frameworks

Jump to bottom
Tom Clarke edited this page Feb 18, 2019 · 10 revisions

HOWTO

  • Create a new module TestStuff (replace Stuff with appropriate name)
  • Create a new dotnet core console project with TestStuff after all the emulator modules in compile order
  • Add Expecto, Expecto.FsCheck NuGet packages to your project
  • Add top-level Expecto test functions tagged with [<Tests>] to test any emulator project code, see below for details
  • Make program.fs main function run a top-level runTestsInAssembly function and then as usual wait for a console key before terminating.

EXAMPLES AND HELP

HLP2019-PARSING

  • fsharp-parse-examples.fs (module ParseHelpers) tokeniser and parser code
  • program.fs (top-level and Gen modules) testing code

The tests illustrated in program.fs uses Expecto and FsCheck in various ways:

  • [<TESTS>] introduces an Expecto Test value which contains one test, or a list of tests
  • All Expecto test values tagged with [<TESTS>] get run by runTestsInAssembly from main.
    • allTests - some Expecto unit tests
    • AllFsChecks - some demo propert tests
    • testCustomTokeniser - complex FsCheck test using custom random number generation

NOTE ON FSCHECK

  • FsCheck is very simple to use with built-in generators. You write a test function with parameters for the random data you want, this function gets reflected by FsCheck and the correct random data generators (one for each parameter) are used. There are losts of examples of this which you can copy.
  • FsCheck also have very sophisticated (and easy to use) ways to combine together generators or make your own generators. the documentation for this is here
  • Unfortunately there is almost no accessible example documentation on how to actually use the custom generators you write.

DETAILS

Current Status

The Visual2 code at the moment has a specific custom test framework that checks instruction functionality is compatible with Visual. This is useful, but not relevant to add-on functionality.

There is currently no integration of test frameworks into the Visual2 code, mainly because this is normally compiled under FABLE and the low friction testing is all under dotnet core which is not compatible with this. This project does not currently have test frameworks and integration testing that correctly runs emulator-level code through a set of easy to write tests. That is a (bad) lack, and therefore tests must be implemented as separate F# modules importing the emulator code and running it under an ad hoc dotnet core project.

Sorting out the Visual2 repo with a set of test modules (under a separate project) and yarn code to run the tests would be easy and desirable - it just has not been done yet. Once this is done, after the repo is made open source, free Travis or whatever continuous integration testing could be added. ATM this has little value, but soon it will be useful.

Emulator Testing

The Emulator project code will all compile and run under dotnet core and can initially be tested using Expecto / Fscheck which is very convenient.

Canopy is an alternate (and perhaps better) F# test framework, again for code that can be compiled under dotnet core.

Emulator FABLE/Javascript compiler compatibility

In addition to checking that new emulator F# works when compiled to dotnet core, it is helpful to test whether this code will compile when compiled with FABLE. FABLE implements core F# and some but not all of the many .Net libraries and methods available, see the compatibility guide. If you use random .Net functionality you may have no substitute in FABLE and therefore have to rewrite your code. For this check you don't need a test framework: just run your code under the full Visual2 yarn start compile and check there are no compile or bundle errors.

Emulator FABLE / F# incompatibility

FABLE is another compiler for F# and whether your code runs under .Net (normal compiler) or Javascript via FABLE should make no difference to operation. If a pure function passes tests on one platform it will work the same on the other. This is nearly always true, but not quite because of:

  • Small differences in how numbers are handled. See Numbers
  • Other differences (see guide

In practice you can ignore this issue when doing individual coding - it is unlikely to hit you.

Renderer Testing

The Renderer project code must run compiled by FABLE into Javascript since it has Javascript dependencies. It cannot be tested with Expecto and would need to use a Javascript-oriented test framework such as Jest or Mocha. See the FABLE FAQ. Integrating either these tests frameworks with Visual2 has not been attempted. It would be useful, but some work, to do. You should be familiar with Javascript to do so, because these frameworks use Javascript rather than F# techniques and although the translation from F# to Javascript is pretty good there will be some aspects of using them which are unfamiliar.

I don't expect Visual2 add-ons to have renderer-level tests incorporated now - though long-term this is clearly desirable.

Clone this wiki locally