Support adapter that runs as a server (client-server mode)
In the first draft of the TCK harness, the code assumes that the adapter is a CLI that is invoked per test (CLI mode). The input is passed over stdin and the result is passed over stdout. Then the CLI exits. While this mode may work for some implementations, it isn't going to scale for others. Some implementations will be based on a language runtime that has a cold start time (such as Java), Eclipse Austen being an example. If there are over a 1,000 tests (a conservative estimate), and the start time of the CLI is 1 second, the tests will take over 15 minutes to run. That's not reasonable. This mode also limits parallelization since systems may restrict the number of processes that run at once. And the process thrashing is just largely unnecessary.
If we allow the adapter to run as a server instead, we can avoid these limitations. We can assume that this server is a web server, and that information is exchanged over HTTP. Almost every language runtime provides an HTTP server out of the box or has libraries that are readily available. And Node.js has been able to communicate over HTTP from its inception.
When the TCK starts, it will call start
on the adapter. If that method returns a PID, it will keep that process running. For each test, the TCK will post the test to the server over HTTP and retrieve the response. When the TCK ends, it will call stop
on the adapter so it can stop the server.
If the start
method of the adapter returns falsy (or perhaps the string cli
), the TCK will run the adapter command instead of posting to the server, just as it does today.