Evaluate SQLite backend
The Greycat backend suffers noticeable slowdowns when using derived attributes due to the Lucene integration, and the OrientDB backend has always been the slowest of the three (even when skipping their graph layer and working directly with documents). The hawk-sqlite has some experiments done with SQLite as a backend, testing different database structures. In general:
-
LOAD CSV
has the best load performance by far, but it would require backend-specific updaters in order to be used. Failing that, avoiding autocommit and using prepared statements seems to be reasonably fast. - For the node indices, using one table per index and indexing key, name, and node ID seems to work well enough.
It's definitely worth trying, at least - SQLite is designed as an embedded database, after all, and it could keep things much simpler than other backends.