Switch from MongoDB to MariaDB+Solr
As discussed in an internal issue (#3581), we will shift away from MongoDB to support a mixed solution of MariaDB+Solr. This presents a more stable option considering the current data structure and product support available for this project.
Below I have included current implementation notes for this issue:
- DAOFilters will remain and be converted to create PreparedStatements in one call rather than 2
- MongoQuery will become RDBMSQuery which will compile prepared statements
- New Solr DAO will be added, which will trigger when a text search is used.
- We should use a service to combine the 2 result sets.
- The combination of datasets will be done post-cache. This allows for multiple searches to be done with the same filter sets or keyword searches much quicker.
- New sync action needs to be run on a schedule that updates Solr cores for all listings. This should run every hour or so to keep results as fresh as possible.
- We should manage the Cron timestamp in application.properties to allow for quicker changes of run time
- Should be smart enough to check when the last update was and compare it to the Solr doc when present.
- We should track the doctype in the Solr document in case we ever want to add a text search on other document types.
- Should be made in an extensible way so that we can easily harness a document type to be indexed with little work or fuss (error reports is a good candidate)
Additionally, some investigation will be done to whether this is an appropriate use case for Hibernate. It looks promising as an engine to manage data models with its management of nested data. The querying on the nested data may be complex, however, and needs further investigation. The fallback strategy is as follows:
- Codec, in general, will remain but should be collapsed into a single class rather than spread across up to 3 separate classes.
- DAO will be rewritten to make use of JDBC Agroal data connections + prepared statements (++security)
- DAO will need to collapse rows into a single object, as nested data will create multiple rows for the parent objects.