WIP: fix various errors (placeholder to be split in multiple issues)

last a4f manual run logs contains some errors:

We need to go through them, identify each problem, decompose this issue in multiple issues, assign priorities and fix them

20211024_a4f_fossy_harvest_manual_run.log

@pmoser

• Moreover, it seems that the fossy command doesn't use variants, and this is wrong and leads to inconsistent results in the dashboard (more investigation required)
  • Bugfix: Fossy does not use variants sometimes (upload creates fossy.json files without variants)
  • ALIENSPDX and FOSSY files should have the group id in their filenames
  • Adapt pool methods to handle group ids in filenames
  • Adapt harvest that it chooses only current group id files that match with the group id inside .env

Related MR !39 (merged)

20211023_a4f_manual_run.log

@pianon

• wrong assumption in algorithm to hash variants: file names for the same content may change

example:

21-10-22 16:34:58 ERROR    aliens4friends.commands.command     | ModelError: can't merge tags and packages for recipe busybox-1.31.1-r0-e95eb6a2, because some fields mismatch, diff is: {'values_changed': {'root.recipe.source_files[51].relpath': {'new_value': 'oniro.cfg', 'old_value': 'allscenarios.cfg'}}}

• issue #59 (closed) Also Debian packages may have variants, but snapmatcher doesn't handle it (solved by !40 (merged))

example

21-10-22 16:59:25 ERROR    aliens4friends.commands.command     | AlienSnapMatcherError: checksum mismatch in debian package libcroco 0.6.13-1: debian control files are ['cd54520c6c3c42e53685dc706e28721742aa3fff libcroco_0.6.13.orig.tar.xz', '2e9123ced7b7077fb7d67e0ec762d19a980ab57b libcroco_0.6.13-1.debian.tar.xz'] while snap match srcfiles are ['2e9123ced7b7077fb7d67e0ec762d19a980ab57b libcroco_0.6.13-1.debian.tar.xz', 'cd54520c6c3c42e53685dc706e28721742aa3fff libcroco_0.6.13.orig.tar.xz', 'b94d7b1023e38ba2de8db46bd25a485c9ca52623 libcroco_0.6.13-1.debian.tar.xz']

• Missing support for lzip archives (fixed via b533906d)

21-10-22 21:00:19 ERROR    aliens4friends.commands.command     | CommandError: [ed-1.15-r0] Archive type unknown for files/ed-1.15.tar.lz.

• Conversion error by java spdx-tools

21-10-23 07:57:07 INFO     aliens4friends.commons.fossyupload  | [libcheck@0.14.0-r0-5153fb0b] Uploading alien SPDX
21-10-23 07:57:10 ERROR    aliens4friends.commands.command     | SPDXUtilsException: Error in converting spdxtv file into RDF with java spdx-tools. Command output is: WARNING: sun.reflect.Reflection.getCallerClass is not supported. This will impact performance.