added readme for BANG

development/bang/README.md

+---
+title: How to setup and run the Binary Analysis Next Generation framework (BANG) on binaries in the Oniro IP Compliance Toolchain
+author: Martin Rabanser <martin.rabanser@rmb-consulting.tech>
+date: 2022-08-30
+lastmod: 2022-08-30
+SPDX-License-Identifier: CC-BY-SA-4.0 AND LicenseRef-Trademark-Notice
+SPDX-FileCopyrightText: Martin Rabanser <martin.rabanser@rmb-consulting.tech>
+---
+
+# Introduction
+This howto describes how to install and getting started with the Binary Analysis Next Generation (BANG) framework to analyze binaries in the Oniro IP Compliance Toolchain.
+
+# Setup
+
+* Clone the BANG project from Github 
+
+```
+      cd ~
+      git clone https://github.com/armijnhemel/binaryanalysis-ng.git
+```
+* Read INSTALL.ubuntu and as root user execute the `apt-get` command to add the recommended packages
+
+* Install the Nix package manager as user `builder` and answer `y` to all questions
+  
+  More background info here https://nixos.org/download.html#nix-install-linux
+
+```
+      su - builder
+      sh <(curl -L https://nixos.org/nix/install) --daemon
+
+      # When completed exit and re-enter the shell
+      
+      nix-shell -p nix-info --run "nix-info -m"
+```
+
+# Usage 
+
+For using BANG without running into dependency hell it is recommended to use the Nix <https://nixos.org/> package manager. There are three different phases in BANG and so there are three different environments. Read the `README-nix.md` to understand the different environments.
+
+  1. The firmware unpacking phase using the `shell.nix` environment
+  1. The knowledgebase maintenance phase using the `maintenance.nix` environment
+  1. The analysis phase using the `analysis.nix` environment
+
+## Phase 1: Unpacking
+
+### Unpacking prerequisites
+  
+  * Prepare directories
+
+```
+      mkdir -p ~/bang-scan-results
+```
+
+* To invoke the unpack environment using the `shell.nix` execute the following
+
+```
+      cd ~/binaryanalysis-ng/
+      nix-shell shell.nix
+```
+
+* To do the unpacking a firmware the Kaitai Struct compiler is needed, which comes preinstalled with the `shell.nix` environment. So before executing BANG, the kaitai-struct-compiler must generate the python modules from the parser definition. 
+
+Execute the following
+
+```
+      cd ~/binaryanalysis-ng/src
+      make clean
+      make
+```
+
+Further information here https://github.com/armijnhemel/binaryanalysis-ng/blob/master/doc/kaitai-struct.md
+
+  > Kaitai Struct is "a declarative language used to describe various binary data structures, laid out in files or in memory".
+  
+  > https://github.com/kaitai-io/kaitai_struct http://kaitai.io/
+  
+  > Basically it is a language to describe binary formats using a description file, which can then be used to generate parsers for various languages.
+
+
+### Unpacking execution
+
+* (Optional) Modify the bang configuration file 
+
+```
+    vim ~/binaryanalysis-ng/src/bang.config
+```
+
+* Start the unpack tool inside the nix-shell environment
+
+```
+      cd ~/binaryanalysis-ng/
+      nix-shell shell.nix
+      
+      python3 ~/binaryanalysis-ng/src/bang-scanner -c ~/binaryanalysis-ng/src/bang.config -t ~/bang-scan-results -u ~/bang-scan-results -f <PATH_TO_IMAGES>/<YOUR_IMAGE.xyz>
+      
+      # Example 1: 
+      # nohup python3 ~/binaryanalysis-ng/src/bang-scanner -c ~/binaryanalysis-ng/src/bang.config -t ~/bang-scan-results -u ~/bang-scan-results -f /build/oniro/kirkstone-v2.0.0-alpha2-198-g92756ed/build-linux-qemux86-64/tmp/deploy/images/qemux86-64/oniro-image-base-qemux86-64-20220822164154.rootfs.wic.gz &
+      # tail -f nohup.out
+      
+      # Example 2: 
+      # nohup python3 ~/binaryanalysis-ng/src/bang-scanner -c ~/binaryanalysis-ng/src/bang.config -t ~/bang-scan-results -u ~/bang-scan-results -f ~/test-images/oniro-image-base-tests-qemux86.rootfs.wic.gz &
+      # tail -f nohup.out
+```
+
+### Unpacking result verification
+
+* See the result by simply reading the `report.txt` or by opening a more detailed pickle file
+
+```
+      python -m pickle bang.pickle
+```
+
+* Start bangshell to see some statistics on the result
+
+```
+      cd ~/binaryanalysis-ng/
+      nix-shell shell.nix
+      ./src/bangshell
+      
+      # Inside bangshell
+      
+      load <PATH TO SCAN RESULT>
+      [OUTPUT]
+      
+      ls 
+      [OUTPUT]
+      
+      summary
+      [OUTPUT]
+```
+
+
+### Unpacking troubleshooting
+
+* JSON Output doesn't work
+
+```
+    [nix-shell:~/binaryanalysis-ng]$ python3 src/bang-scanner -c src/bang.config -f /build/oniro/kirkstone-v2.0.0-alpha2-198-g92756ed/build-linux-qemux86-64/tmp/deploy/images/qemux86-64/modules--5.10.61+git0+3b283fa8d4_452ea6a15e-r0-qemux86-64-20220822164154.tgz -t ~/bang-scan-results -u ~/bang-scan-results
+    Traceback (most recent call last):
+      File "/home/builder/binaryanalysis-ng/src/bang-scanner", line 351, in <module>
+        main(sys.argv)
+      File "/home/builder/binaryanalysis-ng/src/bang-scanner", line 319, in main
+        JsonReporter(jsonfile).report(scanresult)
+      File "/home/builder/binaryanalysis-ng/src/JsonReporter.py", line 43, in report
+        for a, h in fileresult.get_hashresult().items():
+    AttributeError: 'dict' object has no attribute 'get_hashresult'
+```
+
+## Phase 2: Maintenance
+There are different maintenance scripts for operating with BANG results, each one in a dedicated subdirectory. 
+
+### Maintenance prerequisite
+To run them, the `maintenance.nix` environment has to be invoked as it comes with some preinstalled 3rd party maintenance tools. 
+
+```
+    nix-shell src/maintenance.nix
+```
+
+### Maintenance scripts
+Look into the single directories to understand how to use the specific tool.
+For some of them further downloads are needed as adviced in the according README files in their specific folder.
+
+* crawlers - crawlers for various datasources
+* cvehound - scripts to process `cvehound` results
+* database - scripts to process downloaded data and fill a PostgreSQL database
+* meilisearch - scripts to process data and fill a Meilisearch database (use with care)
+* proximity - scripts to process data and to populate Vantage Point Trees for a
+  proximity matcher webservice
+* semgrep_rules - rules for `semgrep`
+* sql - database schemas for PostgreSQL
+* yara - scripts to generate YARA rules
+* yara\_rules - handcrafted YARA rules (licenses, forges, copyright)
+
+
+## Phase 3: Analysis
+There are three analysis scripts, marked as "proof of concept".
+
+- apk_identifier using `APKiD`
+- cve_finder using `cve-bin-tool`
+- nsrl_lookup connecting to a database
+
+### Analysis prerequisites
+To run them, the `analysis.nix` environment has to be invoked as it comes with the preinstalled 3rd party analysis tools.
+
+```
+    nix-shell src/analysis.nix
+```
+
+###  Analysis script `apk_identifier`
+This script searches the BANG results for any APK (Android packages) and runs `APKiD` (<https://github.com/rednaga/APKiD>) on every APK that is found. For this the script needs access to the original scan data.
+
+The script `apk_identifier.py`, the configuration `apk-config.yaml` and more information can be found in the directory `src/apk_identifier`.
+
+Example call:
+
+```
+    python3 src/apk_identifier/apk_identifier.py -c src/apk_identifier/apk-config.yaml -r <PATH_TO_BANG_SCAN_RESULT>
+    
+    # Example
+    # python3 src/apk_identifier/apk_identifier.py -c src/apk_identifier/apk-config.yaml -r ~/bang-scan-results/bang-scan-4ap_7q2o/
+``` 
+
+### Analysis script `cve_finder` 
+This script searches the BANG results and runs ´cve-bin-tool` (<https://github.com/intel/cve-bin-tool>) on every ELF file that was detected.
+
+The script `cve_finder.py`, the configuration `cve-config.yaml` and more information can be found in the directory `src/cve`.
+
+Example call:
+
+```    
+    python3 src/cve/cve_finder.py -c src/cve/cve-config.yaml -r <PATH_TO_BANG_SCAN_RESULT>
+
+    # Example
+    # python3 src/cve/cve_finder.py -c src/cve/cve-config.yaml -r ~/bang-scan-results/bang-scan-4ap_7q2o/
+```
+
+### Analysis script `nsrl_lookup`
+This script searches the BANG results and runs a query on the NSRL database for every file that is found.
+
+The script and configuration can be found in the directory `src/nsrl`.
+