Let SmartRAG use GPU if available
The SmartRAG node could detect if a GPU is available and accelerate creation of the embeddings see new ai-runner-gpu
Tasks
-
The nodes are able to detect GPU and assign resources to them -
speed enhancement of rag-node in SmartRAG - The .to(device) searches for CUDA and move some processing layers to GPU.
- Even when we do not use the .to(device) syntax, they run in CPU but still speed up --> this might be due to the powerful hosts, faster RAM etc. ( needs investigation )
Edited by Sangamithra Panneer Selvam