SmartRAG Enhacements

The issue aims to track the following enhancements or improvements made to smartRAG over the course of time.

Tasks

• Interactive visualization for Cluster embeddings and Specific QA points
• Formulation of Questions on Public adminstration document ( Open-data Handbuch )
• Opengpt-X-Teuken-Rag and Opengpt-X-Teuken-Instruct with RAG pipline:
  • Results are documents in word document
  • Obtained Prompt template from Dreseden team
• Deployment of new Opengpt-X-Teueken commercial and research model on DGX server
• Onboarding Comercial and Research models into AI-Builder
• SmartRAG republish and Rag-node delimeter issue
• RAG output speed improvisation
• Generate English responses from SmartRAG Teuken models:
  • Published separate models with simple prompt and EN tag on AI-Builder dev and DeployAI-Prod
  • Prompt Engineering for V0.4 models and hyperparameter tuning ( seed )
  • Prompt Engineering for V0.6 sigma model
  • Preparation for English webinar with Schengen Visa document
  • V0.6 and V0.4 EN models and pipeline published on AI-Builder dev and AI-LAB NRW
  • Documentation of Results
• Advance Prompt tuning feature:
  • Frontend webUI and backend scripting on rag node
  • Backend scripting on LLM's rag_chain node#
• Generate German response for SmartRAG Teuken V0.6 Sigma :
  • Prompt Engineering for V0.6 model with seed
  • Published model on AI-LAB NRW and AI-Builder dev
• Unifying rag_chain scripts to work with common LLMs ( uses langchain ) and opengptx llms.
  • Langchain or Prompting techniques?
• Adding more document feature
• parameter tunning feature
  • temp, topk, seed, ?

assigned to @spanneers

changed the description

changed the description

marked the checklist item Generate English responses from SmartRAG Teuken models: as completed

marked the checklist item Published separate models with simple prompt and EN tag on AI-Builder dev and DeployAI-Prod as completed

marked the checklist item Prompt Engineering for V0.4 models and hyperparameter tuning ( seed ) as completed

marked the checklist item Prompt Engineering for V0.6 sigma model as completed

marked the checklist item Preparation for English webinar with Schengen Visa document as completed

marked the checklist item V0.6 and V0.4 EN models and pipeline published on AI-Builder dev and AI-LAB NRW as completed

marked the checklist item Documentation of Results as completed

marked the checklist item Advance Prompt tuning feature: as completed

marked the checklist item Frontend webUI and backend scripting on rag node as completed

marked the checklist item Backend scripting on LLM's rag_chain node# as completed

marked the checklist item Generate German response for SmartRAG Teuken V0.6 Sigma : as completed

marked the checklist item Prompt Engineering for V0.6 model with seed as completed

marked the checklist item Published model on AI-LAB NRW and AI-Builder dev as completed

marked the checklist item Unifying rag_chain scripts to work with common LLMs ( uses langchain ) and opengptx llms. as completed

marked the checklist item Langchain or Prompting techniques? as completed

marked the checklist item Langchain or Prompting techniques? as incomplete

marked the checklist item Langchain or Prompting techniques? as completed

changed the description