Conversational memory for LLM pipelines

Aim is to create a conversational AI system that leverages the GraphCypherQAChain or RAGChain to query a database and also retains memory of past interactions. This way, your AI system can maintain context across multiple turns in a conversation, similar to a chatbot with memory.

Aproaches

1. With langchain library: The langchain provides Conversation Chain and Conversation Memory libraries with which we can connect it to the GraphCypherQA and RAGChain. There are different types of Conversationmemory.

• Integrating Memory with the Chain:
  ConversationMemory is used to store previous conversations. LangChain allows you to inject memory into chains so that they can remember past interactions.
• Creating a Chain with Memory:
  This involves combining memory with the chain, so that when you call chain.run(question), it not only processes the new question but also references past conversations.
• Querying Neo4j:
  The Chain will continue querying database as needed. If the question requires information from the graph, it will generate Cypher queries and fetch the relevant data.

2. Without langchain library : ( TBD )

Types of Memory:

S.no	Types	Pros and cons
1.	ConversationBufferMemory	Storing everything gives the LLM the maximum amount of information But more tokens mean slowing response times and higher costs. Long conversations cannot be remembered as we hit the LLM token limit
2.	ConversationSummaryMemory	Shortens the number of tokens for long conversations. Can result in higher token usage for smaller conversations. Also requires token usage for the summarization LLM; this increases costs
3.	ConversationBufferWindowMemory	keep a given number of past interactions before “forgetting” them. Although this method isn’t suitable for remembering distant interactions
4.	ConversationSummaryBufferMemory	Summarizer means we can remember distant interactions and Buffer prevents us from missing information from the most recent interactions. Summarizer increases token count for shorter conversations.
5.	ConversationKGMemory	Stores conversation as entities and relation in triple format but consumes large amount of tokens.

Tasks

• Read articles about ConversationMemory and its types
• Integration of chains with memory i.e. ConversationMemory
  • GraphCypherQAChain with ConversationMemory and querying Neo4j
  • RAGChain with ConversationMemory and querying FAISS
• Design a chatbot UI

Conclusion

This setup will allow you to build a conversational AI that remembers past questions and queries database to retrieve information when needed. You could further refine the memory to summarize, structure, or filter out unnecessary details for better performance over longer conversations.

Reference

1. https://www.pinecone.io/learn/series/langchain/langchain-conversational-memory/
2. https://dev.to/aarushikansal/personal-movie-recommendation-agent-with-gpt4-neo4j-3aib
3. https://github.com/langchain-ai/langchain/issues/12635
4. https://python.langchain.com/v0.1/docs/modules/memory/conversational_customization/