Relative Content

Tag Archive for pythonlangchainlarge-language-model

LangChain failure with long input texts

I am working with LangChain’s LLMChain to process large documents. I see quite some problems when the input text exceeds the token limit of the language model. For long input text, it has significant slowdowns and can even sometimes fail due to overflow of tokens.

Using LLM to convert a word document that contains tables to an excel spreadsheet (could be csv too)

I am successfully able to use ChatGPT and upload a word document that is a course script and then upload a spreadsheet with a few exact examples showing the LLM how I want the word doc script to be converted to the same format as the spreadsheet. Then I ask the LLM to do the same for similar page types in the word document without further examples and it uses its knowledge of the previous examples to extract out the appropriate data. This all works.