I am close to deploying my QLORA-tuned LLM in a specific domain. The fine-tuned model demonstrates performance superior to GPT-4 in generating answers for customers on specific tasks. However, I’ve noticed an issue: the fine-tuned LLM sometimes fails to provide coherent answers to general questions, and more importantly, it fails to provide safe answers.
I’m aware of the solution that involves incorporating general-domain data during fine-tuning. However, I’m quite skeptical about this approach since it may require a significant amount of data and resources.
I’m more optimistic about using a properly instruction-tuned model to interact with customers. If the customer’s intention is to perform a specific task, we could re-route their request to the fine-tuned LLM.
I believe this strategy aligns well with the agentic view for developing LLM-based applications.
I would appreciate your professional opinion on this topic.