Evaluation of an LLM-based Chatbot
One relevant academic paper that evaluates an LLM-based chatbot is a study on AI chatbots for mental health support published in the Journal of Artificial Intelligence and Autonomous Intelligence .(DOI: 10.54364/JAIAI.2024.1105 ) The paper investigates the effectiveness of a chatbot designed as a mental health coach . The evaluation was conducted using a User Experience Questionnaire (UEQ) , which measures dimensions such as efficiency, dependability, stimulation, and novelty. The results show that users found the chatbot engaging and helpful, particularly in providing motivational and supportive responses. However, slightly lower scores in efficiency and dependability indicate limitations in maintaining consistent conversational flow . I selected this paper for three main reasons. First, it clearly involves a large language model-based chatbot in a specific context (mental health support) , which aligns with the assignment requirements. Second, the paper includes a substantive ev...