Mit research on the generalization of large language models and the impact of human beliefs on their effectiveness in real-world situations

Researchers at MIT faced the challenge of evaluating large language models (LLMs) due to their broad application. Traditional approaches struggle to encompass all types of questions that models can answer. To address this problem, they focused on human perceptions and beliefs about these models' capabilities. A key concept in their research is the human generalization function, which models how people update their beliefs about LLMs after interacting with them.

For example, a student must decide whether a model will help compose a specific email, while a doctor must assess when a model will be useful in diagnosing patients. The researchers developed a framework for evaluating LLMs based on their alignment with human beliefs about performance on specific tasks.

Research on the human generalization function
As we communicate with others, we form beliefs about their knowledge. If a friend is prone to correcting grammar, we might assume they are good at sentence composition, even though we never asked them. Similarly, the researchers wanted to show that the same process occurs when forming beliefs about language models.

They defined the human generalization function as involving asking questions, observing responses, and inferring the person's or model's ability for similar questions. If someone sees that an LLM correctly answers questions about matrix inversion, they might assume it is also good at simple arithmetic. A model that does not align with this function may fail when used.

The researchers conducted a survey to measure how people generalize when interacting with LLMs and other people. They showed participants questions that people or LLMs answered correctly or incorrectly and asked them if they believed the person or LLM would answer a related question correctly. The results showed that participants were quite good at predicting human performance but were worse at predicting LLM performance.

Measuring misalignment
The research revealed that participants were more likely to update their beliefs about LLMs when models gave incorrect answers than when they answered correctly. They also believed that LLM performance on simple questions did not impact their performance on more complex questions. In situations where participants gave more weight to incorrect answers, simpler models outperformed larger models like GPT-4.

Further research and development
One possible explanation for why people are worse at generalizing for LLMs could be their novelty – people have much less experience interacting with LLMs than with other people. In the future, researchers want to conduct additional studies on how human beliefs about LLMs develop over time with increased interaction with the models. They also want to explore how human generalization could be incorporated into LLM development.

One of the key points of the research is the need for a better understanding and integration of human generalization into the development and evaluation of LLMs. The proposed framework takes into account human factors when applying general LLMs to improve their real-world performance and increase user trust.

The practical implications of this research are significant. If people do not have the right understanding of when LLMs will be accurate and when they will err, they are more likely to notice errors and possibly be discouraged from further use. This study emphasizes the importance of aligning models with human understanding of generalization. As increasingly complex language models are developed, it is necessary to integrate the human perspective into their development and evaluation.

Practical implications
This research is partially funded by the Harvard Data Science Initiative and the Center for Applied AI at the University of Chicago Booth School of Business. It is important to note that the researchers also want to use their dataset as a reference point for comparing LLM performance against the human generalization function, which could help improve model performance in real-world situations.

Additionally, the researchers plan further studies to understand how human beliefs about LLMs develop over time through interaction with models. They want to explore how human generalization can be integrated into LLM development to improve performance and increase user trust. The practical implications of this research are far-reaching, especially in the context of LLM applications in various industries, where understanding and user trust are key to successful technology adoption.

One of the key points of the research is the need for a better understanding and integration of human generalization into the development and evaluation of LLMs. The proposed framework takes into account human factors when applying general LLMs to improve their real-world performance and increase user trust. It is important to emphasize that the practical implications of this research are significant. If people do not have the right understanding of when LLMs will be accurate and when they will err, they are more likely to notice errors and possibly be discouraged from further use.

This study emphasizes the importance of aligning models with human understanding of generalization. As increasingly complex language models are developed, it is necessary to integrate the human perspective into their development and evaluation. This research is partially funded by the Harvard Data Science Initiative and the Center for Applied AI at the University of Chicago Booth School of Business. It is important to note that the researchers also want to use their dataset as a reference point for comparing LLM performance against the human generalization function, which could help improve model performance in real-world situations.

The practical implications of this research are far-reaching, especially in the context of LLM applications in various industries, where understanding and user trust are key to successful technology adoption. One of the key points of the research is the need for a better understanding and integration of human generalization into the development and evaluation of LLMs. The proposed framework takes into account human factors when applying general LLMs to improve their real-world performance and increase user trust.

Source: Massachusetts Institute of Technology