Critical lack of transparency in training datasets for large language models
Researchers have developed a tool that allows artificial intelligence experts to more easily select data that best fits their models, potentially increasing model accuracy and reducing bias.

In training powerful language models, researchers rely on extensive data collections that encompass diverse information gathered from thousands of websites. However, as these datasets are combined and reused in different collections, crucial details about their origins are often lost or become unclear.

This lack of information not only raises legal and ethical concerns but can also negatively impact model performance. For example, if a dataset is misclassified, a researcher training a model for a specific task might unintentionally use data that is not suitable for that purpose.

Moreover, data from unknown sources may contain biases that lead to unfair predictions when the model is used in real-world situations, such as credit scoring or customer service interactions.

To increase data transparency, a team of multidisciplinary researchers from MIT and other institutions conducted a systematic review of over 1,800 textual datasets on popular websites. They found that more than 70 percent of these datasets lacked critical licensing information, while around 50 percent had errors in documentation.

Development of tools for greater data transparency
Researchers have developed a tool called Data Provenance Explorer that enables experts to easily review and assess the origin of datasets. This tool generates an overview of authors, sources, licenses, and permissible usage methods, which can significantly enhance the responsible use of AI technologies.

Data Provenance Explorer not only helps in selecting appropriate datasets for specific tasks but also allows users to download cards with detailed information about datasets, facilitating the understanding of risks and limitations associated with the data used.

Risks of bias and unethical use
The study also revealed that almost all dataset creators come from developed countries, which may limit the model’s ability to function correctly in different regions. For example, a dataset for Turkish developed by researchers in the US and China may not cover important cultural aspects, potentially affecting model accuracy in the Turkish context.

Researchers noted a significant increase in restrictions in datasets created in 2023 and 2024, indicating a growing concern among the academic community that their data might be misused for commercial purposes.

Challenges and future directions for research
To facilitate the collection of this information without manual review, Data Provenance Explorer offers users the ability to sort and filter datasets according to various criteria. This tool allows for the downloading of summarized dataset characteristics, marking a step forward in understanding the data used to train AI models.

In the future, researchers plan to expand their analysis to multimodal data, including videos and audio, and to explore how the terms of use on websites serving as data sources reflect on the use of datasets. They also intend to collaborate with regulators to address unique issues of copyright and ethics related to data finetuning.

MIT’s research highlights the need for data transparency, laying the foundation for a more ethical and legally compliant development of artificial intelligence in the future.