OpenAI Announces Data Partnership Program for Public and Private Training Datasets

On November 10, OpenAI announced that it will collaborate with organizations to generate public and private datasets for training AI models, a data partnership aimed at enabling more organizations to help shape the future of AI and benefit from more useful models.

According to its blog, OpenAI said: “In order to ultimately make AI safer and beneficial to all of humanity, we want AI models to have deep understanding of all topics, industries, cultures, and languages, which requires training datasets that are as broad as possible.”

As part of the data partnership program, OpenAI says it will collect large‑scale datasets that “reflect human society” and are currently difficult to access online. While the company plans to work across multiple modalities, including images, audio, and video, it is especially seeking data that express human intent across different languages, topics, and formats (such as long‑form writing or dialogue).

OpenAI states that, if necessary, it will work with organizations and use optical character recognition and automatic speech recognition tools to digitize training data and remove sensitive or personal information when needed.

OpenAI hopes to create two types of datasets: an open‑source dataset that anyone can use for AI model training, and a private dataset for training proprietary AI models.

OpenAI says the private set is intended for organizations that wish to keep their data private while improving OpenAI’s models’ understanding of their domain; so far, OpenAI has collaborated with the Icelandic government and Miðeind ehf to improve GPT‑4’s Icelandic capabilities, and with the Free Law Project to enhance its model’s comprehension of legal documents.