In September, OpenAI initiated a new evaluation process aimed at comparing its AI models' performance against that of human professionals in diverse industries. The company views this comparison as a crucial metric in its progress toward achieving artificial general intelligence (AGI), defined as an AI system capable of surpassing human capabilities in most economically valuable tasks.

One confidential OpenAI document stated, "We’ve hired folks across occupations to help collect real-world tasks modeled off those you’ve done in your full-time jobs, so we can measure how well AI models perform on those tasks. Take existing pieces of long-term or complex work (hours or days) that you’ve done in your occupation and turn each into a task."

The data collection effort highlights the ongoing challenge of evaluating AI performance, particularly as models become more sophisticated. By comparing AI outputs against real-world human work, OpenAI aims to gain a more accurate understanding of its models' strengths and weaknesses. This approach reflects a growing trend in the AI field toward more rigorous and human-centered evaluation methods.

The implications of achieving AGI are far-reaching, potentially transforming industries and reshaping the nature of work. While OpenAI emphasizes the potential benefits of AGI, such as increased productivity and innovation, the development also raises concerns about job displacement and the ethical considerations of increasingly autonomous AI systems.

The request for contractors to submit their work raises questions about data privacy and intellectual property. It is not clear what measures OpenAI is taking to protect the confidentiality of sensitive information contained in the submitted tasks. The company has not yet released details about the specific safeguards in place to prevent misuse of the data.

OpenAI continues to refine its evaluation methodologies as it develops more advanced AI models. The company's focus on human-level performance underscores the importance of aligning AI development with human values and ensuring that AI systems are beneficial to society. The results of these evaluations will likely influence the future direction of OpenAI's research and development efforts.