When A.I. Passes This Test, Look Out

New York Times. January 23, 2025.

In the phrase "When A.I. Passes This Test, Look Out," the term "Look Out" is an idiomatic expression meaning "be cautious" or "pay attention." It serves as a warning that significant changes or challenges may arise once artificial intelligence (A.I.) achieves a particular milestone.

Contextual Explanation:

The phrase suggests that when A.I. reaches a certain level of capability—such as passing a specific test designed to measure its intelligence or abilities—we should be vigilant and prepared for the potential implications. This could include A.I. performing tasks previously thought to require human intelligence, leading to societal, ethical, or economic impacts.

Example:

Consider the Turing Test, proposed by Alan Turing, which evaluates a machine's ability to exhibit intelligent behavior indistinguishable from that of a human. If an A.I. were to pass such a test, it would indicate a significant advancement in A.I. capabilities, prompting us to carefully consider the consequences and responsibilities associated with such a development.

In summary, "Look Out" in this context is a cautionary prompt, urging awareness and preparedness for the profound effects that advanced A.I. could have on various aspects of society.

If you’re looking for a new reason to be nervous about artificial intelligence, try this: Some of the smartest humans in the world are struggling to create tests that A.I. systems can’t pass.

For years, A.I. systems were measured by giving new models a variety of standardized benchmark tests. Many of these tests consisted of challenging, S.A.T.-caliber problems in areas like math, science and logic. Comparing the models’ scores over time served as a rough measure of A.I. progress.

But A.I. systems eventually got too good at those tests, so new, harder tests were created — often with the types of questions graduate students might encounter on their exams.

Those tests aren’t in good shape, either. New models from companies like OpenAI, Google and Anthropic have been getting high scores on many Ph.D.-level challenges, limiting those tests’ usefulness and leading to a chilling question: Are A.I. systems getting too smart for us to measure?

This week, researchers at the Center for AI Safety and Scale AI are releasing a possible answer to that question: A new evaluation, called “Humanity’s Last Exam,” that they claim is the hardest test ever administered to A.I. systems.

Humanity’s Last Exam is the brainchild of Dan Hendrycks, a well-known A.I. safety researcher and director of the Center for AI Safety. (The test’s original name, “Humanity’s Last Stand,” was discarded for being overly dramatic.)

Mr. Hendrycks worked with Scale AI, an A.I. company where he is an advisor, to compile the test, which consists of roughly 3,000 multiple-choice and short answer questions designed to test A.I. systems’ abilities in areas ranging from analytic philosophy to rocket engineering.

Questions were submitted by experts in these fields, including college professors and prizewinning mathematicians, who were asked to come up with extremely difficult questions they knew the answers to.

Here, try your hand at a question about hummingbird anatomy from the test:

Hummingbirds within Apodiformes uniquely have a bilaterally paired oval bone, a sesamoid embedded in the caudolateral portion of the expanded, cruciate aponeurosis of insertion of m. depressor caudae. How many paired tendons are supported by this sesamoid bone? Answer with a number.

Or, if physics is more your speed, try this one:

A block is placed on a horizontal rail, along which it can slide frictionlessly. It is attached to the end of a rigid, massless rod of length R. A mass is attached at the other end. Both objects have weight W. The system is initially stationary, with the mass directly above the block. The mass is given an infinitesimal push, parallel to the rail. Assume the system is designed so that the rod can rotate

Episode 3936 - Jan 27 - Tiếng Anh - Khi AI vượt qua bài kiểm tra này, hãy cảnh giác - Vina Technology at AI time

Vina Technology at AI time - Công nghệ Việt Nam thời AI

Episode 3936 - Jan 27 - Tiếng Anh - Khi AI vượt qua bài kiểm tra này, hãy cảnh giác - Vina Technology at AI time