What kind of results will the AI (artificial intelligence) chatbot ‘ChatGPT’, which has passed the US doctor’s license exam, law school (law school) and MBA (business school) exam, take the Korean SAT?
Anna, an AI technology validation startup, announced on the 6th that as a result of getting ChatGPT to solve the ‘2023 College Scholastic Ability Mathematics Test’ with the research team of Professor Siho Kim of Yonsei University, English was grade 2, but mathematics failed.
This experiment was conducted by entering the entire CSAT question as a question and checking the answer, except for the case where it was impossible to enter the question because it contained a picture. For example, if you enter your fingerprint and ask the question, ‘What is the most appropriate topic for this article?’ along with options 1 to 5, ChatGPT selects an answer from among them.
“In the English test, I got 14 out of 16 questions in the listening test and 13 out of 17 questions in the reading comprehension test,” said Anna. Everyone answered correctly.” The explanation is that “the combined score was 82 points, and the ability was at the level of the second grade in the CSAT.”
However, there were conflicting results in the maths test. Anna CTO Lee Sang-ho (Chief Technology Officer) said, “Only 6 out of 20 questions were answered correctly in the common subject area.” It seems to be very lacking.”
On the 6th, when this magazine gave the CSAT English listening evaluation question to ChatGPT, it actually got the right answer. In the Korean language area, a long passage is presented and ‘Which of the above does not match?’ When I got him to troubleshoot, he said, “Um… Something seems wrong. Please try again later.” It seems that they haven’t been able to find a solution properly due to the lack of Korean data learning yet.
CTO Lee Sang-ho said, “ChatGPT shows excellent capabilities in sentence summarization and reasoning, so application services using it will be endless,” and “talent recruitment requirements will also change in the industry.”
