On March 2nd, researchers from NetEase China published a paper on character model generation technology using AI. It is a technology that generates a 3D character according to the input of a famous name or words such as “dog”, and is expected to be used for creating characters. Natsology reports for the domestic market.
The text published this time is a report on the research results of using AI to output the parameters of a 3D model of text using the “text-to-parameter (T2P) translation” method. It was created by NetEase’s Fuxi AI Lab and seven researchers from Nankai University and Beijing University of Aeronautics and Astronautics. This article has been published on the arXiv preprint server and is a pre-peer-reviewed paper.
What the research group shows in the text is, for example, from the text of the names of famous people such as “Bruce Lee” and “Emilia Clarke” and names such as “dog”, the corresponding “character face” is be generated automatically. Technology. It would be good to look at the image below to see what kind of face model is actually output from the text.
* From left to right: Bruce Lee, Emilia Clarke, dog output
It can be seen that the output of each famous person’s name has the impression that it is a bit of a caricature, but it has many characteristics. As for the “dog”, it’s a bit creepy because it’s a human replica, but if you show it to 10 people, 9 people will judge it as “dog man”, so it has a “like” finish to a dog”. . There are many other examples of output that use celebrity names, such as “Barack Obama,” “Donald Trump,” and “Joe Biden,” for example, which are quite reproducible. Also, “Jackie Chan” and “Emma Watson” would be a pin-up face.
The research group uses OpenAI’s CLIP image classification model and neural rendering technology to enable such output. In addition, in the long-used automatic face model generation method, it is difficult to learn discrete parameters (hairstyle, beard, makeup, etc.), regardless of continuous parameters (skeleton, etc.). On the other hand, the method implemented by the research group this time also supports such discrete face elements. He seems to have the ability to reproduce the typical Peking Opera style beards and makeup just by inputting text.
Character creation in games often requires a lot of effort if you stick with it. Desperately adjusting a large number of sliders to create the ideal face would be a fun and painful task. If technology like this takes off, it might be possible to create characters like “order your favorite type with words and then refine it.” Another advantage of this method for games is that it is strong against superficial elements such as makeup and hairstyles.
These technologies are being developed with the intention of using them to help create game characters etc. In addition, elements such as NPC conversations using ChatGPT will be implemented in the mobile version of the MMORPG “Justice Online” by NetEase. The same work is expected to implement AI-based functions such as “text-based character creation” (IT House). The technology presented in this article will also be included in the same work. A mobile version of the game is expected to be released in 2023. The PC version is currently being developed in China, and a global launch is also planned. It seems likely that AI technology will be applied here as well.
