The iOS operating system on the iPhone receives regular updates, introducing new features each year. The latest version, iOS 17, released on September 19, includes a new feature called “Object Capture” that has been gaining attention. “Object Capture” allows users to scan objects using their iPhone and convert them into 3D models. Originally intended for developers, this feature enables users to create 3D models of plastic models and figures and take augmented reality (AR) photos. In this article, we will explain how to use “Object Capture” on iOS 17 to take AR photos, highlighting its ability to easily create life-size 3D models of plastic models and figures. Please note that this feature requires an iPhone 12 Pro or later from the Pro series.
Before using “Object Capture,” it is important to check if your iPhone is compatible. This feature relies on a sensor called a “LiDAR Scanner,” which is available on the iPhone Pro series starting from the iPhone 12 Pro (2020) and the iPad Pro (2021) and later models. iPhones without the Pro designation do not have a LiDAR scanner and are not compatible with Object Capture.
To begin, ensure that your iPhone is running the latest iOS 17 (currently iOS 17.0.2 as of September 28). If you haven’t updated yet, consider doing so. Next, download an app compatible with “Object Capture” from the App Store. Since Apple’s native camera app does not support Object Capture, you will need to use a third-party app. Several apps supporting Object Capture are already available on the App Store. In this example, we used the app “Photogrammetry,” which offers a simple user interface and is available for free with minimal advertisements and no in-app purchases.
Once you have installed a compatible app, such as Photogrammetry, you can start using “Object Capture” to create 3D models of plastic models and figures. Open the app and create a new project by tapping the “+” button. Enter a name for the project, and the camera will launch. Adjust the object you want to scan within the display frame and tap “Continue.” A square frame will appear around the object, indicating the scanning area. Adjust the frame’s dimensions to fit the object. You can easily visualize this in three dimensions and ensure the object remains within the frame by moving your iPhone. Once the frame is set, tap “Start Capture” to begin scanning.
During scanning, you will be prompted to move around the object while holding the iPhone. It is important to follow the instructions on the screen, ensuring you have enough space and avoiding fixing the iPhone in one position. The first scan completes when you have circled the object. You will then receive a message indicating if the object is suitable for scanning from a different angle. The object will be turned around, and you will perform an additional scan while it lies on its back. After the second scan, the object will be scanned once again while it lies on its stomach. After three complete scans, the scanning process is complete. It is worth noting that delicate models may not be suitable for tipping sideways, so you have the option to “Capture more from different heights” instead.
When scanning is finished, your iPhone takes care of the rest. Tapping “Generate Model” initiates a series of automated processes, including preprocessing, image processing, point cloud generation, mesh generation, texture mapping, and finishing. This entire process, from three scans to a completed 3D model, takes about 15 minutes. However, the model generation process is demanding and may drain your battery quickly and cause your iPhone to heat up. For best results, it is recommended to scan and generate 3D models when you have access to a stable power source and can cool your device.
The generated 3D model is saved in the “Model” folder within the iOS app “File.” You can access and view your model by navigating to “This iPhone” → “Photogrammetry” → “File.” With your 3D model ready, you can now take AR photos. Simply tap on the saved model file (which has the “.usdz” extension) in the “File” app, and it will automatically download and open in an app like “AR Quick Look.” This app allows you to view the 3D model in virtual space or use augmented reality (AR) to overlay the model onto your surrounding environment.
With “Object Capture” on iOS 17, you can easily create 3D models of your hobby objects using your iPhone. Whether you want to showcase your plastic models or figures or explore creative AR photography, this feature offers a convenient and accessible tool for capturing and experiencing virtual objects in the real world.
The “iOS” operating system installed on the iPhone is updated year after year, and various new features are added every year. “iOS 17” was released on September 19 this year, and the new feature “Object Capture” included in iOS 17 has been quietly attracting attention.
“Object Capture” is a function that allows you to scan objects using your iPhone and convert them into data such as 3D models. Originally a function for developers, by using “Object Capture” to create 3D models from plastic models and figures, you can make 3D models appear anywhere and take “AR photos.”
In this article, we will introduce how to use iOS 17’s “Object Capture” to take AR photos. We will show you the appeal of “Object Capture,” which allows you to easily create 3D models of your plastic models and figures and take life-size photos.
iPhone 12 Pro or later “Pro series” required. Easy scanning by going around the target only 3 times!
First of all, before using “Object Capture”, let’s check if your iPhone is compatible with “Object Capture”. “Object Capture” can be used on an iPhone or iPad that has a sensor called a “LiDAR Scanner.” Specifically, the compatible devices are the iPhone Pro series starting with the iPhone 12 Pro released in 2020 , and the iPad Pro released in 2021 and later.
Although it has the name Pro, the “iPhone 11 Pro” released in 2019 does not have a LiDAR scanner, and the “iPad Pro (model A12Z)” released in 2020 has a LiDAR scanner. Note that this is not true. support because of the chip it has. Additionally, this verification uses the “iPhone 14 Pro Max” released in 2022.
From the specifications page of the latest model “iPhone 15 Pro”. The specifications page for the latest model “iPhone 15”, where a sensor called “LiDAR scanner” is the core of “Object Capture”. iPhones that do not carry the Pro name do not have a LiDAR scanner, so they are not compatible with Object Capture. This time, I used the author’s iPhone 14 Pro Max. It’s a little hard to tell from the image, but from the flash The black circle opposite the lens is the “LiDAR scanner”
If you have a compatible device, make sure the iOS installed is the latest iOS 17 (iOS 17.0.2 as of September 28). If you haven’t updated it yet, consider updating it depending on your usage situation.
If you own an iPhone Pro series from 2020 or later and need the latest iOS 17, the hurdles are a little high, but if you meet these conditions, proceed to the step next. Install an app compatible with “Object Capture” from the App Store. “Object Capture” is installed in “iOS 17” as an interface called API. However, Apple’s genuine camera app does not support “Object Capture”, so you need to download a compatible app from the App Store.
Several apps that support “Object Capture” have already been released on the App Store. This time, we used “Photogrammetry,” which supports Japanese and features a simple UI. It’s a decent app that can be downloaded for free from the App Store, and although it does show a small ad, there are no in-app purchases.
The installed iOS is a simple menu screen that uses “Photogrammetry” as an “Object Capture” compatible app that can be checked from “General” → “Info” → “ iOS version ” in the Settings app. Only a small advertisement is displayed, and there are no additional charges.
I immediately started using “Photogrammetry” to create 3D models of plastic models and figures. You can create a new project by pressing the “+” button at the top of the menu, and a field to enter the name will appear. This time, we will create a 3D model of the YOASOBI collaboration sticker specification of the “HG 1/144 Gundam Aerial” plastic model from the anime “Mobile Suit Gundam: Witch of Mercury”.
Decide on a name and press “Create” to start the camera. When I adjusted the item I wanted to scan into the display frame and press “Continue,” Gundam Aerial was surrounded by a square frame. This is the frame that determines the area to be scanned, and the length, width and height of this frame need to be adjusted so that the Gundam Aerial can fit inside. It is very easy to understand as it is displayed in three dimensions and you can see if it protrudes out of the frame by moving the iPhone.
After adjusting the frame, press “Start Capture” to start scanning. A pointer-like 3D model immediately appears, but I don’t care and slowly move my iPhone around, focusing on “Gundam Aerial.” At this time, it will warn you if the iPhone is too close to the object or if the brightness is insufficient, so follow the instructions on the screen and scan.
Moreover, when scanning, it is necessary to “hold the iPhone itself” and walk around the model or plastic figure. In addition to making sure you have as much space as possible, you also need to be careful because if the iPhone is fixed, you won’t be able to scan it even if you rotate the plastic model.
The first scan ends when you go around “Gundam Aerial”. Then, a message will appear saying “This object is suitable for taking pictures in a different direction,” and this time the object will be turned around and scanned once while lying on its back. After the second scan, the robot will scan again while lying on its stomach, and after a total of three laps, the scan will be complete.
Gundam Aerial was able to change its orientation, such as lying on its back or lying on its stomach, but with plastic models and figures that have delicate parts, you may want to avoid tipping them to the side. In that case, you can select “Capture more from different heights” to scan from a lower or higher position than the first time without changing the orientation. Damage to plastic models and figures is something owners want to avoid, so this is a relief.
Also, this time I used a photography box when scanning, but normally you wouldn’t need a photography box or green screen. Even if other objects are included in the scan, they will be cropped automatically, creating a highly accurate 3D model. Of course, if you merge the background with one color, it will produce a better 3D model.
The first scan is complete. In the case of “Gundam Aerial”, where the second and third scans are performed by changing the orientation of the object or the scanning height, the point cloud shown on the screen after scanning with the orientation changed is interesting.
Once the scan is complete, your iPhone will take care of the rest. When you tap “Generate Model” after scanning, the process will automatically proceed in the following order: Preprocessing → Image processing → Point cloud generation → Mesh generation → Texture mapping → Finish. The whole process, from three scans to create a 3D model, took about 15 minutes.
However, generating a 3D model from the scan is taxing, which significantly reduces the remaining battery power and causes the iPhone itself to heat up. We recommend that you avoid scanning on the go and create a 3D model beforehand at home, where it is easy to charge and cool.
The generated 3D model is saved in a folder called “Model” from “This iPhone → Photogrammetry” in the authentic iOS app “File”. Let’s take a look at the generated 3D model of “HG 1/144 Gundam Aerial”.
“Gundam Aerial” scan complete! When you tap model generation… the iPhone will automatically process the rest. You can check the generated 3D model from “File” in the authentic iOS app.
Makes you want to go out into the city and take “AR photos”! Create a 3D model of your hobby easily using your iPhone
Next, we will take pictures using AR while checking the 3D model we created. The 3D model of “Gundam Aerial” mentioned earlier is saved with the extension “.usdz”, and by simply tapping “Gundam Aerial.usdz” saved in the “File” app, it will is automatically downloaded to iOS “An app called “AR Quick Look” will open.
With “AR Quick Look,” you can choose to see the 3D model created in virtual space or use AR (augmented reality) to have it appear in the real world through a display. It’s fun to look at the 3D model of “Gundam Aerial” from different angles, but the “AR” shows its true value.
authentic iOS “File” app.The 3D model of “Gundam Aerial” mentioned earlier is saved with the extension .usdz.This is the 3D model of “Gundam Aerial” that was scanned earlier. Although a little rough, this 3D model can be created in about 15 minutes Of course it’s a 3D model, so you can spin it around.
The “Second Gundam Aerial” that appears in the real world through AR is so realistic that it resembles the real thing. Of course, there are variations depending on the light source and environment when scanning, so it is possible to tell the difference to some extent. However, in my case, I had several “accidents” during filming where I tried to touch the 3D model through the display to change the angle of the aerial, only to find that it didn’t actually exist.
All you have to do is press the shutter button and take a picture. Furthermore, the 3D model displayed in AR can be expanded. Although there is a limit of up to 1,000% (=10x), the height of “HG 1/144 Gundam Aerial” is about 1 m 40 cm, so it seems possible to take pictures of “Gundam Aerial” blending into the real world ok .
From here, I will not only create a 3D model of the “HG 1/144 Gundam Aerial” mentioned earlier, but also create 3D models of various figures I have in my house using “Object Capture” , and then create AR images taken using “AR Quick Look” while walking around. Submit photos. Using this function, you can take pictures that look like characters from various anime and games appearing in your daily life, which is a lot of fun.
However, unlike regular camera apps, “AR Quick Look” has a slightly rough image quality, and the color tone of the scanned 3D model cannot be easily corrected, so the 3D model may float depending on the scenes be. Also, since it is not possible to place two or more 3D models at the same time, it is a bit difficult to recreate battle scenes. I look forward to being updated on this area in the future.
On the other hand, if a 3D model is placed in an area where the LiDAR scanner cannot measure distance, it is sometimes possible to take an AR photo with a size of more than 1,000%, increasing the sense of realism. The images presented here should be viewed with the above points in mind. Please note that the settings at work may not match the actual size, so please forgive us.
“Object Capture” allows you to create 3D models of plastic models and figures by downloading just one app, and then take AR photos as they are. In the past, 3D scanning required large hardware, but now it’s convenient to be able to do all this with a single iPhone.
Many people have probably dreamed of making this figure or a larger than life plastic model. Why not create a 3D model of your hobby using iOS 17 x iPhone and take pictures of your favorite people in their everyday lives?
A 3D model being drawn. If you look at it from the side, you might think you’re drawing nothing.
(c) 2023 Apple Inc. All rights reserved.
#Prosiect #ArbennigOversized #gunpla #figures #photos #iPhone #Object #Capture #HOBBY #Watch
