What is the 3D Creator and how can I use it?

The 3D Creator is an app that allows you to scan objects in 3D. The latest version offers five different scan modes: selfie scan, face scan, head scan, food scan and freeform scan. The 3D Creator is preinstalled in the Xperia 1, Xperia XZ1, Xperia XZ1 Compact, Xperia XZ2, Xperia XZ2 Compact, Xperia XZ2 Premium, and Xperia XZ3. The Xperia XZ Premium receives the 3D Creator as part of the Android 8.0 (Oreo) update.

The first part of the scanning process is the calibration. The camera takes images as you circulate slowly around the object and an algorithm looks for characteristics that stand out in the images. Based on the camera orientation and captured images a point cloud is created. The point cloud is quite rough and has no color. Based on the point cloud information a mesh is created. The mesh is, simply put, a group of 3D points called vertices. Using the points from the point cloud the algorithm sculpts the mesh allowing it to shape itself after the scanned object. Finally, the application creates the texture. The texture in this context is the colors of the object. Using the images captured during sculpting, the colors are projected onto the mesh. The texture has a fixed resolution but the quality and richness of detail depends on how well the sculpting is performed.

The app offers informative tutorials for every scan mode. Below are screenshots from the tutorial for making a face scan.

After finishing a scan you can save and edit the creation, as well as share them or have them 3D printed. Scans are saved in the .OBJ file format which is compatible with most 3D printers. The creation can also be used as a 3D wallpaper, as well as integrated into the Sony Mobile AR effect application as an AR Object or AR Avatar.

There are currently no plans to handle other file formats than .OBJ.