Export and draw ARSCNFaceGeometry mesh from saved obj file into UIImage

  Kiến thức lập trình

I want to save only face mesh image from ARSCNView for each frame. Below is the code I a using.

override func viewDidAppear(_ animated: Bool) {
        
        let configuration = ARFaceTrackingConfiguration()
        sceneView.session.run(configuration)
        
    }

 func renderer(_ renderer: SCNSceneRenderer, nodeFor anchor: ARAnchor) -> SCNNode? {
    
    let faceMesh = ARSCNFaceGeometry(device: sceneView.device!)
    let node = SCNNode(geometry: faceMesh)
    
    node.name = "Test Node"
    node.geometry?.name = "Test geometry"
    node.geometry?.materials.first?.name = "Test material"        
    node.geometry?.firstMaterial?.fillMode = .lines
    
    return node
}

 func renderer(_ renderer: SCNSceneRenderer, didUpdate node: SCNNode, for anchor: ARAnchor) {
    
    if let faceAnchor = anchor as? ARFaceAnchor, let faceGeometry = node.geometry as? ARSCNFaceGeometry {
        
        faceGeometry.update(from: faceAnchor.geometry)
        
        // let vertices = faceAnchor.geometry.vertices
        let textureCoordinates = faceAnchor.geometry.textureCoordinates
        
        exportMesh(node: node)
        self.index = self.index + 1            
        
    }
}

func exportMesh(_ node: SCNNode, to url: URL, completion: (() -> ())?) {
    DispatchQueue.global().async {
        let mesh = MDLMesh(scnGeometry: node.geometry!)
        let asset = MDLAsset()
        
        asset.add(mesh)
        do {
            try asset.export(to: url)
        } catch {
            print("Can't write mesh to url")
        }
        DispatchQueue.main.async {
            completion?()
        }
    }
}

Above code is to export the node geometry to files app.
Next I am fetching same file from document directory.

func loadMeshes() {
   let asset = MDLAsset(url: path)
   let scene = SCNScene(mdlAsset: asset)

   let renderer = SCNRenderer(device: MTLCreateSystemDefaultDevice(), options: nil)
        
        scene.rootNode.childNodes.first?.geometry?.firstMaterial?.fillMode = .lines
        scene.rootNode.childNodes.first?.geometry = geometry
        
        let renderTime = TimeInterval(0)
        
        // Output size
        let size = CGSize(width:1080, height: 1920)
        
        // Render the image
        let image = renderer.snapshot(atTime: renderTime, with: size,
                                      antialiasingMode: SCNAntialiasingMode.multisampling4X)
        
}

Above code will retrieve mesh object, then convert it to scene. Update scene geometry with lines. Then take snapshot of the scene.

This snapshot is just the geometry of the face. I want the snapshot or image of the face mesh with respect to the original position and angle where it was positioned in the live view.

If I look on the side or up down, then mesh image generated from the scene should also reflect same.

Theme wordpress giá rẻ Theme wordpress giá rẻ Thiết kế website

LEAVE A COMMENT