Using Apple CreateML for Object Detection: From Data Annotation to Model Deployment

The business requirement is to count or identify specific objects in photos, but no pre‑trained model exists, so the author investigates Apple’s CreateML as a solution.

Three possible approaches are compared: using a third‑party platform, building a self‑hosted platform, or leveraging CreateML, with the latter chosen to avoid the overhead of platform development.

The overall CreateML pipeline consists of six steps: gather a large set of sample images, annotate every sample, train a model on the annotated data, validate the model’s recognition rate, test the model, and finally export the model for use.

CreateML requires annotations in a specific JSON format. An example of the required structure is shown below:

{
    "image": "image_name.jpg",
    "annotations": [
        {
            "label": "label_name",
            "coordinates": {"x": 133, "y": 240.5, "width": 113.5, "height": 185}
        }
    ]
}

To generate this JSON efficiently, the author uses Roboflow: after logging in, a workspace and project are created, images are uploaded, and each image is annotated with rectangular boxes and labels. Once all images are annotated, the project can export the annotations in CreateML‑compatible JSON.

In Xcode, the CreateML app is opened via Xcode → Open Developer Tool → Create ML . A new Object Detection project is created, the exported JSON folder is imported as training data, and several parameters are configured – the default algorithm is FullNetworking (YOLOv2), with alternatives like TransferLearning , as well as Iterations , Batch Size , and Grid Size . After setting the parameters (e.g., 100 iterations), the Train button starts the training process.

During training, the UI shows evaluation metrics such as I/U 50% (percentage of predictions with Intersection‑over‑Union above 50%) and Varied I/U (average IoU). When training finishes, the model and its annotation files are exported as a .mlmodel bundle inside a zip file.

The exported .mlmodel is added to an iOS project. Using Apple’s Vision framework, the model is loaded with VNCoreMLModel and a VNCoreMLRequest is created. The article provides a complete Swift example that sets up the request, handles the results, draws bounding boxes, and includes a utility to randomly load test images.

class PipeImageDetectorVC: UIViewController {
    fileprivate var coreMLRequest: VNCoreMLRequest?
    fileprivate var drawingBoxesView: DrawingBoxesView?
    fileprivate var displayImageView: UIImageView = UIImageView()
    fileprivate var randomLoadBtn: UIButton = UIButton(type: .custom)
    // viewDidLoad, setupCoreMLRequest, handleVMRequestDidComplete, handleRandomLoad, etc.
}

The final demo shows the model correctly detecting objects in sample images, and the full source code is available on GitHub under the repository CreateMLDemo . The article concludes with a list of references for further reading on CreateML, Core ML, and object‑detection techniques.