# Train Your Object Recognition Model from Scratch

[Nikita Beresnev](https://www.strv.com/blog/authors/nikita) iOS Engineer

---

Machine learning has been around for a while now, but only recently has it begun to accelerate. Why? The reason is pretty simple. People have managed to radically improve the computational power of everyday devices in a relatively short period of time.

The first consumer PC was introduced in 1975. That’s a mere 44 years, and look where we are now. The tech we wear on our wrists or carry around in the pockets of your jeans is much more powerful than the computers available to the public a decade ago.

People are lazy by nature. We tend to invent things we don’t actually need for the sake of saving a few extra minutes or hours of our daily routine. We now have endless power and data, and we continue to automate any task we can. Funnily enough, our laziness yields pretty ingenious results.

For a while, image recognition, text generation or even playing games were believed to be something only humans could do. But we’ve managed to use modern tools to help shake ourselves of full responsibility in these cases, too. Machine learning has already been successful in doing “human” activities — like detecting cancer, preventing car crashes, etc. Machines keep surpassing limitations.

And yet, no matter how powerful they get, all machines understand are 0s and 1s. They will always need a bit of extra help from us. With this article, I want to show you exactly how to provide that help.

## ML OPTIONS IN THE APPLE ECOSYSTEM

In the Apple ecosystem, the seemingly obvious solution is to choose Core ML model. However, there are multiple paths you can take in order to create a model. For now, let's focus on solutions that do not require much machine learning expertise, that simplify development with prepared out-of-the-box solutions with decent performance for typical Machine Learning tasks and that work best with the Apple ecosystem. Good examples are Turi Create and Create ML. However, both come with pros and cons.

**TURI CREATE**

**Pros:**  
- More flexible (not tied to the UI)  
- Supports more use cases (one-shot object detection, etc.)  
- Not tied only to macOS (also supports Windows and Linux)  
- Supports various annotation formats

**Cons:**  
- Cumbersome installation process

**CREATE ML**

**Pros:**  
- Has pretty simple UI  
- Comes with the latest XCode

**Cons:**  
- Not as flexible as Turi Create (more scenarios are supported by Turi Create, users can see the bounding boxes in the previews, etc.)  
- Currently, not much info is available

In this article, we’re going to concentrate mainly on the Turi Create solution and will briefly touch on Create ML.

## TURI CREATE INSTALLATION

**Prerequisites:**  
- [Python 2.7.x/3.7.x](https://www.python.org/?ref=strv.ghost.io)  
- [Python Virtual Environment](https://virtualenv.pypa.io/en/latest/?ref=strv.ghost.io)  
- [Jupyter Notebook](https://jupyter.org/index.html?ref=strv.ghost.io)

**Virtual Environment**  
We will be using `virtualenv` - a tool that’ll help us create an isolated Python environment, where we’ll be training our model. You can install `virtualenv` by running the following command in your terminal:

```bash
pip install virtualenv
```

To verify the currently installed version, run:

```bash
virtualenv --version
```
Output should be like:

```
~> 16.5.0
```

If you’re looking for more information about Virtual Environments, please refer to these links:  
- [Docs](https://virtualenv.pypa.io/en/latest/?ref=strv.ghost.io)  
- [Installation](https://virtualenv.pypa.io/en/latest/installation/?ref=strv.ghost.io)

**Jupyter Notebook**  
Navigate to your preferred location in terminal and create a virtual environment:

```bash
virtualenv TuriSample
```

The environment contains directories: bin, include, and lib, which hold dependencies, Python references, and other files.

To activate it, run:

```bash
source TuriSample/bin/activate
```
You should see:

```
(TuriSample) Nick Beresnev:Untracked strv$
```

Install Turi Create within the environment:

```bash
pip install turicreate
```

After installation, install Jupyter:

```bash
pip install jupyter
```

And launch it:

```bash
jupyter notebook
```

This will open a browser window with the notebooks interface.

---

## 6-STEP RECIPE

The 6-step recipe is a recommended set of steps required to create your own trained models:

1. Task understanding  
2. Data collection  
3. Data annotation  
4. Model training  
5. Model evaluation  
6. Model deployment

### Task understanding  
We need to understand the problem, data requirements, and model responsibilities. Here, we'll work on an object detection task to recognize and locate mango, pineapple, banana, and dragonfruit in images.

### Data collection  
Gather a **representative** dataset with varied examples: different angles, scales, lighting, backgrounds, and enough instances per object.

Examples:  
We prepared some training data for you, available in our [repository](https://github.com/strvcom/ios-research-ml-object-detection/tree/master/DataSet/images?ref=strv.ghost.io). 

### Data annotation  
Annotate each image manually with coordinates and labels (JSON or CSV format). To avoid tedious work, tools like MakeML (macOS) or [IBM Cloud Annotation](https://cloud.annotations.ai/?ref=strv.ghost.io) can help.  

Here is how an annotated image looks:

> The annotation includes the image path, bounding box coordinates (center-based), and a label. Coordinates are in pixels, with height and width from the bounding box surrounding the object.  
> ![annotation example]

Annotations are saved in a CSV file, e.g., [annotations.csv](https://github.com/strvcom/ios-research-ml-object-detection/blob/master/DataSet/annotations.csv?ref=strv.ghost.io).

### Model training  
Once data is ready, train the model. Expect it to take time depending on data size, iterations, and hardware.

We'll use a Convolutional Neural Network (CNN):

- Suggested iterations: around 1200.
- Split data into training (80%) and testing (20%).

Example:  
Navigate to the terminal, activate your environment, and start Jupyter Notebook:

```bash
cd ~/Documents/TuriSample
source bin/activate
jupyter notebook
```

In the notebook, import Turi Create:

```python
import turicreate as tc
```

Load images:

```python
images = tc.image_analysis.load_images("TuriSample/Images")
images
```

Explore images interactively:

```python
images.explore()
```

Load annotations:

```python
annotations = tc.SFrame.read_csv("TuriSample/annotations.csv")
```

Join images and annotations:

```python
joined_sframe = images.join(annotations)
```

Split into training and testing sets:

```python
training_sframe, testing_sframe = joined_sframe.random_split(0.8)
```

### Model training in detail:

Train with 50 iterations (can increase later for better accuracy):

```python
model_50 = tc.object_detector.create(training_sframe, max_iterations=50)
```

Monitor training progress with logs showing loss per iteration.

### Comparing models:

Evaluate the model:

```python
model_50
```

And get performance metrics:

```python
metrics_50 = model_50.evaluate(testing_sframe)
```

Sample output:

```
{'average_precision_50': {'banana': 0.172, 'mango': 0.206},
 'mean_average_precision_50': 0.189}
```

Increasing iterations (e.g., 800) improves precision:

```python
metrics_800 = model_800.evaluate(testing_sframe)
```

Results will show improved average precision.

### Model deployment  
Export the trained model:

```python
model.export_coreml("custom_model")
```

The model is saved above your working directory.

---

## SAMPLE PROJECT

The app you will use contains:  
- Real-time camera feed with bounding boxes, labels, confidence scores  
- Switch between models (pretrained and trained)

Parts of the project include:  
- `TrackItemType.swift`  
- `VisionService.swift` (detects objects)  
- `MLModelService.swift` (loads models)

To add your model:

- Import `custom_model.mlmodel` into `Model/Trained Models` in Xcode  
- Switch team in Settings  
- Build and run on an iPhone (cannot run on Simulator due to camera requirement)

Feel free to train your own model! Remember, the trained model's accuracy depends on data diversity and quantity.  

The dataset and code are available in our [repository](https://github.com/strvcom/ios-research-ml-object-detection?ref=strv.ghost.io).

Thanks to [Jaime López](https://www.linkedin.com/in/jaime-andr%C3%A9s-l%C3%B3pez-mora-96b1a910b/?ref=strv.ghost.io) and [Jan Maly](https://www.linkedin.com/in/jan-maly/?ref=strv.ghost.io) for their input.

---

## SOURCES

- [Turi Create User Guide](https://apple.github.io/turicreate/docs/userguide/?ref=strv.ghost.io)  
- [GitHub: Turi Create](https://github.com/apple/turicreate?ref=strv.ghost.io)  
- [Turi Create API](https://apple.github.io/turicreate/docs/api/?ref=strv.ghost.io)  
- [WWDC 2019 - Object Detection](https://developer.apple.com/videos/play/wwdc2019/420/?ref=strv.ghost.io)  
- [WWDC 2018 - Vision & Core ML](https://developer.apple.com/videos/play/wwdc2018/712/?ref=strv.ghost.io)

---

Don't miss anything