AI Project Reporting Example

“THE CATS AND DOGS Classification”

Each project requires adequate reporting. Here, in APRO, we know how important it is for our clients and partners and for the success of the project itself.

A good report is an essential part of the OpenX method: it is a way to share information, keep our clients updated and establish trust.

Let me show you an example of a Project State Report for an abstract machine learning project about the classification of cats and dogs. 

Abstract

The goal of the sprint: to reduce the level of misclassification by 5%.

Period: 13 Apr 2021 – 24 Apr 2021.

The results have been obtained with a new ANN (artificial neural network) model, which was trained on the new data set. The data set is balanced and consists of:

• 4000 augmented examples of cats (from 500 original images);
• 4000 augmented examples of dogs (from 500 original images);

The implementation of the model allowed to decrease the misclassification by 6.5% (from 12.2% to 5.7%) by the Project metric.

During the implementation, we faced the issue of overfitting. We plan to solve the issue during the next sprint. Thus the increase in the level of quality will exceed 4%.

The description of the work done in a more detailed way you can find below.

Process: New ANN Model

The new ANN model (4) was integrated into the updated structure of the Cats and Dogs Detection System (see the image below).

The current version of the system consists of the following blocs:

1. – the input data layer (API wrapper – to get an image via the API);
2. – the input image color normalization module;
3. – the input image geometric distortions normalization model;
4. – the new cats and dogs classification model;
5. – the visualization module;
6. – the output result layer  (API wrapper – to send an image via the API).

Thus, the model has a unique custom architecture and well-fitted for the business needs of the project.

Process: Issues

During the sprint, we faced the issue of the new model has overfitting: the quality of results during the training was better than for evaluation.

This issue did not influence the goals of the sprint.

We think the reason for the issue is: 

1. In the volume of data set, because of the lack of data augmentation. We still do not use all the possible augmentation approaches. 
2. In non-optimal macroparameters of the model, because we have not done deep optimization for them.

The possible solution to improve data augmentation algorithm, that should allow increasing the number of examples to at least 8000 items per class and to make a deep optimization for the macro parameters of the model.

Results

The main result of the sprint:

1. The new ANN classification model was developed in embedded in the system. 
2. The use of the new AI data processing model allowed to decrease the misclassification by 6.5% to 5.7% by the Quality Metric.  

   Conclusion

   
   The goal of the sprint was successfully achieved. But the new model has overfitting: the quality of results during the training was better than for evaluation.

The possible solution to improve data augmentation algorithm, that should allow increasing the number of examples to at least 8000 items per class and to make a deep optimization for the macro parameters of the model.

Next Steps

During the next sprint we are planning:

1. To add new augmentation approaches to increase the training data set to at least 8000 items of cats and dogs. 
2. To make deep optimization of macro parameters 
3. To re-train the ANN model. 

Worksheet