artus.evaluate_model package

Submodules

artus.evaluate_model.coco_stats module

The module provides basic descriptive statisitics on a coco file (coco.json).

The COCOStats class can be used to get basic statistics on a coco file after deleting images without annotations in a dataset and removing under-representated classes with the min_nb_occurrences argument.

Typical usage examples:

cocostats = COCOStats(coco_path=’/a/path/to/coco.json’, min_nb_occurrences=50) nb_images = cocostats.get_nb_images() class_occ = cocostats.get_class_stats() cocostats.export_stats(export_path=’/path/to/dir/export.csv)

class artus.evaluate_model.coco_stats.COCOStats(coco_path, min_nb_occurrences=None)

Bases: object

A class to explore the basic stats of a coco file.

The COCOStats class can be used to get basic statistics on a coco file after deleting images without annotations in a dataset and removing under-representated classes with the min_nb_occurrences argument. Statistics can be printed or exported in a csv file.

coco_path

a path to a coco file

Type:

str

min_nb_occurrences

an integer that is the minimum number of occurrences of a

Type:

int

class to be kept in the dataframe.

export_stats(export_path)

Export the number of occurrences per classin csv format.

Statistics are exported after removing images without annotations and (optional) under-representated classes.

Parameters:

export_path (str) – csv path to export the class counts.

get_class_stats()

Print the number of occurrences per classes and number of classes in dataframe after removing images without annotations and under-representated classes.

get_nb_images()

Print the number of images in dataframe after removing images without annotations and under-representated classes.

process_coco(coco_path, min_nb_occurrences)

Read coco file and removes the samples without annotations and underrepresetned classes if needed.

Parameters:

• coco_path (str) – a path to a coco file

• min_nb_occurrences (int) – an integer that is the minimum number of occurrences of a

• dataframe (class to be kept in the) –

• classes. (remove under representated) –

Returns:

A dataframe where one row is one label.

artus.evaluate_model.coco_stats.rm_min_classes(dataset, min_nb_occurrences)

Remove classes that do not reach the minimum number of occurences per class.

Allows to get rid of under-representated in a dataset before splitting a coco file for deep learning purposes.

Parameters:

• dataset (of a class to be kept in the) – a coco file read with importer.ImportCoco() from pylabel. This function

• label (returns a dataframe where one row is one) –

• min_nb_occurrences (int) – an integer that is the minimum number of occurrences

• dataset –

Returns:

The same dataframe format but classes that have less than the minimum number of occurrences are removed.

artus.evaluate_model.coco_stats.rm_tiles_without_annot(dataset)

Remove tiles without annotations.

Parameters:

• dataset – a coco file read with importer.ImportCoco() from pylabel. This function

• label (returns a dataframe where one row is one) –

Returns:

The same dataframe format but rows with empty column ‘cat_name’ are removed.

artus.evaluate_model.evaluate module

artus.evaluate_model.write_eval_results module

Export plots or csv after COCO-style evaluation of a trained deep learning model.

A csv file can be exported or merged to an already existed one. This csv contains AP, AP50, AP75, APm, APl and AP-perclass. Plots can also be exported after having exporting this csv. The plots created are multiple histograms where each bar stands for a model.

class artus.evaluate_model.write_eval_results.ModelsMetricsFormat(session, eval_results, export_dir, csv_name)

Bases: object

A class that writes the evaluation results in a CSV format.

If the CSV already exists, it will merge the results at the last row creating a dataframe where each row is one model evaluated along with their metrics.

session

the name of the directory containing a model_final.pth to evaluate

Type:

str

eval_results

the evaluation results from detectron2.trainer.test()

Type:

dict

export_dir

the path to the directory where metrics will be exported or where previous metrics.csv are saved.

Type:

str

csv_name

a csv name to save the results of evaluation

Type:

str

check_csv_exists()

Check if a csv containing model’s metrics exists or not.

If csv already exists,results of this session will be appended to this csv. Otherwise, a csv will be created.

to_pandas()

Take results from COCOevaluator and format it into a pandas dataframe.

write_to_csv()

Export results to csv format at the csv_path

class artus.evaluate_model.write_eval_results.ModelsMetricsPlots(csv_metrics_path, export_dir, plot_name, title)

Bases: object

A class that plots the results of different models trained.

The plots created are multiple histograms where each bar stands for a model. Plots are more relevant if the csv contains multiple models metrics.

csv_metrics_path

the path to a csv dataframe.

Type:

str

export_dir

the directory where the interactive plots will be exported

Type:

str

plot_name

the name of the plot file with .html extension

Type:

str

title

title to display on plot

Type:

str

export_plots()

Export the plots in html format. Plots are interactives.

plot_metrics()

Plot the metrics for all the models described in the csv file.

Plots created are facet bar plots. One facet is created per variables (AP50, Ap-class1 etc…) and each model precision is described by one bar.

Returns:

A plotly interactive barplot that can be seen with Python(fig.show())

process_csv()

Read csv and pivot the dataframe.

Create a dataframe with one row for each variable and each model.

Module contents