Skip to main content
Version: 3.16.1 (latest)

Performance Parameters

List of abbreviations:

  • FAR (false accept rate) โ€“ the probability that the system incorrectly accepts a non-authorized person.
  • TAR (true accept rate) โ€“ the probability that the system correctly accepts an authorized person.
  • FRR (false reject rate) โ€“ the probability that the system incorrectly rejects an authorized person.
  • IR (identification rate) โ€“ identification rate.

Capturer Characteristics#

Capturer Configuration Files#

Click here to see the list of the capturer configuration files
FileDetectorSet of pointsAngles (roll/yaw/pitch)Description and use case
common_capturer4_fda.xmllbffda[-30;30][-60;60][-60;60]Frontal face detector.
common_capturer4_fda_with_angles.xmllbffda[-90;90][-60;60][-60;60]Frontal face detector. Adapted for a wide range of head rotation angles.
common_capturer4_fda_with_angles_noise.xmllbffda[-90;90][-60;60][-60;60]Frontal face detector. Adapted for a wide range of head rotation angles. Suitable for images with high noise level.
common_capturer4_fda_singleface.xmllbffda[-30;30][-60;60][-60;60]Only one frontal face is detected.
common_capturer4_fda_singleface_with_angles.xmllbffda[-90;90][-75;75][-60;60]Only one frontal face is detected. The detector is adapted for a wide range of head rotation angles.
common_capturer4_fda_singleface_with_angles_noise.xmllbffda[-90;90][-75;75][-60;60]Only one frontal face is detected. The detector is adapted for a wide range of head rotation angles. Suitable for images with high noise level.
common_capturer4_lbf.xmllbfdoublelbf[-30;30][-60;60][-60;60]Frontal face detector.
common_capturer4_lbf_singleface.xmllbfdoublelbf[-30;30][-60;60][-60;60]Only one frontal face is detected.
common_capturer4_mesh.xmllbfmesh[-30;30][-60;60][-60;60]Frontal face detector that allows you to get a 3D face mask.
common_capturer4_mesh_with_angles.xmllbfmesh[-90;90][-60;60][-60;60]Frontal face detector adapted for a wide range of head rotation angles that allows you to get a 3D face mask.
common_capturer4_mesh_with_angles_noise.xmllbfmesh[-90;90][-60;60][-60;60]Frontal face detector adapted for a wide range of head rotation angles and suitable for images with high noise level. The detector allows you to get a 3D face mask.
common_capturer4_mesh_singleface.xmllbfmesh[-30;30][-60;60][-60;60]Only one frontal face is detected. The detector allows you to get a 3D face mask.
common_capturer4_mesh_singleface_with_angles.xmllbfmesh[-90;90][-75;75][-60;60]Only one frontal face is detected. The detector is adapted for a wide range of head rotation angles and allows you to get a 3D face mask.
common_capturer4_mesh_singleface_with_angles_noise.xmllbfmesh[-90;90][-75;75][-60;60]Only one frontal face is detected. The detector is adapted for a wide range of head rotation angles, suitable for images with high noise level and allows you to get a 3D face mask.
common_capturer_blf_fda_front.xmlblffda[-70;70][-90;90][-70;70]Detection of large face images (the face should take up most of the frame size). Suitable for detection of masked faces.
common_capturer_blf_fda_back.xmlblffda[-70;70][-90;90][-70;70]Detection of several faces or small face images. Suitable for detection of masked faces.
common_capturer_blf_fda_auto.xmlblffda[-70;70][-90;90][-70;70]Detection of large and small face images (the parameters `resolution_width` and `min_face_size` should be specified in the configuration file). Suitable for detection of masked faces.
common_capturer_refa_fda_a.xmlrefafda[-70;70][-90;90][-70;70]Face detector recommended for use in expert systems. The detector provides face detection with the largest coverage of rotation angles and maximum quality (including masked faces).
common_capturer_uld_fda.xml.xmluldfda[-70;70][-90;90][-70;70]Detection of large and small face images. Suitable for detection of masked faces.
common_video_capturer_fda.xmllbffda[-30;30][-60;60][-60;60]Frontal face video tracker (RGB only).
common_video_capturer_lbf.xmllbfsinglelbf[-30;30][-60;60][-60;60]Frontal face video tracker (RGB only).
common_video_capturer_mesh.xmllbfmesh[-30;30][-60;60][-60;60]Frontal face video tracker (RGB only) that allows you to get a 3D face mask.
fda_tracker_capturer.xmllbffda[-30;30][-60;60][-60;60]Frontal face video tracker.
fda_tracker_capturer.w.xmllbffda[-30;30][-60;60][-60;60]Frontal face video tracker that can be used in case of insufficient lighting. Note that false detections can occur a bit more often in this case.
fda_tracker_capturer_mesh.xmllbffda[-30;30][-60;60][-60;60]Frontal face video tracker that allows you to get a 3D face mask.
fda_tracker_capturer_fake_detector.xmllbffda[-30;30][-60;60][-60;60]Detection speed is higher because only fitter is used (no detector). Suitable only if a face takes up most of the image size.
fda_tracker_capturer_blf.xmlblffda[-30;30][-60;60][-60;60]Frontal face video tracker. Suitable for detection of masked faces.
fda_tracker_capturer_refa_a.xmlrefafda[-70;70][-90;90][-70;70]Frontal face video tracker. Recommended for use in expert systems. The tracker provides face detection with the largest coverage of rotation angles and maximum quality (including masked faces).
fda_tracker_capturer_uld_fda.xml.xmluldfda[-70;70][-90;90][-70;70]Frontal face video tracker that can be used to detect faces of different size. Suitable for detection of masked faces.
manual_capturer_fda.xmllbffda[-30;30][-60;60][-60;60]Eye points should be manually specified. The remaining points are calculated based on the eye points.
manual_capturer_mesh.xmllbfmesh[-30;30][-60;60][-60;60]Eye points should be manually specified. The remaining points are calculated based on the eye points that allows you to get a 3D face mask.
video_worker_fdatracker_refa_fda.xmlrefafda[-70;70][-90;90][-70;70]Face detector recommended for use in expert systems. The detector provides face detection with the largest coverage of rotation angles and maximum quality (including masked faces).

Capturer Timing Characteristics#

Core i7 4.5 GHz (Single-Core)#

Configuration fileCapture time (ms)
640x480, 1 face640x480, 4 faces1280x720, 1 face1280x720, 4 faces1920x1080, 1 face1920x1080, 4 faces
common_capturer4_fda.xml13 25 34 49 81 103
common_capturer4_fda_with_angles.xml282 387 260 356 273 370
common_capturer4_mesh.xml18 47 39 72 87 735
common_capturer4_mesh_with_angles.xml291 415 268 383 281 398
common_capturer_blf_fda_auto.xml6-30 12-36 8-32 14-38 19-44 26-51
common_capturer_blf_fda_back.xml30 36 32 38 44 51
common_capturer_blf_fda_front.xml6 12 8 14 19 26
common_capturer_refa_fda_a.xml644 650 512 518 580 586
common_capturer_uld_fda.xml (min_size=150)12 18 13 19 21 28
common_capturer_uld_fda.xml (min_size=90)58 70 60 73 77 91
common_capturer_uld_fda.xml (min_size=50)253 272 253 273 281 302
common_capturer4_fda_singleface.xml16 - 51 - 123 -
common_capturer4_mesh_singleface.xml23 - 58 - 129 -

GPU#

Configuration fileCapture time (ms)
640x480, 1 face640x480, 4 faces1280x720, 1 face1280x720, 4 faces1920x1080, 1 face1920x1080, 4 faces
common_capturer_blf_fda_auto.xml4-5 10-12 6-8 13-14 17-20 24-27
common_capturer_blf_fda_back.xml5 12 8 14 20 27
common_capturer_blf_fda_front.xml4 10 6 13 17 24
common_capturer_refa_fda_a.xml236 240 229 235 170 176
common_capturer_uld_fda.xml (min_size=150)4 10 5 11 13 20
common_capturer_uld_fda.xml (min_size=90)14 21 17 23 26 34
common_capturer_uld_fda.xml (min_size=50)27 34 27 35 47 49

Note: Actual capture time may vary depending on the image content.

Identification Performance#

Timing Characteristics for Core i7 4.5 GHz*#

Recognition methodTemplate generation (ms)Accelerated Matching 1:N (ms)Matching 1:1 (ms)
N = 104 N = 106 N = 107
6.7 40 (45**) 0.25 12.1 126 0.04
7.7 170 (180**) 0.25 12.1 126 0.04
8.7 20 (20**) 0.25 12.1 126 0.04
9.30 30 0.18 12.0 117 0.04
9.300 260 (125***) 0.18 12.0 117 0.04
9.1000 730 (305***) 0.18 12.0 117 0.04
9.30mask 20 0.18 12.0 117 0.04
9.300mask 160 (79***) 0.18 12.0 117 0.04
9.1000mask 290 (144***) 0.18 12.0 117 0.04
10.30 24 (16***) 0.18 12.0 117 0.04
10.100 40 (24***) 0.18 12.0 117 0.04
10.1000 690 (355***) 0.18 12.0 117 0.04
11.1000 865 (425***) 0.22 15.0 151 0.04

* โ€“ characteristics specified in this table are given for a single-core CPU.
** โ€“ template creation time when processing_less_memory_consumption was set to true in the FacerecService.createRecognizer call for recognizer creation.
*** โ€“ template creation time using the AVX2 instruction set (see Face identification).

Note:

  • Accelerated search time is given for k=1. As for larger values of k, the time will increase up to the search time without acceleration.
  • Accelerated search is implemented only for the recognition methods 6.5, 6.6, 6.7, 7.3, 7.6, 7.7, 8.6, 8.7, 9.30, 9.300, 9.1000, 10v30, 10v100, 10v1000, 11v1000.
  • To achieve this speed, the templates in the index must be located in order of creation (by using the Recognizer.processing or Recognizer.loadTemplate method).
  • To achieve higher speed, use GPU (see GPU Usage).

Memory Characteristics#

Recognition methodSerialized template size (Bytes)Template size in RAM (Bytes)Memory consumption* (MB)
6.7536636105 (85**)
7.7536636195 (163**)
8.753663652 (40**)
9.30280380155
9.300280380210
9.1000280380290
10.30 280 380 160
10.100 280 380 180
10.1000 280 380 270
11.1000 296 396 480

* โ€“ the amount of memory consumed doesn't depend on the number of the Recognizer objects created by this method
** โ€“ memory consumption when processing_less_memory_consumption was set to true in the FacerecService.createRecognizer call for recognizer creation

ROC on internal dataset with the faces in the wild#

In this test the set of mismatched pairs was increased and LFW errors were fixed to get accurate measurements at low FAR.

FAR9.300 TAR (%)10.30 TAR (%)10.100 TAR (%)11.1000 TAR (%)
1e-4 98.3 95.3 97.3 99.7
1e-5 97.2 91.4 95.0 99.6
1e-6 94.8 86.3 91.7 99.3

Performance Test#

Modes:

Create a Dataset Configuration File#

  • Dataset Configuration File is a text file with 3 lines per each image (<person_id> <image_id> <path_to_image>).

Example:

person0_id
image0_id
path_to_image0
person0_id
image1_id
path_to_image1
...

Example of creating an image dataset configuration file#

Collect a list of image files#
find -type f | sort > ../lfw_simple_format.txt
  • lfw_simple_format.txt is a text file with the lines in the following format: <path/to/person/dir/image_file>. Images with the same path/to/person/dir belong to the same person. Images for one person must be in a row.

Example:

person1_dir/image1
person1_dir/image2
person2_dir/image1
...

The paths in the configuration files must be relative to the directory passed through the dataset_root_dir parameter.

Convert a list to a Configuration File#

In this mode the program converts the config file from list format to dataset format (used in other modes):

Launch parameters:

  • mode โ€“ program mode (convert_config_format)
  • result_dataset_config โ€“ config file of the image dataset
  • FILE โ€“ one file that contains a list of image files

Example of launch:

./test_sdk \
--mode convert_config_format \
--result_dataset_config dataset_config.txt \
lfw_simple_format.txt

The conversion result is a config file of the image dataset.

Detection#

In this mode the program detects faces in images with the id in the range of [begin_image_id, end_image_id). Each image should contain only one person. The images, containing more than one person detected, will be discarded.
Note: when using a GPU, it is recommended to run no more than one of any test at the same time.

ATTENTION

Before starting the program, you need to get the database file with images (see Create a Configuration File)

Launch parameters:

  • mode โ€“ program mode (detection)
  • dll_path โ€“ path to the libfacerec.so or facerec.dll library file
  • sdk_config_dir โ€“ path to the conf/facerec directory
  • dataset_config โ€“ config file of the image dataset (see Create a Configuration File)
  • capturer_config โ€“ capturer config file name
  • dataset_root_dir โ€“ path to the dataset directory
  • detection_result_file โ€“ file for storing the detection results
  • [begin_image_id] โ€“ index of the image, from which the detection begins (the default setting is 0)
  • [end_image_id] โ€“ index of the image, to which detection is performed (by default, the processing is performed to the end of the file)
  • [use_cpu_cores_count] โ€“ number of cores used for detection (the default setting is 1)

Example of launch from the bin directory:

./test_sdk \
--mode detection \
--dll_path ../lib/libfacerec.so \
--sdk_config_dir ../conf/facerec \
--capturer_config common_capturer4_lbf.xml \
--dataset_config dataset_config.txt \
--dataset_root_dir /path/to/data \
--detection_result_file \
detection_result.bin

The detection result is a text file with the lines in the following format: <image_id> <points_count> <points>

Processing#

In this mode the program creates templates from the faces detected in the images with the id in the range of [begin_image_id, end_image_id).

ATTENTION

Before starting the program, you need to get the detections file for the recognizer under test (see Detection)

Launch parameters:

  • mode โ€“ program mode (processing)
  • dll_path โ€“ path to the libfacerec.so or facerec.dll library file
  • sdk_config_dir โ€“ path to the conf/facerec directory
  • dataset_config โ€“ config file of the image dataset (see Create a Configuration File)
  • dataset_root_dir โ€“ path to the dataset directory
  • recognizer_config โ€“ recognizer config file name
  • processing_result_file โ€“ file for storing the resulting templates
  • [begin_image_id] โ€“ index of the image, from which processing starts (the default setting is 0)
  • [end_image_id] โ€“ index of the image, to which processing is performed (by default, the processing is performed at the end of the file)
  • [use_cpu_cores_count] โ€“ number of cores used for processing (the default setting is 1)
  • FILES โ€“ file(s) with the detection results

Note: a trial license allows to run the test only when [use_cpu_cores_count] is set to 1.

Example of launch from the bin directory:

./test_sdk \
--mode processing \
--dll_path ../lib/libfacerec.so \
--sdk_config_dir ../conf/facerec \
--dataset_config dataset_config.txt \
--dataset_root_dir /path/to/data \
--recognizer_config method6v7_recognizer.xml \
--processing_result_file ./templates_6v7.bin \
detection_result.bin

The processing result is a binary file, which contains one record per each generated template. Each record is a 64-bit unsigned integer (image_id) followed by a serialized template.

1:1 Recognition Test#

In this mode the program performs the 1:1 recognition test using the templates generated from the images with the id in the range of [begin_image_id, end_image_id).

ATTENTION

Before running the test, it is necessary to calculate the template file for the recognizer under test (see Processing)

Launch parameters:

  • mode โ€“ program mode (recognition_test_11)
  • dll_path โ€“ path to the libfacerec.so or facerec.dll library file
  • sdk_config_dir โ€“ path to the conf/facerec directory
  • dataset_config โ€“ config file of the image dataset (see Create a Configuration File)
  • recognizer_config โ€“ recognizer config file name
  • result_roc_file โ€“ file to save the ROC curve
  • result_closest_mismatches_file โ€“ result file with the closest mismatches
  • [begin_image_id] โ€“ index of the first image used in the test (the default setting is 0)
  • [end_image_id] โ€“ index of the first image after begin_image_id not used in the test (by default use all from begin_image_id to the end of the file)
  • [use_cpu_cores_count] โ€“ number of cores used for matching (the default setting is 1)
  • FILES โ€“ file(s) with the processing results

Note: a trial license allows to run the test only when [use_cpu_cores_count] is set to 1.

Example of launch from the bin directory:

./test_sdk \
--mode recognition_test_11 \
--dll_path ../lib/libfacerec.so \
--sdk_config_dir ../conf/facerec \
--dataset_config dataset_config.txt \
--recognizer_config method6v7_recognizer.xml \
--result_roc_file ./roc11_6v7.txt \
--result_closest_mismatches_file ./closest_mismatches_file.txt \
templates_6v7.bin

1:1 Recognition of test results:

  • A text file, which contains a ROC curve; the file line describes the curve point in the following format: <far> <tar> <distance>.
  • A text file with the lines in the following format: <distance> <image_id1> <image_id2> <path_to_image1> <path_to_image2>. This file contains pairs of images marked as belonging to different people, but having a minimum distance between the templates. We recommend you to generate this file using the best method and view the images from the top of it to check the errors of the dataset annotation.
Building a ROC curve

To draw a ROC curve, you need to run the draw_roc_curves utility (see Building a ROC curve)

1:N Recognition Test#

In this mode the program performs the 1:N recognition test using the templates generated from the images with the id in the range of [begin_image_id, end_image_id).

ATTENTION

Before running the test, it is necessary to calculate the template file for the recognizer under test (see Processing)

Launch parameters:

  • mode โ€“ program mode (recognition_test_1N)
  • dll_path โ€“ path to the libfacerec.so or facerec.dll library file
  • sdk_config_dir โ€“ path to the conf/facerec directory
  • dataset_config โ€“ config file of the image dataset (see Create a Configuration File)
  • recognizer_config โ€“ recognizer config file name
  • result_roc_file โ€“ file to save the ROC curve.
  • [begin_image_id] โ€“ index of the first image used in the test (the default setting is 0)
  • [end_image_id] โ€“ index of the first image after begin_image_id not used in the test (by default use all from begin_image_id to the end of the file)
  • [use_cpu_cores_count] โ€“ number of cores used for searching (the default setting is 1)
  • [acceleration] โ€“ search acceleration type (the default setting is 0)
  • 0 โ€“ search with pbio::Recognizer::search with the pbio::Recognizer::SearchAccelerationType::NO_SEARCH_ACCELERATION acceleration
  • 1 โ€“ search with pbio::Recognizer::search with the pbio::Recognizer::SearchAccelerationType::SEARCH_ACCELERATION_1 acceleration -1 โ€“ search with pbio::Recognizer::verifyMatch in a single thread (independent of search_threads_count)
  • FILES โ€“ file(s) with the processing results

Note: a trial license allows to run the test only when [use_cpu_cores_count] is set to 1. In addition, it may be needed to specify value for the [end_image_id] parameter so that "gallery_templates size" does not exceed 1000 (printed to the console after running the test).

Example of launch from the bin directory:

./test_sdk \
--mode recognition_test_1N \
--dll_path ../lib/libfacerec.so \
--sdk_config_dir ../conf/facerec \
--dataset_config dataset_config.txt \
--recognizer_config method6v7_recognizer.xml \
--result_roc_file ./roc1N_6v7.txt \
--acceleration 1 \
templates_6v7.bin

The result of the 1:N recognition test is a text file, which contains a ROC curve. The file line describes the curve point in the following format: <far> <tar> <distance>.

Building a ROC curve

To draw a ROC curve, you need to run the draw_roc_curves utility (see Building a ROC curve)

Search Speed Test#

In this mode the program performs the search speed test using the templates generated in processing mode or using the utility (see Template Generator).

Launch parameters:

  • mode โ€“ program mode (search_speed_test).
  • dll_path โ€“ path to the libfacerec.so or facerec.dll library file
  • sdk_config_dir โ€“ path to the conf/facerec directory
  • recognizer_config โ€“ recognizer config file name
  • [templates_count] โ€“ number of templates used (by default, processing is performed until the end of the file)
  • [queries_count] โ€“ number of queries (the default setting is 1)
  • [query_k_nearest] โ€“ number of the nearest templates for searching (the default setting is 1)
  • [search_threads_count] โ€“ number of threads used for searching (the default setting is 1)
  • [acceleration] โ€“ search acceleration type (the default setting is 0)
  • 0 โ€“ search with pbio::Recognizer::search with the pbio::Recognizer::SearchAccelerationType::NO_SEARCH_ACCELERATION acceleration
  • 1 โ€“ search with pbio::Recognizer::search with the pbio::Recognizer::SearchAccelerationType::SEARCH_ACCELERATION_1 acceleration
  • -1 โ€“ search with pbio::Recognizer::verifyMatch in a single thread (independent of search_threads_count)
  • FILES โ€“ file(s) obtained from the processing mode or by using the utility

Note: if a trial license is used, the value of [templates_count] cannot exceed 1000.

Example of launch from the bin directory:

./test_sdk \
--mode search_speed_test \
--dll_path ../lib/libfacerec.so \
--sdk_config_dir ../conf/facerec \
--recognizer_config method6v7_recognizer.xml \
templates_6v7.bin

The result of the search speed test is a message with test results.

Utilities#

Template Generator#

A utility for creating random templates.

Launch parameters:

  • recognizer_version โ€“ recognizer version (select one of [11v1000, 10v30, 10v100, 10v1000, 9v30, 9v300, 9v1000, 8v7, 8v6, 7v7, 7v6, 7v3, 7v2, 7, 6v7, 6v6, 6v5, 6v4, 6v3, 6v2, 6])
  • templates_count โ€“ number of generated templates
  • result_file โ€“ a result binary file, which contains the generated random templates

Example of launch:

./template_generator \
6v7 \
100000 \
random_templates.bin

The template generator result is a binary file, which contains the generated random templates of the same format as in the processing mode.

Source code: examples/cpp/test_sdk

ROC-curve construction#

A utility for constructing a ROC curve from a complete set of points.

WARNING
  • Python 3 required to run the app
  • Before running the program, you need to install the matplotlib package:
    pip3 install matplotlib

Launch parameters:

  • rocs_folder - path to the folder where the files with the calculated set of points for the ROC curve are stored
  • roc_points_files โ€“ names of files for which it is necessary to build a curve (it is possible to specify several files). The file name will be displayed as the name of the ROC curve on the graph

Run example:

python3 ./draw_roc_curves.py\
./base_roc \
roc11_11v1000.txt \
roc11_10v100.txt \
roc11_10v30.txt \
roc11_9v300.txt

Source code: examples/python/draw_roc_curves.py

Last updated on