Open Source optimized NN models for Ethos-U55 microNPU Benchmarked on Alif Ensemble MCU
Keyword Spotting model
|
|
Alif E5 High-Performance core
|
Alif E5 High-Efficiency core
|
||||
|---|---|---|---|---|---|---|
|
|
Cortex-M55 CPU
|
Ethos-U55 NPU 256 MAC
|
Gain
x times |
Cortex-M55 CPU
|
Ethos-U55 NPU 128 MAC
|
Gain
x times |
|
Inference time, ms
|
11,86
|
0,15
|
79
|
29,55
|
0,56
|
53
|
|
Power consumptions per inference, uJ
|
428,77
|
4,63
|
93
|
477,36
|
9,05
|
53
|
visual_wake_words
|
|
Alif E5 High-Performance core
|
Alif E5 High-Efficiency core
|
||||
|---|---|---|---|---|---|---|
|
|
Cortex-M55 CPU
|
Ethos-U55 NPU 256 MAC
|
Gain
x times |
Cortex-M55 CPU
|
Ethos-U55 NPU 128 MAC
|
Gain
x times |
|
Inference time, ms
|
24,14
|
0,33
|
73
|
59,12
|
1,25
|
47
|
|
Power consumptions per inference, uJ
|
873,10
|
10,16
|
86
|
955,21
|
20,20
|
47
|
image_classification
|
|
Alif E5 High-Performance core
|
Alif E5 High-Efficiency core
|
||||
|---|---|---|---|---|---|---|
|
|
Cortex-M55 CPU
|
Ethos-U55 NPU 256 MAC
|
Gain
x times |
Cortex-M55 CPU
|
Ethos-U55 NPU 128 MAC
|
Gain
x times |
|
Inference time, ms
|
33,69
|
0,29
|
116
|
83,24
|
1,20
|
69
|
|
Power consumptions per inference, uJ
|
1 218,32
|
8,96
|
136
|
1 344,84
|
19,35
|
69
|
anomaly_detection
|
|
Alif E5 High-Performance core
|
Alif E5 High-Efficiency core
|
||||
|---|---|---|---|---|---|---|
|
|
Cortex-M55 CPU
|
Ethos-U55 NPU 256 MAC
|
Gain
x times |
Cortex-M55 CPU
|
Ethos-U55 NPU 128 MAC
|
Gain
x times |
|
Inference time, ms
|
1,30
|
0,13
|
10
|
2,61
|
0,33
|
8
|
|
Power consumptions per inference, uJ
|
47,09
|
3,98
|
12
|
42,17
|
5,27
|
8
|
ns_rnnoise_int8
|
|
Alif E5 High-Performance core
|
Alif E5 High-Efficiency core
|
||||
|---|---|---|---|---|---|---|
|
|
Cortex-M55 CPU
|
Ethos-U55 NPU 256 MAC
|
Gain
x times |
Cortex-M55 CPU
|
Ethos-U55 NPU 128 MAC
|
Gain
x times |
|
Inference time, ms
|
0,88
|
0,11
|
8
|
1,99
|
0,28
|
7
|
|
Power consumptions per inference, uJ
|
31,90
|
3,49
|
9
|
32,12
|
4,46
|
7
|
yolo_v4_192
|
|
Alif E5 High-Performance core
|
Alif E5 High-Efficiency core
|
||||
|---|---|---|---|---|---|---|
|
|
Cortex-M55 CPU
|
Ethos-U55 NPU 256 MAC
|
Gain
x times |
Cortex-M55 CPU
|
Ethos-U55 NPU 128 MAC
|
Gain
x times |
|
Inference time, ms
|
224,41
|
3,24
|
69
|
554,84
|
12,19
|
46
|
|
Power consumptions per inference, uJ
|
8 116,57
|
100,14
|
81
|
8 963,96
|
196,93
|
46
|
ad_micronet_large
|
|
Alif E5 High-Performance core
|
Alif E5 High-Efficiency core
|
||||
|---|---|---|---|---|---|---|
|
|
Cortex-M55 CPU
|
Ethos-U55 NPU 256 MAC
|
Gain
x times |
Cortex-M55 CPU
|
Ethos-U55 NPU 128 MAC
|
Gain
x times |
|
Inference time, ms
|
149,07
|
1,32
|
113
|
345,84
|
5,66
|
61
|
|
Power consumptions per inference, uJ
|
5 391,53
|
40,71
|
132
|
5 587,36
|
91,44
|
61
|
ad_micronet_medium
|
|
Alif E5 High-Performance core
|
Alif E5 High-Efficiency core
|
||||
|---|---|---|---|---|---|---|
|
|
Cortex-M55 CPU
|
Ethos-U55 NPU 256 MAC
|
Gain
x times |
Cortex-M55 CPU
|
Ethos-U55 NPU 128 MAC
|
Gain
x times |
|
Inference time, ms
|
126,64
|
1,21
|
105
|
290,85
|
5,11
|
57
|
|
Power consumptions per inference, uJ
|
4 580,35
|
37,28
|
123
|
4 699,00
|
82,54
|
57
|
ad_micronet_small
|
|
Alif E5 High-Performance core
|
Alif E5 High-Efficiency core
|
||||
|---|---|---|---|---|---|---|
|
|
Cortex-M55 CPU
|
Ethos-U55 NPU 256 MAC
|
Gain
x times |
Cortex-M55 CPU
|
Ethos-U55 NPU 128 MAC
|
Gain
x times |
|
Inference time, ms
|
44,00
|
0,47
|
94
|
105,03
|
1,91
|
55
|
|
Power consumptions per inference, uJ
|
1 591,50
|
14,52
|
110
|
1 696,85
|
30,84
|
55
|
ds_cnn_large_clustered
|
|
Alif E5 High-Performance core
|
Alif E5 High-Efficiency core
|
||||
|---|---|---|---|---|---|---|
|
|
Cortex-M55 CPU
|
Ethos-U55 NPU 256 MAC
|
Gain
x times |
Cortex-M55 CPU
|
Ethos-U55 NPU 128 MAC
|
Gain
x times |
|
Inference time, ms
|
86,26
|
0,74
|
116
|
204,07
|
3,29
|
62
|
|
Power consumptions per inference, uJ
|
3 119,71
|
22,92
|
136
|
3 297,00
|
53,20
|
62
|
kws_cnn_large
|
|
Alif E5 High-Performance core
|
Alif E5 High-Efficiency core
|
||||
|---|---|---|---|---|---|---|
|
|
Cortex-M55 CPU
|
Ethos-U55 NPU 256 MAC
|
Gain
x times |
Cortex-M55 CPU
|
Ethos-U55 NPU 128 MAC
|
Gain
x times |
|
Inference time, ms
|
28,06
|
0,35
|
81
|
61,47
|
1,34
|
46
|
|
Power consumptions per inference, uJ
|
1 015,02
|
10,72
|
95
|
993,08
|
21,58
|
46
|
kws_cnn_medium
|
|
Alif E5 High-Performance core
|
Alif E5 High-Efficiency core
|
||||
|---|---|---|---|---|---|---|
|
|
Cortex-M55 CPU
|
Ethos-U55 NPU 256 MAC
|
Gain
x times |
Cortex-M55 CPU
|
Ethos-U55 NPU 128 MAC
|
Gain
x times |
|
Inference time, ms
|
24,19
|
0,21
|
117
|
54,80
|
0,86
|
64
|
|
Power consumptions per inference, uJ
|
874,80
|
6,39
|
137
|
885,27
|
13,83
|
64
|
kws_cnn_small
|
|
Alif E5 High-Performance core
|
Alif E5 High-Efficiency core
|
||||
|---|---|---|---|---|---|---|
|
|
Cortex-M55 CPU
|
Ethos-U55 NPU 256 MAC
|
Gain
x times |
Cortex-M55 CPU
|
Ethos-U55 NPU 128 MAC
|
Gain
x times |
|
Inference time, ms
|
8,78
|
0,12
|
75
|
21,17
|
0,43
|
50
|
|
Power consumptions per inference, uJ
|
317,59
|
3,61
|
88
|
341,99
|
6,90
|
50
|
kws_dnn_large
|
|
Alif E5 High-Performance core
|
Alif E5 High-Efficiency core
|
||||
|---|---|---|---|---|---|---|
|
|
Cortex-M55 CPU
|
Ethos-U55 NPU 256 MAC
|
Gain
x times |
Cortex-M55 CPU
|
Ethos-U55 NPU 128 MAC
|
Gain
x times |
|
Inference time, ms
|
2,30
|
0,21
|
11
|
4,56
|
0,51
|
9
|
|
Power consumptions per inference, uJ
|
83,08
|
6,39
|
13
|
73,66
|
8,22
|
9
|
kws_dnn_medium
|
|
Alif E5 High-Performance core
|
Alif E5 High-Efficiency core
|
||||
|---|---|---|---|---|---|---|
|
|
Cortex-M55 CPU
|
Ethos-U55 NPU 256 MAC
|
Gain
x times |
Cortex-M55 CPU
|
Ethos-U55 NPU 128 MAC
|
Gain
x times |
|
Inference time, ms
|
0,95
|
0,10
|
9
|
1,89
|
0,25
|
7
|
|
Power consumptions per inference, uJ
|
34,32
|
3,21
|
11
|
30,52
|
4,10
|
7
|
kws_dnn_small
|
|
Alif E5 High-Performance core
|
Alif E5 High-Efficiency core
|
||||
|---|---|---|---|---|---|---|
|
|
Cortex-M55 CPU
|
Ethos-U55 NPU 256 MAC
|
Gain
x times |
Cortex-M55 CPU
|
Ethos-U55 NPU 128 MAC
|
Gain
x times |
|
Inference time, ms
|
0,39
|
0,07
|
6
|
0,80
|
0,17
|
5
|
|
Power consumptions per inference, uJ
|
14,18
|
2,16
|
7
|
12,94
|
2,70
|
5
|
kws_ds_cnn_large
|
|
Alif E5 High-Performance core
|
Alif E5 High-Efficiency core
|
||||
|---|---|---|---|---|---|---|
|
|
Cortex-M55 CPU
|
Ethos-U55 NPU 256 MAC
|
Gain
x times |
Cortex-M55 CPU
|
Ethos-U55 NPU 128 MAC
|
Gain
x times |
|
Inference time, ms
|
86,20
|
0,74
|
116
|
204,08
|
3,29
|
62
|
|
Power consumptions per inference, uJ
|
3 117,75
|
22,95
|
136
|
3 297,07
|
53,19
|
62
|
kws_ds_cnn_medium
|
|
Alif E5 High-Performance core
|
Alif E5 High-Efficiency core
|
||||
|---|---|---|---|---|---|---|
|
|
Cortex-M55 CPU
|
Ethos-U55 NPU 256 MAC
|
Gain
x times |
Cortex-M55 CPU
|
Ethos-U55 NPU 128 MAC
|
Gain
x times |
|
Inference time, ms
|
43,83
|
0,38
|
117
|
108,02
|
1,60
|
67
|
|
Power consumptions per inference, uJ
|
1 585,21
|
11,58
|
137
|
1 745,20
|
25,91
|
67
|
kws_ds_cnn_small
|
|
Alif E5 High-Performance core
|
Alif E5 High-Efficiency core
|
||||
|---|---|---|---|---|---|---|
|
|
Cortex-M55 CPU
|
Ethos-U55 NPU 256 MAC
|
Gain
x times |
Cortex-M55 CPU
|
Ethos-U55 NPU 128 MAC
|
Gain
x times |
|
Inference time, ms
|
11,87
|
0,15
|
79
|
29,58
|
0,56
|
53
|
|
Power consumptions per inference, uJ
|
429,17
|
4,66
|
92
|
477,81
|
9,06
|
53
|
kws_micronet_large
|
|
Alif E5 High-Performance core
|
Alif E5 High-Efficiency core
|
||||
|---|---|---|---|---|---|---|
|
|
Cortex-M55 CPU
|
Ethos-U55 NPU 256 MAC
|
Gain
x times |
Cortex-M55 CPU
|
Ethos-U55 NPU 128 MAC
|
Gain
x times |
|
Inference time, ms
|
183,35
|
1,55
|
118
|
432,07
|
7,12
|
61
|
|
Power consumptions per inference, uJ
|
6 631,48
|
47,97
|
138
|
6 980,59
|
114,95
|
61
|
kws_micronet_medium
|
|
Alif E5 High-Performance core
|
Alif E5 High-Efficiency core
|
||||
|---|---|---|---|---|---|---|
|
|
Cortex-M55 CPU
|
Ethos-U55 NPU 256 MAC
|
Gain
x times |
Cortex-M55 CPU
|
Ethos-U55 NPU 128 MAC
|
Gain
x times |
|
Inference time, ms
|
71,30
|
0,57
|
124
|
177,03
|
2,55
|
69
|
|
Power consumptions per inference, uJ
|
2 578,60
|
17,70
|
146
|
2 860,03
|
41,23
|
69
|
kws_micronet_small
|
|
Alif E5 High-Performance core
|
Alif E5 High-Efficiency core
|
||||
|---|---|---|---|---|---|---|
|
|
Cortex-M55 CPU
|
Ethos-U55 NPU 256 MAC
|
Gain
x times |
Cortex-M55 CPU
|
Ethos-U55 NPU 128 MAC
|
Gain
x times |
|
Inference time, ms
|
44,07
|
0,37
|
119
|
109,65
|
1,56
|
70
|
|
Power consumptions per inference, uJ
|
1 593,92
|
11,43
|
139
|
1 771,54
|
25,27
|
70
|
micronet_vww2
|
|
Alif E5 High-Performance core
|
Alif E5 High-Efficiency core
|
||||
|---|---|---|---|---|---|---|
|
|
Cortex-M55 CPU
|
Ethos-U55 NPU 256 MAC
|
Gain
x times |
Cortex-M55 CPU
|
Ethos-U55 NPU 128 MAC
|
Gain
x times |
|
Inference time, ms
|
14,55
|
0,33
|
44
|
35,86
|
1,09
|
33
|
|
Power consumptions per inference, uJ
|
526,10
|
10,32
|
51
|
579,29
|
17,53
|
33
|
micronet_vww3
|
|
Alif E5 High-Performance core
|
Alif E5 High-Efficiency core
|
||||
|---|---|---|---|---|---|---|
|
|
Cortex-M55 CPU
|
Ethos-U55 NPU 256 MAC
|
Gain
x times |
Cortex-M55 CPU
|
Ethos-U55 NPU 128 MAC
|
Gain
x times |
|
Inference time, ms
|
95,08
|
1,17
|
81
|
234,76
|
4,66
|
50
|
|
Power consumptions per inference, uJ
|
3 438,74
|
36,17
|
95
|
3 792,85
|
75,35
|
50
|
micronet_vww4
|
|
Alif E5 High-Performance core
|
Alif E5 High-Efficiency core
|
||||
|---|---|---|---|---|---|---|
|
|
Cortex-M55 CPU
|
Ethos-U55 NPU 256 MAC
|
Gain
x times |
Cortex-M55 CPU
|
Ethos-U55 NPU 128 MAC
|
Gain
x times |
|
Inference time, ms
|
65,60
|
1,07
|
61
|
161,59
|
3,99
|
41
|
|
Power consumptions per inference, uJ
|
2 372,51
|
33,05
|
72
|
2 610,68
|
64,45
|
41
|
mobilenet_v2_int8
|
|
Alif E5 High-Performance core
|
Alif E5 High-Efficiency core
|
||||
|---|---|---|---|---|---|---|
|
|
Cortex-M55 CPU
|
Ethos-U55 NPU 256 MAC
|
Gain
x times |
Cortex-M55 CPU
|
Ethos-U55 NPU 128 MAC
|
Gain
x times |
|
Inference time, ms
|
684,17
|
8,46
|
81
|
1 610,57
|
35,80
|
45
|
|
Power consumptions per inference, uJ
|
24 744,92
|
261,16
|
95
|
26 020,40
|
578,38
|
45
|
mobilenet_v2_uint8
|
|
Alif E5 High-Performance core
|
Alif E5 High-Efficiency core
|
||||
|---|---|---|---|---|---|---|
|
|
Cortex-M55 CPU
|
Ethos-U55 NPU 256 MAC
|
Gain
x times |
Cortex-M55 CPU
|
Ethos-U55 NPU 128 MAC
|
Gain
x times |
|
Inference time, ms
|
0,00
|
8,39
|
0
|
0,00
|
35,68
|
0
|
|
Power consumptions per inference, uJ
|
0,00
|
259,15
|
0
|
0,00
|
576,45
|
0
|
tiny_wav2letter_pruned
|
|
Alif E5 High-Performance core
|
Alif E5 High-Efficiency core
|
||||
|---|---|---|---|---|---|---|
|
|
Cortex-M55 CPU
|
Ethos-U55 NPU 256 MAC
|
Gain
x times |
Cortex-M55 CPU
|
Ethos-U55 NPU 128 MAC
|
Gain
x times |
|
Inference time, ms
|
545,50
|
6,43
|
85
|
1 131,79
|
30,30
|
37
|
|
Power consumptions per inference, uJ
|
19 729,75
|
198,55
|
99
|
18 285,26
|
489,59
|
37
|
ssd_slim_v1
|
|
Alif E5 High-Performance core
|
Alif E5 High-Efficiency core
|
||||
|---|---|---|---|---|---|---|
|
|
Cortex-M55 CPU
|
Ethos-U55 NPU 256 MAC
|
Gain
x times |
Cortex-M55 CPU
|
Ethos-U55 NPU 128 MAC
|
Gain
x times |
|
Inference time, ms
|
86,64
|
2,85
|
30
|
213,49
|
8,15
|
26
|
|
Power consumptions per inference, uJ
|
3 133,60
|
88,03
|
36
|
3 449,13
|
131,59
|
26
|
har_int8
|
|
Alif E5 High-Performance core
|
Alif E5 High-Efficiency core
|
||||
|---|---|---|---|---|---|---|
|
|
Cortex-M55 CPU
|
Ethos-U55 NPU 256 MAC
|
Gain
x times |
Cortex-M55 CPU
|
Ethos-U55 NPU 128 MAC
|
Gain
x times |
|
Inference time, ms
|
6,84
|
0,32
|
22
|
15,04
|
0,82
|
18
|
|
Power consumptions per inference, uJ
|
247,32
|
9,76
|
25
|
243,03
|
13,23
|
18
|
ssd_mobilenet_v3_int8
|
|
Alif E5 High-Performance core
|
Alif E5 High-Efficiency core
|
||||
|---|---|---|---|---|---|---|
|
|
Cortex-M55 CPU
|
Ethos-U55 NPU 256 MAC
|
Gain
x times |
Cortex-M55 CPU
|
Ethos-U55 NPU 128 MAC
|
Gain
x times |
|
Inference time, ms
|
757,21
|
12,30
|
62
|
0,00
|
0,00
|
0
|
|
Power consumptions per inference, uJ
|
27 386,77
|
379,89
|
72
|
0,00
|
0,00
|
0
|