NNs list – Grovety

Open Source optimized NN models for Ethos-U55 microNPU Benchmarked on Alif Ensemble MCU

Keyword Spotting model

Visit GitHub

	Alif E5 High-Performance core			Alif E5 High-Efficiency core
	Cortex-M55 CPU	Ethos-U55 NPU 256 MAC	Gain x times	Cortex-M55 CPU	Ethos-U55 NPU 128 MAC	Gain x times
Inference time, ms	11,86	0,15	79	29,55	0,56	53
Power consumptions per inference, uJ	428,77	4,63	93	477,36	9,05	53

visual_wake_words

Visit GitHub

	Alif E5 High-Performance core			Alif E5 High-Efficiency core
	Cortex-M55 CPU	Ethos-U55 NPU 256 MAC	Gain x times	Cortex-M55 CPU	Ethos-U55 NPU 128 MAC	Gain x times
Inference time, ms	24,14	0,33	73	59,12	1,25	47
Power consumptions per inference, uJ	873,10	10,16	86	955,21	20,20	47

image_classification

Visit GitHub

	Alif E5 High-Performance core			Alif E5 High-Efficiency core
	Cortex-M55 CPU	Ethos-U55 NPU 256 MAC	Gain x times	Cortex-M55 CPU	Ethos-U55 NPU 128 MAC	Gain x times
Inference time, ms	33,69	0,29	116	83,24	1,20	69
Power consumptions per inference, uJ	1 218,32	8,96	136	1 344,84	19,35	69

anomaly_detection

Visit GitHub

	Alif E5 High-Performance core			Alif E5 High-Efficiency core
	Cortex-M55 CPU	Ethos-U55 NPU 256 MAC	Gain x times	Cortex-M55 CPU	Ethos-U55 NPU 128 MAC	Gain x times
Inference time, ms	1,30	0,13	10	2,61	0,33	8
Power consumptions per inference, uJ	47,09	3,98	12	42,17	5,27	8

ns_rnnoise_int8

Visit GitHub

	Alif E5 High-Performance core			Alif E5 High-Efficiency core
	Cortex-M55 CPU	Ethos-U55 NPU 256 MAC	Gain x times	Cortex-M55 CPU	Ethos-U55 NPU 128 MAC	Gain x times
Inference time, ms	0,88	0,11	8	1,99	0,28	7
Power consumptions per inference, uJ	31,90	3,49	9	32,12	4,46	7

yolo_v4_192

Visit GitHub

	Alif E5 High-Performance core			Alif E5 High-Efficiency core
	Cortex-M55 CPU	Ethos-U55 NPU 256 MAC	Gain x times	Cortex-M55 CPU	Ethos-U55 NPU 128 MAC	Gain x times
Inference time, ms	224,41	3,24	69	554,84	12,19	46
Power consumptions per inference, uJ	8 116,57	100,14	81	8 963,96	196,93	46

ad_micronet_large

Visit GitHub

	Alif E5 High-Performance core			Alif E5 High-Efficiency core
	Cortex-M55 CPU	Ethos-U55 NPU 256 MAC	Gain x times	Cortex-M55 CPU	Ethos-U55 NPU 128 MAC	Gain x times
Inference time, ms	149,07	1,32	113	345,84	5,66	61
Power consumptions per inference, uJ	5 391,53	40,71	132	5 587,36	91,44	61

ad_micronet_medium

Visit GitHub

	Alif E5 High-Performance core			Alif E5 High-Efficiency core
	Cortex-M55 CPU	Ethos-U55 NPU 256 MAC	Gain x times	Cortex-M55 CPU	Ethos-U55 NPU 128 MAC	Gain x times
Inference time, ms	126,64	1,21	105	290,85	5,11	57
Power consumptions per inference, uJ	4 580,35	37,28	123	4 699,00	82,54	57

ad_micronet_small

Visit GitHub

	Alif E5 High-Performance core			Alif E5 High-Efficiency core
	Cortex-M55 CPU	Ethos-U55 NPU 256 MAC	Gain x times	Cortex-M55 CPU	Ethos-U55 NPU 128 MAC	Gain x times
Inference time, ms	44,00	0,47	94	105,03	1,91	55
Power consumptions per inference, uJ	1 591,50	14,52	110	1 696,85	30,84	55

ds_cnn_large_clustered

Visit GitHub

	Alif E5 High-Performance core			Alif E5 High-Efficiency core
	Cortex-M55 CPU	Ethos-U55 NPU 256 MAC	Gain x times	Cortex-M55 CPU	Ethos-U55 NPU 128 MAC	Gain x times
Inference time, ms	86,26	0,74	116	204,07	3,29	62
Power consumptions per inference, uJ	3 119,71	22,92	136	3 297,00	53,20	62

kws_cnn_large

Visit GitHub

	Alif E5 High-Performance core			Alif E5 High-Efficiency core
	Cortex-M55 CPU	Ethos-U55 NPU 256 MAC	Gain x times	Cortex-M55 CPU	Ethos-U55 NPU 128 MAC	Gain x times
Inference time, ms	28,06	0,35	81	61,47	1,34	46
Power consumptions per inference, uJ	1 015,02	10,72	95	993,08	21,58	46

kws_cnn_medium

Visit GitHub

	Alif E5 High-Performance core			Alif E5 High-Efficiency core
	Cortex-M55 CPU	Ethos-U55 NPU 256 MAC	Gain x times	Cortex-M55 CPU	Ethos-U55 NPU 128 MAC	Gain x times
Inference time, ms	24,19	0,21	117	54,80	0,86	64
Power consumptions per inference, uJ	874,80	6,39	137	885,27	13,83	64

kws_cnn_small

Visit GitHub

	Alif E5 High-Performance core			Alif E5 High-Efficiency core
	Cortex-M55 CPU	Ethos-U55 NPU 256 MAC	Gain x times	Cortex-M55 CPU	Ethos-U55 NPU 128 MAC	Gain x times
Inference time, ms	8,78	0,12	75	21,17	0,43	50
Power consumptions per inference, uJ	317,59	3,61	88	341,99	6,90	50

kws_dnn_large

Visit GitHub

	Alif E5 High-Performance core			Alif E5 High-Efficiency core
	Cortex-M55 CPU	Ethos-U55 NPU 256 MAC	Gain x times	Cortex-M55 CPU	Ethos-U55 NPU 128 MAC	Gain x times
Inference time, ms	2,30	0,21	11	4,56	0,51	9
Power consumptions per inference, uJ	83,08	6,39	13	73,66	8,22	9

kws_dnn_medium

Visit GitHub

	Alif E5 High-Performance core			Alif E5 High-Efficiency core
	Cortex-M55 CPU	Ethos-U55 NPU 256 MAC	Gain x times	Cortex-M55 CPU	Ethos-U55 NPU 128 MAC	Gain x times
Inference time, ms	0,95	0,10	9	1,89	0,25	7
Power consumptions per inference, uJ	34,32	3,21	11	30,52	4,10	7

kws_dnn_small

Visit GitHub

	Alif E5 High-Performance core			Alif E5 High-Efficiency core
	Cortex-M55 CPU	Ethos-U55 NPU 256 MAC	Gain x times	Cortex-M55 CPU	Ethos-U55 NPU 128 MAC	Gain x times
Inference time, ms	0,39	0,07	6	0,80	0,17	5
Power consumptions per inference, uJ	14,18	2,16	7	12,94	2,70	5

kws_ds_cnn_large

Visit GitHub

	Alif E5 High-Performance core			Alif E5 High-Efficiency core
	Cortex-M55 CPU	Ethos-U55 NPU 256 MAC	Gain x times	Cortex-M55 CPU	Ethos-U55 NPU 128 MAC	Gain x times
Inference time, ms	86,20	0,74	116	204,08	3,29	62
Power consumptions per inference, uJ	3 117,75	22,95	136	3 297,07	53,19	62

kws_ds_cnn_medium

Visit GitHub

	Alif E5 High-Performance core			Alif E5 High-Efficiency core
	Cortex-M55 CPU	Ethos-U55 NPU 256 MAC	Gain x times	Cortex-M55 CPU	Ethos-U55 NPU 128 MAC	Gain x times
Inference time, ms	43,83	0,38	117	108,02	1,60	67
Power consumptions per inference, uJ	1 585,21	11,58	137	1 745,20	25,91	67

kws_ds_cnn_small

Visit GitHub

	Alif E5 High-Performance core			Alif E5 High-Efficiency core
	Cortex-M55 CPU	Ethos-U55 NPU 256 MAC	Gain x times	Cortex-M55 CPU	Ethos-U55 NPU 128 MAC	Gain x times
Inference time, ms	11,87	0,15	79	29,58	0,56	53
Power consumptions per inference, uJ	429,17	4,66	92	477,81	9,06	53

kws_micronet_large

Visit GitHub

	Alif E5 High-Performance core			Alif E5 High-Efficiency core
	Cortex-M55 CPU	Ethos-U55 NPU 256 MAC	Gain x times	Cortex-M55 CPU	Ethos-U55 NPU 128 MAC	Gain x times
Inference time, ms	183,35	1,55	118	432,07	7,12	61
Power consumptions per inference, uJ	6 631,48	47,97	138	6 980,59	114,95	61

kws_micronet_medium

Visit GitHub

	Alif E5 High-Performance core			Alif E5 High-Efficiency core
	Cortex-M55 CPU	Ethos-U55 NPU 256 MAC	Gain x times	Cortex-M55 CPU	Ethos-U55 NPU 128 MAC	Gain x times
Inference time, ms	71,30	0,57	124	177,03	2,55	69
Power consumptions per inference, uJ	2 578,60	17,70	146	2 860,03	41,23	69

kws_micronet_small

Visit GitHub

	Alif E5 High-Performance core			Alif E5 High-Efficiency core
	Cortex-M55 CPU	Ethos-U55 NPU 256 MAC	Gain x times	Cortex-M55 CPU	Ethos-U55 NPU 128 MAC	Gain x times
Inference time, ms	44,07	0,37	119	109,65	1,56	70
Power consumptions per inference, uJ	1 593,92	11,43	139	1 771,54	25,27	70

micronet_vww2

Visit GitHub

	Alif E5 High-Performance core			Alif E5 High-Efficiency core
	Cortex-M55 CPU	Ethos-U55 NPU 256 MAC	Gain x times	Cortex-M55 CPU	Ethos-U55 NPU 128 MAC	Gain x times
Inference time, ms	14,55	0,33	44	35,86	1,09	33
Power consumptions per inference, uJ	526,10	10,32	51	579,29	17,53	33

micronet_vww3

Visit GitHub

	Alif E5 High-Performance core			Alif E5 High-Efficiency core
	Cortex-M55 CPU	Ethos-U55 NPU 256 MAC	Gain x times	Cortex-M55 CPU	Ethos-U55 NPU 128 MAC	Gain x times
Inference time, ms	95,08	1,17	81	234,76	4,66	50
Power consumptions per inference, uJ	3 438,74	36,17	95	3 792,85	75,35	50

micronet_vww4

Visit GitHub

	Alif E5 High-Performance core			Alif E5 High-Efficiency core
	Cortex-M55 CPU	Ethos-U55 NPU 256 MAC	Gain x times	Cortex-M55 CPU	Ethos-U55 NPU 128 MAC	Gain x times
Inference time, ms	65,60	1,07	61	161,59	3,99	41
Power consumptions per inference, uJ	2 372,51	33,05	72	2 610,68	64,45	41

mobilenet_v2_int8

Visit GitHub

	Alif E5 High-Performance core			Alif E5 High-Efficiency core
	Cortex-M55 CPU	Ethos-U55 NPU 256 MAC	Gain x times	Cortex-M55 CPU	Ethos-U55 NPU 128 MAC	Gain x times
Inference time, ms	684,17	8,46	81	1 610,57	35,80	45
Power consumptions per inference, uJ	24 744,92	261,16	95	26 020,40	578,38	45

mobilenet_v2_uint8

Visit GitHub

	Alif E5 High-Performance core			Alif E5 High-Efficiency core
	Cortex-M55 CPU	Ethos-U55 NPU 256 MAC	Gain x times	Cortex-M55 CPU	Ethos-U55 NPU 128 MAC	Gain x times
Inference time, ms	0,00	8,39	0	0,00	35,68	0
Power consumptions per inference, uJ	0,00	259,15	0	0,00	576,45	0

tiny_wav2letter_pruned

Visit GitHub

	Alif E5 High-Performance core			Alif E5 High-Efficiency core
	Cortex-M55 CPU	Ethos-U55 NPU 256 MAC	Gain x times	Cortex-M55 CPU	Ethos-U55 NPU 128 MAC	Gain x times
Inference time, ms	545,50	6,43	85	1 131,79	30,30	37
Power consumptions per inference, uJ	19 729,75	198,55	99	18 285,26	489,59	37

ssd_slim_v1

Visit GitHub

	Alif E5 High-Performance core			Alif E5 High-Efficiency core
	Cortex-M55 CPU	Ethos-U55 NPU 256 MAC	Gain x times	Cortex-M55 CPU	Ethos-U55 NPU 128 MAC	Gain x times
Inference time, ms	86,64	2,85	30	213,49	8,15	26
Power consumptions per inference, uJ	3 133,60	88,03	36	3 449,13	131,59	26

har_int8

Visit GitHub

	Alif E5 High-Performance core			Alif E5 High-Efficiency core
	Cortex-M55 CPU	Ethos-U55 NPU 256 MAC	Gain x times	Cortex-M55 CPU	Ethos-U55 NPU 128 MAC	Gain x times
Inference time, ms	6,84	0,32	22	15,04	0,82	18
Power consumptions per inference, uJ	247,32	9,76	25	243,03	13,23	18

ssd_mobilenet_v3_int8

Visit GitHub

	Alif E5 High-Performance core			Alif E5 High-Efficiency core
	Cortex-M55 CPU	Ethos-U55 NPU 256 MAC	Gain x times	Cortex-M55 CPU	Ethos-U55 NPU 128 MAC	Gain x times
Inference time, ms	757,21	12,30	62	0,00	0,00	0
Power consumptions per inference, uJ	27 386,77	379,89	72	0,00	0,00	0