j{ bdZddlmZddlZddlZddlZddlZddlZddlZddl m Z ddl m Z ddl ZddlZddlmZmZddlmZmZddlmZdd lmZmZmZmZmZmZm Z m!Z!m"Z"m#Z#dd l$m%Z%m&Z&m'Z'm(Z(m)Z)dd l*m+Z+dd l,m-Z-dd l.m/Z/m0Z0e"dz ddddddddf dZ1GddZ2GddZ3dS)a- Benchmark YOLO model formats for speed and accuracy. Usage: from ultralytics.utils.benchmarks import ProfileModels, benchmark ProfileModels(['yolo26n.yaml', 'yolov8s.yaml']).run() benchmark(model='yolo26n.pt', imgsz=160) Format | `format=argument` | Model --- | --- | --- PyTorch | - | yolo26n.pt TorchScript | `torchscript` | yolo26n.torchscript ONNX | `onnx` | yolo26n.onnx OpenVINO | `openvino` | yolo26n_openvino_model/ TensorRT | `engine` | yolo26n.engine CoreML | `coreml` | yolo26n.mlpackage TensorFlow SavedModel | `saved_model` | yolo26n_saved_model/ TensorFlow GraphDef | `pb` | yolo26n.pb TensorFlow Lite | `tflite` | yolo26n.tflite TensorFlow Edge TPU | `edgetpu` | yolo26n_edgetpu.tflite TensorFlow.js | `tfjs` | yolo26n_web_model/ PaddlePaddle | `paddle` | yolo26n_paddle_model/ MNN | `mnn` | yolo26n.mnn NCNN | `ncnn` | yolo26n_ncnn_model/ IMX | `imx` | yolo26n_imx_model/ RKNN | `rknn` | yolo26n_rknn_model/ ExecuTorch | `executorch` | yolo26n_executorch_model/ ) annotationsN)deepcopyPath)YOLO YOLOWorld) TASK2DATA TASK2METRIC)export_formats) ARM64ASSETS ASSETS_URL IS_JETSONLINUXLOGGERMACOSTQDM WEIGHTS_DIRYAML)IS_PYTHON_3_13 check_imgszcheck_requirements check_yolo is_rockchip) safe_download) file_size) get_cpu_info select_devicez yolo26n.ptFcpuMbP?c $t|}t|tr|d|dks nn Jdddl} | jd| jd| jd| jd| j d| j dt|d }t|ttfrt|}|pt|j}t"|j} g} t%j} |}|r6t)t+d }||vsJd |d |d t-t+D]\}}}}}}d\}} |r||kr|dkr|jdks Jdnc|dkrt0rt2r JdnD|dkrt0rt2r Jdn%|dkrt4st0rt2r Jd|dkrt6r Jd|dvrt|t8r Jd|dkrSt|t8r Jd|jdks Jdt0rt:rt4s Jd|dkrt|t8r Jd |d!krt|t8r Jd"|d#krRt|t8r Jd$|jd%vs Jd&d'|vs Jd(|d)krHt|t8r Jd*t0s Jd+t?r Jd,|d-krt|t8r Jd.d/|j vr |s Jd0d1|j vr |s Jd2|d3kr%|j!p |j"p|j#}tI|}nStI|j%db||||||dd4| }t||j5}|t|vs Jd6d7}|jd8ks|dks Jd9|d:vs Jd;|dks!tMj'dz |||d?|*|d|d|||dd@A }|j+| |j,dB}}t[dC||zz dD}| .|dEt[t_|dt[|dFt[|dD|g#t`$r}|r&tA|tbusJdG|dH|tej3dG|dH|| .||t[t_|ddddgYd}~d}~wwxYwti|I| 5| dJdKdL| dMdNgdOP}|6dQdR}|7| 89| j:;d3}|j#}t%j| z }dS} dT|dU|dV|dW|dXdY| dZ|dZ }!tej<|!t{d[d\d]d^_5}"|">|!dddn #1swxYwY|rYt|t~rD|| @}#|$tq$fd`|#Ds Jda$|S)caBenchmark a YOLO model across different formats for speed and accuracy. Args: model (str | Path): Path to the model file or directory. data (str | None): Dataset to evaluate on, inherited from TASK2DATA if not passed. imgsz (int): Image size for the benchmark. half (bool): Use half-precision for the model if True. int8 (bool): Use int8-precision for the model if True. device (str): Device to run the benchmark on, either 'cpu' or 'cuda'. verbose (bool | float): If True or a float, assert benchmarks pass with given metric. eps (float): Epsilon value for divide by zero prevention. format (str): Export format for benchmarking. If not supplied all formats are benchmarked. **kwargs (Any): Additional keyword arguments for exporter. Returns: (polars.DataFrame): A Polars DataFrame with benchmark results for each format, including file size, metric, and inference time. Examples: Benchmark a YOLO model with default settings: >>> from ultralytics.utils.benchmarks import benchmark >>> benchmark(model="yolo26n.pt", imgsz=640) rTz'benchmark() only supports square imgsz.NASCII_BORDERS_ONLY_CONDENSEDF)verboseArgumentzExpected format to be one of z , but got 'z'.)❌Npbobbz.TensorFlow GraphDef not supported for OBB taskedgetpuz3Edge TPU export only supported on non-aarch64 Linuxtfjsz)TF.js export not supported on ARM64 Linuxcoremlz;CoreML export only supported on macOS and non-aarch64 Linuxz#CoreML not supported on Python 3.13>r*r-tfliter, saved_modelz;YOLOWorldv2 TensorFlow exports not supported by onnx2tf yetpaddlez,YOLOWorldv2 Paddle exports not supported yetzBPaddle OBB bug https://github.com/PaddlePaddle/Paddle/issues/72024z3Windows and Jetson Paddle exports not supported yetmnnz)YOLOWorldv2 MNN exports not supported yetncnnz*YOLOWorldv2 NCNN exports not supported yetimxz%YOLOWorldv2 IMX exports not supported>posedetectsegmentclassifyzbIMX export is only supported for detection, classification, pose estimation and segmentation tasksC2fz*IMX only supported for YOLOv8n and YOLO11nrknnz*YOLOWorldv2 RKNN exports not supported yetzRKNN only supported on Linuxz1RKNN Inference only supported on Rockchip devices executorchz0YOLOWorldv2 ExecuTorch exports not supported yetr zinference not supported on CPUcudazinference not supported on GPU-)imgszformathalfint8datadevicer')taskz export failedu❎r5z(GraphDef Pose inference is not supported>r-r,zinference not supportedDarwinz(inference only supported on macOS>=10.13zbus.jpg)r>rCr@r'r!) rBbatchr>plotsrCr@rAr'conf inferenceu✅zBenchmark failure for : )rCFormatu Status❔z Size (MB)zInference time (ms/im)FPSrow)schemaorient )offsetudBenchmarks legend: - ✅ Success - ❎ Export passed but validation failed - ❌️ Export failedz Benchmarks complete for z on z at imgsz=z (z.2fzs)  zbenchmarks.logaignoreutf-8)errorsencodingc3LK|]}tj||kVdSN)npisnan).0xfloors a/home/longshao/multi-rider-rag/.venv/lib/python3.11/site-packages/ultralytics/utils/benchmarks.py zbenchmark..s4AARXa[[A1u9AAAAAAz%Benchmark failure: metric(s) < floor )Ar isinstancelistpolarsConfig set_tbl_cols set_tbl_rowsset_tbl_width_charsset_tbl_hide_column_data_typesset_tbl_hide_dataframe_shapeset_tbl_formattingrstrrrr rDr timelower frozensetr zipvaluesrr rrrr__str__rtypept_path ckpt_path model_namerexportplatformsystempredictr val results_dictspeedroundappendr ExceptionAssertionErrorrerrorr DataFramewith_row_index with_columnsallcastString fill_nullinfoopenwritefloatto_numpy)%modelrBr>r@rArCr'epsr?kwargsplkeyyt0 format_argformatsnamesuffixr gpu_emojifilenameexported_modelresultsmetricrfpsedf df_displaydtlegendsfmetricsras% @rb benchmarkr7sN F   E#-eT#:#:o58uQx     Fo   DI2I2I!!"%%%I,,T222I**4000I  !?@@@ 65 1 1 1F%#t%%U   (9UZ(D ej !C A BJeN,,Z899   "d'"d"dV`"d"d"d   -0.2B2B2I2I2K2K-LRURU)ffc3%xP U jF22~~zU***,\****9$$aUaa,aaa*a6!!!YeYY.YYY,8##ttutt7ttt5!!)PP+PPP)KKK%eY77vv9vvv7!!%eY77gg9ggg7zU***,p***pipEpp;ppp9%eY77dd9ddd7%eY77ee9eee7%eY77``9```7z%NNNNxONN ///1]///%eY77ee9eee7<<<<>>;e>>9!44446O444X%%):):h)F)F)FHr)F)FF  " "6I#5U6X\fk " l l l%(()  G$05w}[7QEF ,a00C HHdE58)<) > > HHdE58)<)>> benchmark = RF100Benchmark() >>> benchmark.set_key("your_roboflow_api_key") roboflowr)Roboflow)rN)rrrr)rrrs rbset_keyzRF100Benchmark.set_keys< :&&&%%%%%%(7+++rddatasets_links.txt ds_link_txtctjdr(tjdtjdfntjdtjdtjdttdt|d5}|D]} tj d| \}}}}}|j ||d|} t| sT|j|||dnt+jd |jtj| z d z #t2$rYwxYw d d d n #1swxYwY|j |jfS) aParse dataset links and download datasets. Args: ds_link_txt (str): Path to the file containing dataset links. Returns: (tuple[list[str], list[Path]]): List of dataset names and list of paths to dataset configuration files. Examples: >>> benchmark = RF100Benchmark() >>> benchmark.set_key("api_key") >>> benchmark.parse_dataset("datasets_links.txt") zrf-100zultralytics-benchmarksz/datasets_links.txtrXrZz/+r=yolov8zDataset already downloaded.z data.yamlN)ospathexistsshutilrmtreemkdirchdirrrrresplitstriprrrr workspaceprojectversiondownloadrrrcwdr) rrfileliner_urlrrr proj_versions rb parse_datasetzRF100Benchmark.parse_datasets:<9Q9Qix "(8"4"455WYW_`hWiWi  )***888999 + 0 0 0 D   ;=8D$**,,;W;W8AtYM((111&-#9#9#9#9L --4466C)))44<>> benchmark = RF100Benchmark() >>> benchmark.evaluate("path/to/data.yaml", "path/to/val_log.txt", "path/to/eval_log.txt", 0) )u🚀u⚠️u💡r)namesrXrc3 K|]}|vV dSr\re)r_symbolrs rbrcz*RF100Benchmark.evaluate..Hs'AA&v~AAAAAArdrSc|dkS)Nr"re)rs rbz)RF100Benchmark.evaluate..Ks #)rdc8g|]}|dS)rU)r)r_rs rb z+RF100Benchmark.evaluate..Ls":::Q1774==:::rdc 3K|]J}|vs|dk dvdvddddddd d VKd S) rz(AP)z(AR)rr$rKrLrNre)r_r class_namesentriess rbrcz*RF100Benchmark.evaluate..Ms " "K''AJJ6;P;PU[cjUjUj")")!*#*1:%,QZ")!*!(!(VkUjUjUj " "rdNr$zMultiple dicts foundrrrzSingle dict foundc3&K|] }|dV dS)rNre)r_ress rbrcz*RF100Benchmark.evaluate..bs&>>C3w<>>>>>>rdrVrMrU)rrr readlinesanyrrgfilterextendlenrrnextrrr)rrrrr skip_symbolsrlines eval_linesmap_vallstrrrs @@@rbevaluatezRF100Benchmark.evaluate1s"988 i **73 , 1 1 1 QKKMMEJ  AAAALAAAAA**S//v&;&;WEEFF::':::!! " " " " "% " " "                     , z??Q   K. / / /! + +w<5((!'lG + K+ , , ,>>:>>>>>G -w 7 7 7 ?1 GGt}X.=='=== > > > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?W~~s$BCC C&'FF FN)rrp)r)rrp)rr)rrprrprrprr) __name__ __module__ __qualname____doc__rrr staticmethodrrrerdrbrrs"ccc,,,, !/!/!/!/!/F###\#666666rdrceZdZdZ d2d3dZdZdZed4dZed5d6d!Z d7d8d%Z ed&Z d7d9d'Z d:d.Z ed:d/Zed;d1ZdS)< ProfileModelsa*ProfileModels class for profiling different models on ONNX and TensorRT. This class profiles the performance of different models, returning results such as model speed and FLOPs. Attributes: paths (list[str]): Paths of the models to profile. num_timed_runs (int): Number of timed runs for the profiling. num_warmup_runs (int): Number of warmup runs before profiling. min_time (float): Minimum number of seconds to profile for. imgsz (int): Image size used in the models. half (bool): Flag to indicate whether to use FP16 half-precision for TensorRT profiling. trt (bool): Flag to indicate whether to profile using TensorRT. device (torch.device): Device used for profiling. Methods: run: Profile YOLO models for speed and accuracy across various formats. get_files: Get all relevant model files. get_onnx_model_info: Extract metadata from an ONNX model. iterative_sigma_clipping: Apply sigma clipping to remove outliers. profile_tensorrt_model: Profile a TensorRT model. profile_onnx_model: Profile an ONNX model. generate_table_row: Generate a table row with model metrics. generate_results_dict: Generate a dictionary of profiling results. print_table: Print a formatted table of results. Examples: Profile models and print results >>> from ultralytics.utils.benchmarks import ProfileModels >>> profiler = ProfileModels(["yolo26n.yaml", "yolov8s.yaml"], imgsz=640) >>> profiler.run() d <TNpaths list[str]num_timed_runsrnum_warmup_runsmin_timerr>r@booltrtrCtorch.device | str | Nonec ||_||_||_||_||_||_||_t|tj r|nt||_ dS)a[Initialize the ProfileModels class for profiling models. Args: paths (list[str]): List of paths of the models to be profiled. num_timed_runs (int): Number of timed runs for the profiling. num_warmup_runs (int): Number of warmup runs before the actual profiling starts. min_time (float): Minimum time in seconds for profiling a model. imgsz (int): Size of the image used during profiling. half (bool): Flag to indicate whether to use FP16 half-precision for TensorRT profiling. trt (bool): Flag to indicate whether to profile using TensorRT. device (torch.device | str | None): Device used for profiling. If None, it is determined automatically. Notes: FP16 'half' argument option removed for ONNX as slower on CPU than FP32. N) rrrrr>r@rrftorchrCr) rrrrrr>r@rrCs rbrzProfileModels.__init__s`4 ,.     *65< @ @[ffmTZF[F[ rdc |}|stjdgSg}g}|D]}|d}|jdvrt t |}|||j }|j rM|j j dkr=| s)|d|j|j |j d}|d |j |j d }n%|jd kr||}|}n |t |} |t |} |||j| | ||||j| | ||||S) aProfile YOLO models for speed and accuracy across various formats including ONNX and TensorRT. Returns: (list[dict]): List of dictionaries containing profiling results for each model. Examples: Profile models and print results >>> from ultralytics.utils.benchmarks import ProfileModels >>> profiler = ProfileModels(["yolo26n.yaml", "yolo11s.yaml"]) >>> results = profiler.run() z'No matching *.pt or *.onnx files found.z.engine.pt.yml.yamlr>r engineF)r?r@r>rCr'onnx)r?r>rCr'z.onnx) get_filesrwarning with_suffixrrrpfuserr>rrCrwis_filer{r@get_onnx_model_infoprofile_tensorrt_modelprofile_onnx_modelrgenerate_table_rowstemgenerate_results_dict print_table) rfiles table_rowsoutputr engine_filer model_info onnx_filet_enginet_onnxs rbrunzProfileModels.runs    ND E E EI  _ _D**955K{666SYY "ZZdjZ99 8 0E 9 9+BUBUBWBW 9"',,'!Y"j#{ % #/##K"LL!*;! ) ''!55d;;  223{3C3CDDH,,S^^<.s=```sTYsSWZ]S]E_E_``Td````rdrz Profiling: c,g|]}t|Srer)r_rs rbrz+ProfileModels.get_files..s555tT 555rd) rris_dirrrrrpr:rrsorted)rr/ extensionsrs @rbr#zProfileModels.get_filess J 3 3D::D{{}} 3999  ````j```aaaa 888 SYY'''' TYs4yy112222 1&--1122255ve}}5555rdr4rpcdS)z\Extract metadata from an ONNX model file including layers, parameters, gradients, and FLOPs.)rrrrre)r4s rbr(z!ProfileModels.get_onnx_model_infos "!rdrKrrB np.ndarraysigma max_itersc(tj|}t|D]m}tj|tj|}}|||||zz k||||zzkz}t |t |krn|}n|S)a{Apply iterative sigma clipping to data to remove outliers. Args: data (np.ndarray): Input data array. sigma (float): Number of standard deviations to use for clipping. max_iters (int): Maximum number of iterations for the clipping process. Returns: (np.ndarray): Clipped data array with outliers removed. )r]arrayrangemeanstdr)rBrBrCrrGrH clipped_datas rbiterative_sigma_clippingz&ProfileModels.iterative_sigma_clippingsx~~y!!  A rvd||#Dus{(:!:tdUUX[FX?X YZL<  CII--DD rdr!r2rch|jr!t|sdSt|}t j|j|jdftj}d}tdD]U}tj }t|j D]}|||jdtj |z }Vtt|j ||zz |j z|jdz}g} tt||D];}|||jd} | | d jd <|t j| d d } t j| t j| fS) awProfile YOLO model performance with TensorRT, measuring average run time and standard deviation. Args: engine_file (str): Path to the TensorRT engine file. eps (float): Small epsilon value to prevent division by zero. Returns: (tuple[float, float]): Mean and standard deviation of inference time in milliseconds. )rrr)dtyperF)r>r'2descrrIrKrBrC)rrr'rr]zerosr>uint8rFrqrmaxrrrrrrrJrErGrH) rr2rr input_dataelapsedr start_timenum_runs run_timesrs rbr)z$ProfileModels.profile_tensorrt_modelsx tK0088:: 8[!!Xtz4:q9JJJ q / /AJ4/00 C Cj EBBBBBikkJ.GGuT]gm.:s5QQSz#s++8qQQQQQQrd)r) tensor_shapes rb check_dynamiczProfileModels.check_dynamic7s$QQLQQQQQQQrdctdgddl}|}|jj|_d|_|||dg}i}|D]Y}|j }| |j rt|j dkrA| |j ddrtd |j d |jt|j dkrdd |j|jfndg|j ddR} n|j } d |vr t j} nVd |vr t j} nEd|vr t j} n4d|vr t j} n#d|vr t j} ntd|t!jj| | } |j} | || <[|dj} d}t5d D]Y}t7j}t5|jD]}|| g|t7j|z }Zt=t?|j ||zz |jz|j!}g}tEt5||D]X}t7j}|| g||#t7j|z dzY|$t!j%|dd}t!j&|t!j'|fS)atProfile an ONNX model, measuring average inference time and standard deviation across multiple runs. Args: onnx_file (str): Path to the ONNX model file. eps (float): Small epsilon value to prevent division by zero. Returns: (tuple[float, float]): Mean and standard deviation of inference time in milliseconds. ) onnxruntimezonnxruntime-gpurNCPUExecutionProvider) providersrLr$zUnsupported dynamic shape z of rfloat16rdoubleint64int32zUnsupported ONNX datatype rrNrJrKrrP)(rr_SessionOptionsGraphOptimizationLevelORT_ENABLE_ALLgraph_optimization_levelintra_op_num_threadsInferenceSession get_inputsrwr]shaper ValueErrorrr>r]rcfloat32float64rerfrandomrandastype get_outputsrFrqrr7rSrrrrrrJrErGrH)rr4rort sess_optionssessinput_data_dict input_tensor input_type input_shape input_dtyperT input_name output_namerUrrVrWrXs rbr*z ProfileModels.profile_onnx_model<sx >?@@@!!!!))++ 030J0Y -,- )##I|H^G_#`` OO-- 5 5L%*J!!,"455 1|)**a//D4F4F|GYZ[Z\Z\G]4^4^/$%m,BT%m%mZfZk%m%mnnn69,:L6M6MQR6R6RQ4:tz22YZXt]i]opqprpr]sXtXt +0 J&& j J&& j Z'' j J&& h J&& h  !Jj!J!JKKK5<<[IIJ%*J*4OJ ' '&&((+0 q / /AJ4/00 9 9+8888ikkJ.GGuT]gm ? ? ? ?11"(92E2EQZ[1\\ wy!!26)#4#444rdrzr6tuple[float, float]r5r3!tuple[float, float, float, float]c|\}}}}d|dd|jd|ddd|ddd |ddd|ddd |d z dd|dd S) aGenerate a table row string with model performance metrics. Args: model_name (str): Name of the model. t_onnx (tuple): ONNX model inference time statistics (mean, std). t_engine (tuple): TensorRT engine inference time statistics (mean, std). model_info (tuple): Model information (layers, params, gradients, flops). Returns: (str): Formatted table row string with model metrics. z| 18sz | z | - | rz.1f±r$z ms | g.Az |r ) rrzr6r5r3_layersparams _gradientsflopss rbr+z ProfileModels.generate_table_rows$.8*U I p I IDJ I Ivay p I Iq p I I]efg]h p I I{ H I I&,sl H I I;@ H I I I rdc|\}}}}||t|dt|ddt|dddS)aGenerate a dictionary of profiling results. Args: model_name (str): Name of the model. t_onnx (tuple): ONNX model inference time statistics (mean, std). t_engine (tuple): TensorRT engine inference time statistics (mean, std). model_info (tuple): Model information (layers, params, gradients, flops). Returns: (dict): Dictionary containing profiling results. rr)z model/namezmodel/parametersz model/GFLOPszmodel/speed_ONNX(ms)zmodel/speed_TensorRT(ms))r)rzr6r5r3rrrrs rbr-z#ProfileModels.generate_results_dictsT$.8*U$ &!%OO$)&)Q$7$7(-hqk1(=(=    rdr0ctjrtjdnd}ddddt dd|d d d g}d d d |Dzd z}d d d|Dzd z}t jd|t j||D]}t j|dS)zPrint a formatted table of model profiling results. Args: table_rows (list[str]): List of formatted table row strings. rGPUModelzsize
(pixels)zmAPval
50-95zSpeed
CPU (z) ONNX
(ms)zSpeed
z TensorRT
(ms)zparams
(M)zFLOPs
(B)|c3"K|] }d|dV dS)rSNrer_hs rbrcz,ProfileModels.print_table..s*::QA::::::rdc3@K|]}dt|dzzVdS)r=rKN)rrs rbrcz,ProfileModels.print_table..s1"G"G!3#a&&1*#5"G"G"G"G"G"Grdz N)rr< is_availableget_device_namerjoinrr)r0rheadersheader separatorrPs rbr.zProfileModels.print_tables05z/F/F/H/HSej((+++e  # " @,.. @ @ @ 3S 3 3 3  sxx::'::::::S@#(("G"Gw"G"G"GGGG#M  O6OO$$$ I  C K      rd)r rrrTTN)rrrrrrrrr>rr@rrrrCr)r4rp)rKr)rBrArBrrCr)r!)r2rprr)r4rprr)rzrpr6rr5rr3r)r0r)rrrr rr7r#r r(rJr)r]r*r+r-r.rerdrbr r jsdF"!,0!\!\!\!\!\F444l666("""\"\(#5#5#5#5#5JRR\RF5F5F5F5F5P    0   \ 4\rdr )4r  __future__rr:rr|rrrqcopyrpathlibrnumpyr] torch.cudar ultralyticsrrultralytics.cfgr r ultralytics.engine.exporterr ultralytics.utilsr r rrrrrrrrultralytics.utils.checksrrrrrultralytics.utils.downloadsrultralytics.utils.filesrultralytics.utils.torch_utilsrrrrr rerdrbrs-:#"""""  ''''''''22222222666666qqqqqqqqqqqqqqqqqqqqqqqqmmmmmmmmmmmmmm555555------EEEEEEEE  $     ccccLJJJJJJJJZddddddddddrd