j(ddlmZddlZddlmZddlZddlmZmZddl m Z ddgdgd fd+dZ d,d-d*Z dS).) annotationsN)Path) IS_JETSONLOGGER) TORCH_2_4imagesoutput0F torch_modeltorch.nn.Moduleim torch.Tensor onnx_filestropsetint input_names list[str] output_namesdynamic bool | dictreturnNonec dtrddini}tjj|||fd|d|||pdd|dS)acExport a PyTorch model to ONNX format. Args: torch_model (torch.nn.Module): The PyTorch model to export. im (torch.Tensor): Example input tensor for the model. onnx_file (str): Path to save the exported ONNX file. opset (int): ONNX opset version to use for export. input_names (list[str]): List of input tensor names. output_names (list[str]): List of output tensor names. dynamic (bool | dict, optional): Whether to enable dynamic axes. Notes: Setting `do_constant_folding=True` may cause issues with DNN inference for torch>=1.12. dynamoFTN)verbose opset_versiondo_constant_foldingrr dynamic_axes)rtorchonnxexport)r r rrrrrkwargss d/home/longshao/multi-rider-rag/.venv/lib/python3.11/site-packages/ultralytics/utils/export/engine.py torch2onnxr%sn.#, 3h  F J   !_        r* engine_file str | None workspace int | Nonehalfboolint8shapetuple[int, int, int, int]dlametadata dict | Nonerprefixc  ddl|p!t|d}jj} | rjjj| _| } | }tpddz}tj ddddk}|r'|dkr!| jj|n |dkr||_dtjjz}| || jo|}| jo|}t,st/dt1j| d d |s|st/d jj|_t|_|jj !| }|"|stGd |fd tIj%D}fdtIj&D}|D]0}t1j| d|j'd|j(d|j)1|D]0}t1j| d|j'd|j(d|j)1|r| *}d|dddf}g|ddfd|ddDR}|D] }|+|j'|||!|,||r|s|-|t1j| d|rdnd|rdndzd||r|jj.j/j0|_1Gfddj2}||tgt|d |_4n!|r|jj5|r| 6|}|tGd!to|d"5}| stqj9| }|:tw|<d#d$d%&|:|=|:|ddddS#1swxYwYdS| >|5}to|d"5}| stqj9| }|:tw|<d#d$d%&|:|=|:|?dddn #1swxYwYddddS#1swxYwYdS)'a'Export a YOLO model to TensorRT engine format. Args: onnx_file (str): Path to the ONNX file to be converted. engine_file (str | None): Path to save the generated TensorRT engine file. workspace (int | None): Workspace size in GB for TensorRT. half (bool, optional): Enable FP16 precision. int8 (bool, optional): Enable INT8 precision. dynamic (bool, optional): Enable dynamic input shapes. shape (tuple[int, int, int, int], optional): Input shape (batch, channels, height, width). dla (int | None): DLA core to use (Jetson devices only). dataset (ultralytics.data.build.InfiniteDataLoader, optional): Dataset for INT8 calibration. metadata (dict | None): Metadata to include in the engine file. verbose (bool, optional): Enable verbose logging. prefix (str, optional): Prefix for log messages. Raises: ValueError: If DLA is enabled on non-Jetson devices or required precision is not set. RuntimeError: If the ONNX file cannot be parsed. Notes: TensorRT version compatibility is handled for workspace size and engine building. INT8 calibration requires a dataset and generates a calibration cache. Metadata is serialized and written to the engine file if provided. rNz.enginei@.r( z.DLA is only available on NVIDIA Jetson devicesz enabling DLA on core z...ztDLA requires either 'half=True' (FP16) or 'int8=True' (INT8) to be enabled. Please enable one of them and try again.zfailed to load ONNX file: c:g|]}|S) get_input.0inetworks r$ zonnx2engine..s' F F Fqg"" F F Fr&c:g|]}|Sr=) get_outputr?s r$rCzonnx2engine..s'IIIw!!!$$IIIr&z input "z " with shape z output " c3\K|]&}ttdpd|zV'dS)rHN)rmax)r@dr.s r$ zonnx2engine..s>"V"Vq3s1in1'='='A#B#B"V"V"V"V"V"Vr&)minoptrJz building INT8FP1632z engine as cNeZdZdZ ddfd Zdd Zdd Zdd ZddZddZ dS)%onnx2engine..EngineCalibratoraCustom INT8 calibrator for TensorRT engine optimization. This calibrator provides the necessary interface for TensorRT to perform INT8 quantization calibration using a dataset. It handles batch generation, caching, and calibration algorithm selection. Attributes: dataset: Dataset for calibration. data_iter: Iterator over the calibration dataset. algo (trt.CalibrationAlgoType): Calibration algorithm type. batch (int): Batch size for calibration. cache (Path): Path to save the calibration cache. Methods: get_algorithm: Get the calibration algorithm to use. get_batch_size: Get the batch size to use for calibration. get_batch: Get the next batch to use for calibration. read_calibration_cache: Use existing cache instead of calibrating again. write_calibration_cache: Write calibration cache to disk. r+cacherrrcj|||_t||_ jjn jj|_|j |_ t||_ dS)z;Initialize the INT8 calibrator with dataset and cache path.N) IInt8Calibrator__init__datasetiter data_iterCalibrationAlgoTypeENTROPY_CALIBRATION_2MINMAX_CALIBRATIONalgo batch_sizebatchrrU)selfrYrUr5trts r$rXz.onnx2engine..EngineCalibrator.__init__sr #,,T222& !%g+AA0C %/ !%[[ r&trt.CalibrationAlgoTypec|jS)z%Get the calibration algorithm to use.)r_rbs r$ get_algorithmz3onnx2engine..EngineCalibrator.get_algorithms y r&rc|jpdS)z*Get the batch size to use for calibration.r()rarfs r$get_batch_sizez4onnx2engine..EngineCalibrator.get_batch_sizeszQ&r&list[int] | Nonec t|jddz }|jjdkr|dn|}t |gS#t$rYdSwxYw)zOGet the next batch to use for calibration, as a list of device memory pointers.imggo@cpucudaN)nextr[devicetypetordata_ptr StopIteration)rbnamesim0ss r$ get_batchz/onnx2engine..EngineCalibrator.get_batchs} //6>D.2k.>%.G.G4776???TD 0011$   44 sA%A(( A65A6 bytes | Nonec|jr)|jjdkr|jSdSdS)zSUse existing cache instead of calibrating again, otherwise, implicitly return None..cacheN)rUexistssuffix read_bytesrfs r$read_calibration_cachez.EngineCalibrator.read_calibration_cachesK:$$&&34:+<+H+H:0022233+H+Hr&bytesc:|j|}dS)z Write calibration cache to disk.N)rU write_bytes)rbrU_s r$write_calibration_cachez=onnx2engine..EngineCalibrator.write_calibration_cachesJ**511r&N)r+)rUrrr)rrd)rr)rrj)rrx)rUrrr) __name__ __module__ __qualname____doc__rXrgrirwr~r)r5rcsr$EngineCalibratorrTs  .  ) ) ) ) ) ) ) )" ! ! ! ! ' ' ' '     3 3 3 3  2 2 2 2 2 2r&rrz)rYrUz3TensorRT engine build failed, check logs for errorswblittleT) byteordersigned)@tensorrtr with_suffixLoggerINFOSeverityVERBOSE min_severityBuildercreate_builder_configr __version__splitset_memory_pool_limitMemoryPoolType WORKSPACEmax_workspace_sizeNetworkDefinitionCreationFlagEXPLICIT_BATCHcreate_networkplatform_has_fast_fp16platform_has_fast_int8r ValueErrorrinfo DeviceTypeDLAdefault_device_typeDLA_coreset_flag BuilderFlag GPU_FALLBACK OnnxParserparse_from_file RuntimeErrorrange num_inputs num_outputsnamer3dtypecreate_optimization_profile set_shapeadd_optimization_profileset_calibration_profilerOProfilingVerbosityDETAILEDprofiling_verbosityrWrint8_calibratorFP16build_serialized_networkopenjsondumpswritelento_bytesencode build_engine serialize) rr,r.r0r2rr3r5rYr6rr8loggerbuilderconfigworkspace_bytesis_trt10flagparserinputsoutputsinpoutprofile min_shape max_shaperenginetmetarBrcs ` ` @@r$ onnx2enginer4sDNGi!g677O3?((a00344:H4Oa''$$S%7%A?SSSS 1  $3! C5DEE ED$$T**G  ) 2dD  ) 2dD  OMNN N v==S===>>> D G &)^%7"c((4555^^GV , ,F  ! !) , ,EC CCDDDG F F FE'2D,E,E F F FFIIIIeG4G.H.HIIIGVV vTTsxTTSYTTTTUUUUWW vUUUUciUU#)UUVVVV45577a"b) WeBQBiW"V"V"V"VERSRTRTI"V"V"VWW  Q QC   chI5i  P P P P''000  4 4  * *7 3 3 3 K6rrt%XVVAWSW9Xrreprrsss L.,---%(%;%D"? 2? 2? 2? 2? 2? 2? 2? 2s2? 2? 2? 2D"2!1d9oo11(;;<<" " "  .,---(11'6BB >TUU U +t $ $ #z(++D **1*NNOOO &&& GGFOOO                    ! !'6 2 2 (fd;PT>U>U (YZ#z(++D **1*NNOOO &&& GGF$$&& ' ' '  ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( (sJB S//S36S3W$#BW  W$ W W$W W$$W(+W()r r r rrrrrrrrrrrrr) NNFFFr'NNNFr+)rrr,r-r.r/r0r1r2r1rr1r3r4r5r/r6r7rr1r8rrr) __future__rrpathlibrr ultralytics.utilsrrultralytics.utils.torch_utilsrr%rr=r&r$rs#"""""  ////////333333&Z(k #####P# '7  B(B(B(B(B(B(B(r&