jM6ddlmZddlZddlZddlZddlmZddlmZddl Z ddl Z ddl m Z mZmZddlmZmZddlmZddlmZdd lmZgd d d d hdgddddhdge jdhdgddddhddgddddhdgdddd hdge jd!hdgd"d#d$d%hddd&ZGd'd(e jjZdGd-ZdHd0ZdId2Z Gd3d4e jjZ! dJdKdFZ"dS)L) annotationsN)Path)which)DetectPoseSegment)LOGGERWINDOWS)onnx_export_patch) make_anchors) copy_attr)submul_2add_14cat_19g~8CA) layer_namesweights_memoryn_layers)rrrcat_21cat_22mul_4add_15g\EBAiip)rrrrgffffBAi i )detectposeclassifysegment)rmuladd_6cat_15gffffuCA)add_7rcat_17r rr!cat_18gBAI)rr r!r&g.CA)YOLO11YOLOv8c*eZdZdZdfd ZdZxZS)FXModelaA custom model class for torch.fx compatibility. This class extends `torch.nn.Module` and is designed to ensure compatibility with torch.fx for tracing and graph manipulation. It copies attributes from an existing model and explicitly sets the model attribute to ensure proper copying. Attributes: model (nn.Module): The original model's layers. imgsz (tuple[int, int]): The input image size (height, width). r2ctt|||j|_||_dS)zInitialize the FXModel. Args: model (nn.Module): The original model to wrap for torch.fx compatibility. imgsz (tuple[int, int]): The input image size (height, width). Default is (640, 640). N)super__init__r modelimgsz)selfr6r7 __class__s a/home/longshao/multi-rider-rag/.venv/lib/python3.11/site-packages/ultralytics/utils/export/imx.pyr5zFXModel.__init__Is? $[  cg|jD]Yjdkr;tjtr jnfdjDttrqt jt_dttj fd|j Ddj dD\_ _tt urt jt"_tt&urt jt(_[S)aXForward pass through the model. This method performs the forward pass through the model, handling the dependencies between layers and saving intermediate outputs. Args: x (torch.Tensor): The input tensor to the model. Returns: (torch.Tensor): The output tensor from the model. c0g|]}|dkrn|Sr=).0jxys r: z#FXModel.forward..fs*8]8]8]TUa2gg1Q48]8]8]r;c3BK|]}|ddVdS)rN) transpose)rArCs r: z"FXModel.forward..isD((KK1%%((((((r;cJg|]}|jdz  Sr?)stride unsqueeze)rAsms r:rEz#FXModel.forward..ls."R"R"R!1qx'9'9"'='=#="R"R"Rr;rG)dimg?)r6f isinstanceintrtypes MethodType _inferencer torchcatr7rKanchorsstridestyper pose_forwardforwardrsegment_forwardappend)r8rCrNrDs `@@r:r\zFXModel.forwardVs[   Asbyy(c22]AacFF8]8]8]8]8]YZY\8]8]8]!V$$ $/ A>> (() "R"R"R"Rtz"R"R"RXYZZZ\]\dfi((($ 19 Aww$!,\1== Aww'!!!,_a@@ !A HHQKKKKr;)r1__name__ __module__ __qualname____doc__r5r\ __classcell__r9s@r:r0r0=sV        r;r0rCdict[str, torch.Tensor]return!tuple[torch.Tensor, torch.Tensor]c*|||d|jd|jz}|dd|ddddfS)z5Decode boxes and cls scores for imx object detection.boxesrrGscores) decode_bboxesdflrXrLrYrHsigmoidpermute)r8rCdboxs r:rUrUxs{   dhhqz22DL4J4J14M4M N NQUQ] ]D >>!Q  8!4!4!6!6!>!>q!Q!G!G GGr;list[torch.Tensor]/tuple[torch.Tensor, torch.Tensor, torch.Tensor]cpdjdtdjtjfdt jDd}tdr~jjkrn|jd}| j dj ddz|}|ddddddddf}| j|}tj  |}g|dddRS) zBForward pass for imx pose estimation, including keypoint decoding.r nk_outputcrg|]3}j||d4Sr?)cv4view)rAibsnk_outr8rCs r:rEz pose_forward..s?TTT[TXa[1&&++B;;TTTr;r=rGrkN)shapegetattrnkrVrWrangenlhasattrrurx kpt_shaperr\ kpts_decoderp)r8rCkptspatialpred_kptrzr{s`` @@r:r[r[~s9 1AB T; 0 0F )TTTTTTTU47^^TTTVX Y YCt[!!-dn&?&?)B-hhr4>!,dnQ.?!.CWMM!!!QQQQQQ,hhr47G,,tQA$$H (A (x1a(( ( ((r;=tuple[torch.Tensor, torch.Tensor, torch.Tensor, torch.Tensor]c&d}|jdtjfdt jDd}t jg|dd|RS)z"Forward pass for imx segmentation.rc|g|]8}j||jd9Sr?)rwrxnm)rAryrzr8rCs r:rEz#segment_forward..sATTTKDHQK!%%**2tw;;TTTr;rkrG) protor}rVrWrrrr\rH)r8rCpmcrzs`` @r:r]r]s 1Q4A B TTTTTTU47^^TTTVW X XBtQA $A $r||Aq!! $1 $ $$r;c4eZdZdZ ddfd ZdZxZS) NMSWrapperz?Wrap PyTorch Module with multiclass_nms layer from edge-mdt-cl.MbP?ffffff?,rr6torch.nn.Modulescore_thresholdfloat iou_thresholdmax_detectionsrRtaskstrct||_||_||_||_||_dS)aInitialize NMSWrapper with PyTorch Module and NMS parameters. Args: model (torch.nn.Module): Model instance. score_threshold (float): Score threshold for non-maximum suppression. iou_threshold (float): Intersection over union threshold for non-maximum suppression. max_detections (int): The number of detections to return. task (str): Task type, one of 'detect', 'pose', or 'segment'. N)r4r5r6rrrr)r8r6rrrrr9s r:r5zNMSWrapper.__init__sE"  .*, r;c ddlm}||}|d|d}}||||j|j|j}|jdkrs|d}tj|d|j d dd| d}|j |j|j|fS|jdkr||d|d } } tj| d|j d dd| d} |j |j|j| | fS|j |j|j|jfS) z:Forward pass with model inference and NMS post-processing.r)multiclass_nms_with_indicesrG)rjrlrrrrrkr=r)'edgemdt_cl.pytorch.nms.nms_with_indicesrr6rrrrrVgatherindicesrLexpandsizerjrllabelsn_valid) r8imagesroutputsrjrl nms_outputskptsout_kptsrrout_mcs r:r\zNMSWrapper.forwards{WWWWWW**V$$ GAJv11 0,.     9  1:D|D![-@-J-J2-N-N-U-UVXZ\^b^g^ghj^k^k-l-lmmH$k&8+:LhV V 9 ! ! GAJB\"a)<)F)Fr)J)J)Q)QRTVXZ\ZaZabdZeZe)f)fggF$k&8+:LfV[[ [ +"4k6H+J]]]r;)rrrr) r6rrrrrrrRrrr_res@r:rrslII "'"! 0^^^^^^^r;rFr6rfile Path | strconfrioumax_detrRmetadata dict | Nonegptqboolprefixrc ddl} ddl} ddlm} t jd|d| jd|fd} | dd } | j}td | vrd nd |j }tt||dvrtd|dD];}|| jjj|gd<| j| jd| jd|}| j|d}|rA| j|| || jddd|| dn$| j|| ||| d}|j dkrt;||pd|||j }t=t?| |j!d!}|"d"|t=t?|j# |j!d#z }tI5| j%&||| $dddn #1swxYwY| '|}|(D]:\}}|j)*}|t?|c|_+|_,;| -||t=t\j/j0}|tbrd%nd&z }|2stgd&}|stid'tkj6t?|d(t?|d)t?|d*d+gd,to|d-z d.d/05}|8d1|j9(Ddddn #1swxYwY|S)2aExport YOLO model to IMX format for deployment on Sony IMX500 devices. This function quantizes a YOLO model using Model Compression Toolkit (MCT) and exports it to IMX format compatible with Sony IMX500 edge devices. It supports both YOLOv8n and YOLO11n models for detection, segmentation, pose estimation, and classification tasks. Args: model (torch.nn.Module): The YOLO model to export. Must be YOLOv8n or YOLO11n. file (Path | str): Output file path for the exported model. conf (float): Confidence threshold for NMS post-processing. iou (float): IoU threshold for NMS post-processing. max_det (int): Maximum number of detections to return. metadata (dict | None, optional): Metadata to embed in the ONNX model. Defaults to None. gptq (bool, optional): Whether to use Gradient-Based Post Training Quantization. If False, uses standard Post Training Quantization. Defaults to False. dataset (optional): Representative dataset for quantization calibration. Defaults to None. prefix (str, optional): Logging prefix string. Defaults to "". Returns: (Path): Path to the exported IMX model directory. Raises: ValueError: If the model is not a supported YOLOv8n or YOLO11n variant. Examples: >>> from ultralytics import YOLO >>> model = YOLO("yolo11n.pt") >>> path = torch2imx(model, "model.imx", conf=0.25, iou=0.7, max_det=300) Notes: - Requires model_compression_toolkit, onnx, edgemdt_tpc, and edge-mdt-cl packages - Only supports YOLOv8n and YOLO11n models (detection, segmentation, pose, and classification tasks) - Output includes quantized ONNX model, IMX binary, and labels.txt file rN) get_target_platform_capabilities z0 starting export with model_compression_toolkit z...c38K|D]}|d}|dz }|gVdS)Nimggo@r@) dataloaderbatchrs r:representative_dataset_genz-torch2imx..representative_dataset_gens>  E,C+C%KKKK  r;z4.0imx500) tpc_version device_typeC2PSAr-r.rz9IMX export only supported for YOLOv8n and YOLO11n models.r ) num_of_imagesT)concat_threshold_update)mixed_precision_configquantization_configbit_width_configr)riF)n_epochsuse_hessian_based_weightsuse_hessian_sample_attention)r6representative_data_gentarget_resource_utilization gptq_config core_configtarget_platform_capabilities) in_modulerrrrrr)r6rrrr _imx_model)exist_okz _imx.onnx)r6save_model_path repr_datasetzimxconv-pt.exez imxconv-ptzDimxconv-pt not found. Install with: pip install imx500-converter[pt]z-iz-oz--no-input-persistencyz--overwrite-output)checkz labels.txtwzutf-8)encodingcg|] \}}|d S)rr@)rA_names r:rEztorch2imx..Ws"HHHDDHHHr;):model_compression_toolkitonnx edgemdt_tpcrr info __version__coreBitWidthConfig MCT_CONFIG__str__rlenlistmodules ValueErrorset_manual_activation_bit_widthcommonnetwork_editorsNodeNameFilter CoreConfig MixedPrecisionQuantizationConfigQuantizationConfigResourceUtilizationr+pytorch_gradient_post_training_quantizationget_pytorch_gptq_configptq"pytorch_post_training_quantizationrrrreplacesuffixmkdirrr exporterpytorch_export_modelloaditemsmetadata_propsaddkeyvaluesavesys executableparentr existsrFileNotFoundError subprocessrunopen writelinesnames)r6rrrrrrdatasetrmctrrrtpcbit_cfg mct_config layer_nameconfigresource_utilization quant_modelrP onnx_model model_onnxkvmetabin_dirimxconvs r: torch2imxrs8Z,+++KKK<<<<<< KaVaaS_aaabbb.5 + *u( S S SCh%%''G5==??(B(BHHQRWR\]J 4  !!J)???TUUU /rr //1P1_1_`j1k1k0lnpqqqq X "xHHWYHZZH77PT7UU !F 877zRbGc7dd  <<$>(<88]b9), =   W 7 7$>(<), 8   * zZ  ME"     SYY  t{L 9 9::AGGTGT#di..00kJJKKKJ      ))zHb *                  :&&J  ))1(,,.. #a&&$**IIj*%%%3>"")GWF)),GG >>  & %% h fgggN WtS__dCFFr*s#"""""  8888888888--------777777......333333 @??*c   ]\\(c   %'"&seTT???'c   (=<<'c   YXX(c   %'"&rdSS<<<'c   '%% P88888eho888vHHHH ))))"%%%%1^1^1^1^1^1^1^1^t! M M M M M M M r;