/j|dZddlmZddlZddlZddlmZddlmcmZ ddl m Z m Z ddl mZddlmZddlmZmZmZd ZGd d ejZGd d eZGddejZGddejZGddejZGddejZGddejZGddejZGddejZGddejZ dS)zTransformer modules.) annotationsN) constant_xavier_uniform_) TORCH_1_11)Conv) _get_clonesinverse_sigmoid#multi_scale_deformable_attn_pytorch) AIFIMLPDeformableTransformerDecoder!DeformableTransformerDecoderLayer LayerNorm2dMLPBlock MSDeformAttnTransformerBlockTransformerEncoderLayerTransformerLayerceZdZdZdddejdfdfd Zedd dZ d!d"dZ d!d"dZ d!d"dZ xZ S)#raA single layer of the transformer encoder. This class implements a standard transformer encoder layer with multi-head attention and feedforward network, supporting both pre-normalization and post-normalization configurations. Attributes: ma (nn.MultiheadAttention): Multi-head attention module. fc1 (nn.Linear): First linear layer in the feedforward network. fc2 (nn.Linear): Second linear layer in the feedforward network. norm1 (nn.LayerNorm): Layer normalization after attention. norm2 (nn.LayerNorm): Layer normalization after feedforward network. dropout (nn.Dropout): Dropout layer for the feedforward network. dropout1 (nn.Dropout): Dropout layer after attention. dropout2 (nn.Dropout): Dropout layer after feedforward network. act (nn.Module): Activation function. normalize_before (bool): Whether to apply normalization before attention and feedforward. Fc1intcm num_headsdropoutfloatact nn.Modulenormalize_beforeboolc.tddlm}|st dt j|||d|_t j|||_ t j|||_ t j ||_ t j ||_ t j||_t j||_t j||_||_||_dS)aInitialize the TransformerEncoderLayer with specified parameters. Args: c1 (int): Input dimension. cm (int): Hidden dimension in the feedforward network. num_heads (int): Number of attention heads. dropout (float): Dropout probability. act (nn.Module): Activation function. normalize_before (bool): Whether to apply normalization before attention and feedforward. ) TORCH_1_9z]TransformerEncoderLayer() requires torch>=1.9 to use nn.MultiheadAttention(batch_first=True).T)r batch_firstN)super__init__utils.torch_utilsr&ModuleNotFoundErrornnMultiheadAttentionmaLinearfc1fc2 LayerNormnorm1norm2Dropoutrdropout1dropout2r r") selfrrrrr r"r& __class__s g/home/longshao/multi-rider-rag/.venv/lib/python3.11/site-packages/ultralytics/nn/modules/transformer.pyr)z TransformerEncoderLayer.__init__3s& 222222 %o 'IwTXYYY9R$$9R$$\"%% \"%% z'**  7++  7++  0Ntensor torch.Tensorpostorch.Tensor | Nonereturnc||n||zS)z2Add position embeddings to the tensor if provided.r<r>s r:with_pos_embedz&TransformerEncoderLayer.with_pos_embed[vv&3,6r;srcsrc_masksrc_key_padding_maskc |||x}}||||||d}|||z}||}|||||}|||z}| |S)aPerform forward pass with post-normalization. Args: src (torch.Tensor): Input tensor. src_mask (torch.Tensor, optional): Mask for the src sequence. src_key_padding_mask (torch.Tensor, optional): Mask for the src keys per batch. pos (torch.Tensor, optional): Positional encoding. Returns: (torch.Tensor): Output tensor after attention and feedforward. value attn_maskkey_padding_maskr) rDr.r6r3r1rr r0r7r4)r8rFrGrHr>qksrc2s r: forward_postz$TransformerEncoderLayer.forward_post`s$##C---Awwq!3(MawbbcdeDMM$'''jjooxx TXXdhhsmm%<%<==>>DMM$'''zz#r;c ||}|||x}}||||||d}|||z}||}|||||}|| |zS)aPerform forward pass with pre-normalization. Args: src (torch.Tensor): Input tensor. src_mask (torch.Tensor, optional): Mask for the src sequence. src_key_padding_mask (torch.Tensor, optional): Mask for the src keys per batch. pos (torch.Tensor, optional): Positional encoding. Returns: (torch.Tensor): Output tensor after attention and feedforward. rJr) r3rDr.r6r4r1rr r0r7)r8rFrGrHr>rPrNrOs r: forward_prez#TransformerEncoderLayer.forward_prezs$zz###D#...Awwq!48NbwccdefDMM$'''zz#xx TXXdhhtnn%=%=>>??T]]4((((r;cp|jr|||||S|||||S)aForward propagate the input through the encoder module. Args: src (torch.Tensor): Input tensor. src_mask (torch.Tensor, optional): Mask for the src sequence. src_key_padding_mask (torch.Tensor, optional): Mask for the src keys per batch. pos (torch.Tensor, optional): Positional encoding. Returns: (torch.Tensor): Output tensor after transformer encoder layer. )r"rSrQ)r8rFrGrHr>s r:forwardzTransformerEncoderLayer.forwardsF$   N##C3GMM M  h0DcJJJr; rrrrrrrrr r!r"r#Nr<r=r>r?r@r=NNN) rFr=rGr?rHr?r>r?r@r=) __name__ __module__ __qualname____doc__r,GELUr) staticmethodrDrQrSrU __classcell__r9s@r:rr s* !&&1&1&1&1&1&1&1P7777\7)-48#' :)-48#' ))))):)-48#' KKKKKKKKKr;rcreZdZdZdddejdfdfd Zdfd Ze dddZ xZ S) r zAIFI transformer layer for 2D data with positional embeddings. This class extends TransformerEncoderLayer to work with 2D feature maps by adding 2D sine-cosine positional embeddings and handling the spatial dimensions appropriately. rrrFrrrrrrr r!r"r#cTt||||||dS)aInitialize the AIFI instance with specified parameters. Args: c1 (int): Input dimension. cm (int): Hidden dimension in the feedforward network. num_heads (int): Number of attention heads. dropout (float): Dropout probability. act (nn.Module): Activation function. normalize_before (bool): Whether to apply normalization before attention and feedforward. N)r(r))r8rrrrr r"r9s r:r)z AIFI.__init__s.& RGS:JKKKKKr;xr=r@c|jdd\}}}||||}t|dddd||j|j}|ddd d|||g S)zForward pass for the AIFI transformer layer. Args: x (torch.Tensor): Input tensor with shape [B, C, H, W]. Returns: (torch.Tensor): Output tensor with shape [B, C, H, W]. rNr)devicedtype)r>) shape"build_2d_sincos_position_embeddingr(rUflattenpermutetorgrhview contiguous)r8rdchw pos_embedr9s r:rUz AIFI.forwards'!""+1a;;Aq!DD GGOOAIIaLL00Aq99y||STS[cdcj|?k?kO l lyyAq!!&&Aq!}55@@BBBr;@rsrr embed_dim temperaturec|dzdks Jdtj|tj}tj|tj}trtj||dntj||\}}|dz}tj|tj|z }d||zz }|d|d z}|d|d z} tjtj|tj|tj| tj| gd d S) a~Build 2D sine-cosine position embedding. Args: w (int): Width of the feature map. h (int): Height of the feature map. embed_dim (int): Embedding dimension. temperature (float): Temperature for the sine/cosine functions. Returns: (torch.Tensor): Position embedding with shape [1, h*w, embed_dim]. rzHEmbed dimension must be divisible by 4 for 2D sin-cos position embeddingrhij)indexingg?.NNr) torcharangefloat32rmeshgridrlcatsincos) rsrrrwrxgrid_wgrid_hpos_dimomegaout_wout_hs r:rkz'AIFI.build_2d_sincos_position_embeddings<1}!!!#m!!!au}555au}555JTxFFFFZ_ZhioqwZxZxq. WEM:::WD{E)*  +eDk9  +eDk9y%)E**EIe,<,N>NPUPYZ_P`P`acdeefjkkr;rVrdr=r@r=)rurv) rsrrrrrwrrxrr@r=) rZr[r\r]r,r^r)rUr_rkr`ras@r:r r s !&LLLLLLL* C C C C C CCJllll\lllllr;r c,eZdZdZd fd Zd d ZxZS) rzeTransformer layer https://arxiv.org/abs/2010.11929 (LayerNorm layers removed for better performance).rqrrcttj||d|_tj||d|_tj||d|_tj|||_tj||d|_ tj||d|_ dS)zInitialize a self-attention mechanism using linear transformations and multi-head attention. Args: c (int): Input and output channel dimension. num_heads (int): Number of attention heads. F)bias)rwrN) r(r)r,r/rNrOvr-r.r0r1)r8rqrr9s r:r)zTransformerLayer.__init__s 1ae,,,1ae,,,1ae,,,'!yIII9Q...9Q...r;rdr=r@c |||||||d|z}||||zS)zApply a transformer block to the input x and return the output. Args: x (torch.Tensor): Input tensor. Returns: (torch.Tensor): Output tensor after transformer layer. r)r.rNrOrr1r0r8rds r:rUzTransformerLayer.forwardsb GGDFF1IItvvayy$&&)) 4 4Q 7! ;xx $$q((r;)rqrrrrrZr[r\r]r)rUr`ras@r:rrsWoo / / / / / / ) ) ) ) ) ) ) )r;rc,eZdZdZd fd Zd d ZxZS)raVision Transformer block based on https://arxiv.org/abs/2010.11929. This class implements a complete transformer block with optional convolution layer for channel adjustment, learnable position embedding, and multiple transformer layers. Attributes: conv (Conv, optional): Convolution layer if input and output channels differ. linear (nn.Linear): Learnable position embedding. tr (nn.Sequential): Sequential container of transformer layers. c2 (int): Output channel dimension. rrc2r num_layersc,td|_|krt||_t j|_t jfdt|D|_ |_ dS)aLInitialize a Transformer module with position embedding and specified number of heads and layers. Args: c1 (int): Input channel dimension. c2 (int): Output channel dimension. num_heads (int): Number of attention heads. num_layers (int): Number of transformer layers. Nc38K|]}tVdSrW)r).0_rrs r: z,TransformerBlock.__init__..-s.!]!]a"22y"A"A!]!]!]!]!]!]r;) r(r)convrr,r/linear Sequentialrangetrr)r8rrrrr9s `` r:r)zTransformerBlock.__init__s  88R DIiB'' -!]!]!]!]!]5Q[K\K\!]!]!]^r;rdr=r@c\|j||}|j\}}}}|dddd}||||zddd||j||S)zForward propagate the input through the transformer block. Args: x (torch.Tensor): Input tensor with shape [b, c1, h, w]. Returns: (torch.Tensor): Output tensor with shape [b, c2, h, w]. Nrfrr)rrjrlrmrrreshaper)r8rdbrrrrsps r:rUzTransformerBlock.forward0s 9  ! AW 1a IIaLL Aq ) )wwq4;;q>>)**221a;;CCAtwPQSTUUUr;)rrrrrrrrrrras@r:rrsc  " V V V V V V V Vr;rc:eZdZdZejfd fd Zd d ZxZS) rz+A single block of a multi-layer perceptron. embedding_dimrmlp_dimcttj|||_tj|||_||_dS)aInitialize the MLPBlock with specified embedding dimension, MLP dimension, and activation function. Args: embedding_dim (int): Input and output dimension. mlp_dim (int): Hidden dimension. act (type): Activation function class. N)r(r)r,r/lin1lin2r )r8rrr r9s r:r)zMLPBlock.__init__CsR ImW55 Ig}55 355r;rdr=r@cx||||S)zForward pass for the MLPBlock. Args: x (torch.Tensor): Input tensor. Returns: (torch.Tensor): Output tensor after MLP block. )rr rrs r:rUzMLPBlock.forwardPs,yy$))A,,//000r;)rrrrr) rZr[r\r]r,r^r)rUr`ras@r:rr@sa55=?W        1 1 1 1 1 1 1 1r;rc@eZdZdZejdddfdfd ZddZxZS)r aA simple multi-layer perceptron (also called FFN). This class implements a configurable MLP with multiple linear layers, activation functions, and optional sigmoid output activation. Attributes: num_layers (int): Number of layers in the MLP. layers (nn.ModuleList): List of linear layers. sigmoid (bool): Whether to apply sigmoid to the output. act (nn.Module): Activation function. FN input_dimr hidden_dim output_dimrsigmoidr#residualout_normr!c t||_|g|dz z} tjdt |g| g| |D|_||_||_|r||krtd||_ t|tj s|J|ptj |_dS)a7Initialize the MLP with specified input, hidden, output dimensions and number of layers. Args: input_dim (int): Input dimension. hidden_dim (int): Hidden dimension. output_dim (int): Output dimension. num_layers (int): Number of layers. act (type): Activation function class. sigmoid (bool): Whether to apply sigmoid to the output. residual (bool): Whether to use residual connections. out_norm (nn.Module, optional): Normalization layer for the output. rc3FK|]\}}tj||VdSrW)r,r/)rnrOs r:rzMLP.__init__..s0#g#g1BIaOO#g#g#g#g#g#gr;z5residual is only supported if input_dim == output_dimN)r(r)rr, ModuleListziplayersrr ValueErrorr isinstanceModuleIdentityr) r8rrrrr rrrrrr9s r:r)z MLP.__init__is. $ LJN +m#g#gYOQROUeWXUeZdUe@f@f#g#g#ggg  355  V Z//TUU U  (BI..B(2B2BB 1BKMM r;rdr=r@c|}t|jD]R\}}||jdz kr4t|dt j||n ||}St|ddr||z}t|dt j|}t|ddr|n|S)zForward pass for the entire MLP. Args: x (torch.Tensor): Input tensor. Returns: (torch.Tensor): Output tensor after MLP. rr rFrr) enumeraterrgetattrr,ReLUrr)r8rdorig_xilayers r:rUz MLP.forwards!$+.. c cHAu=>STAT=T=T/eRWYY//a999Z_Z_`aZbZbAA 4U + + F A 4GD*bkmm 4 4Q 7 7%dIu==Dqyy{{{1Dr;)rrrrrrrrrr#rr#rr!r) rZr[r\r]r,rr)rUr`ras@r:r r \sy  $ G""2"2"2"2"2"2"2HEEEEEEEEr;r c.eZdZdZd d fd Zdd ZxZS)rap2D Layer Normalization module inspired by Detectron2 and ConvNeXt implementations. This class implements layer normalization for 2D feature maps, normalizing across the channel dimension while preserving spatial dimensions. Attributes: weight (nn.Parameter): Learnable scale parameter. bias (nn.Parameter): Learnable bias parameter. eps (float): Small constant for numerical stability. References: https://github.com/facebookresearch/detectron2/blob/main/detectron2/layers/batch_norm.py https://github.com/facebookresearch/ConvNeXt/blob/main/models/convnext.py ư> num_channelsrepsrcttjt j||_tjt j||_||_ dS)zInitialize LayerNorm2d with the given parameters. Args: num_channels (int): Number of channels in the input. eps (float): Small constant for numerical stability. N) r(r)r, Parameterronesweightzerosrr)r8rrr9s r:r)zLayerNorm2d.__init__s\ l5:l#;#;<< L\!:!:;; r;rdr=r@c|dd}||z ddd}||z tj||jzz }|jddddf|z|jddddfzS)zPerform forward pass for 2D layer normalization. Args: x (torch.Tensor): Input tensor. Returns: (torch.Tensor): Normalized output tensor. rTkeepdimrfN)meanpowrsqrtrrr)r8rduss r:rUzLayerNorm2d.forwards FF1dF # # UKKNN  4  0 0 UejTX.. .{111dD=)A- !!!T4-0HHHr;)r)rrrrrrras@r:rrsh          I I I I I I I Ir;rc8eZdZdZddfd Zd Z dddZxZS)raMultiscale Deformable Attention Module based on Deformable-DETR and PaddleDetection implementations. This module implements multiscale deformable attention that can attend to features at multiple scales with learnable sampling locations and attention weights. Attributes: im2col_step (int): Step size for im2col operations. d_model (int): Model dimension. n_levels (int): Number of feature levels. n_heads (int): Number of attention heads. n_points (int): Number of sampling points per attention head per feature level. sampling_offsets (nn.Linear): Linear layer for generating sampling offsets. attention_weights (nn.Linear): Linear layer for generating attention weights. value_proj (nn.Linear): Linear layer for projecting values. output_proj (nn.Linear): Linear layer for projecting output. References: https://github.com/fundamentalvision/Deformable-DETR/blob/main/models/ops/modules/ms_deform_attn.py rurzrd_modelrn_levelsn_headsn_pointsct||zdkrtd|d|||z}||z|ks Jdd|_||_||_||_||_tj |||z|zdz|_ tj |||z|z|_ tj |||_ tj |||_ |dS)a>Initialize MSDeformAttn with the given parameters. Args: d_model (int): Model dimension. n_levels (int): Number of feature levels. n_heads (int): Number of attention heads. n_points (int): Number of sampling points per attention head per feature level. rz.d_model must be divisible by n_heads, but got z and z(`d_model` must be divisible by `n_heads`@rfN)r(r)r im2col_steprrrrr,r/sampling_offsetsattention_weights value_proj output_proj_reset_parameters)r8rrrr _d_per_headr9s r:r)zMSDeformAttn.__init__s  W  ! !egee\ceeff f( W$///1[///      " '7X3E3PST3T U U!#7Gh4F4Q!R!R)GW559Wg66      r;ct|jjjdt j|jtjdtj z|jz z}t j | | gd}|| dddz |jddd d|j|jd}t'|jD]}|d d d d |d d fxx|dzzcc< t j5t+j|d|j_d d d n #1swxYwYt|jjjdt|jjjdt3|jjjt|jjjdt3|jjjt|jjjdd S) zReset module parameters.rr{g@riTrrrrfN)rrrdatarrrrmathpistackrrabsmaxrorepeatrrrno_gradr,rrrrrr)r8thetas grid_initrs r:rzMSDeformAttn._reset_parameterssA$'.3S999dl%-@@@C$'MTXT`D`aKvzz|| >qA A T$,1a ( ( VAt}dmQ 7 7  t}%% + +A aaaAqqqj ! ! !QU * ! ! ! ! ]__ J J)+innR6H6H)I)ID ! & J J J J J J J J J J J J J J J$(/4c:::$(-2C888.3444$/&+S111(/4555$"',c22222s 2F  FFNqueryr= refer_bboxrK value_shapeslist value_maskr?r@c |jdd\}}|jd}td|D|ksJ||}|)||dt d}||||j|j|jz}|||||j|j |j d} | ||||j|j |j z} tj | d|||j|j |j } |jd} | dkrctj||j|jd} | | ddddddddfz } |ddddddddddf| z}nW| d kr>| |j z |ddddddddddfzd z} |ddddddddddf| z}nt'd | d t)|||| }||S) aPerform forward pass for multiscale deformable attention. Args: query (torch.Tensor): Query tensor with shape [bs, query_length, C]. refer_bbox (torch.Tensor): Reference bounding boxes with shape [bs, query_length, n_levels, 2 or 4], range in [0, 1], top-left (0,0), bottom-right (1, 1), including padding area. value (torch.Tensor): Value tensor with shape [bs, value_length, C]. value_shapes (list): List with shape [n_levels, 2], [(H_0, W_0), (H_1, W_1), ..., (H_{L-1}, W_{L-1})]. value_mask (torch.Tensor, optional): Mask tensor with shape [bs, value_length], True for padding elements, False for non-padding elements. Returns: (torch.Tensor): Output tensor with shape [bs, Length_{query}, C]. References: https://github.com/PaddlePaddle/PaddleDetection/blob/develop/ppdet/modeling/transformers/deformable_transformer.py Nrfrc38K|]}|d|dzVdS)rrNrB)rrs r:rz'MSDeformAttn.forward..-s.5511Q4!A$;555555r;r~rri)rhrgrzg?z5Last dim of reference_points must be 2 or 4, but got .)rjsumr masked_fillrrorrrrrrFsoftmaxr as_tensorrhrgfliprr r)r8rrrKrrbslen_qlen_vrr num_pointsoffset_normalizeraddsampling_locationsoutputs r:rUzMSDeformAttn.forwards2KO E A55 55555>>>>&&  !%%j&;U1XXFFE 2udlDLDL4PQQ0077<>r5$,X\XehlhuXuvvI&7<<AA"eT\[_[hjnjwxx%b) ?? % EKX]Xd e e e j jkm n n "%6tT4DRSRSRS7S%TTC!+AAAqqq$4,B!Cc!I   1__"T]2Z111dAAAtUVUWUW@W5XX[^^C!+AAAqqq$4!,C!Ds!J  bU_bbbcc c4ULJ\^opp'''r;)rurzrrz)rrrrrrrrrW) rr=rr=rKr=rrrr?r@r=)rZr[r\r]r)rrUr`ras@r:rrsy(!!!!!!!<3336+/ 0(0(0(0(0(0(0(0(0(r;rc~eZdZdZddddejddfd$fd Zed%dZd&dZ d'd(d#Z xZ S))ra6Deformable Transformer Decoder Layer inspired by PaddleDetection and Deformable-DETR implementations. This class implements a single decoder layer with self-attention, cross-attention using multiscale deformable attention, and a feedforward network. Attributes: self_attn (nn.MultiheadAttention): Self-attention module. dropout1 (nn.Dropout): Dropout after self-attention. norm1 (nn.LayerNorm): Layer normalization after self-attention. cross_attn (MSDeformAttn): Cross-attention module. dropout2 (nn.Dropout): Dropout after cross-attention. norm2 (nn.LayerNorm): Layer normalization after cross-attention. linear1 (nn.Linear): First linear layer in the feedforward network. act (nn.Module): Activation function. dropout3 (nn.Dropout): Dropout in the feedforward network. linear2 (nn.Linear): Second linear layer in the feedforward network. dropout4 (nn.Dropout): Dropout after the feedforward network. norm3 (nn.LayerNorm): Layer normalization after the feedforward network. References: https://github.com/PaddlePaddle/PaddleDetection/blob/develop/ppdet/modeling/transformers/deformable_transformer.py https://github.com/fundamentalvision/Deformable-DETR/blob/main/models/deformable_transformer.py rurirrzrrrd_ffnrrr r!rrcttj||||_tj||_tj||_t|||||_ tj||_ tj||_ tj |||_||_tj||_tj |||_tj||_tj||_dS)aInitialize the DeformableTransformerDecoderLayer with the given parameters. Args: d_model (int): Model dimension. n_heads (int): Number of attention heads. d_ffn (int): Dimension of the feedforward network. dropout (float): Dropout probability. act (nn.Module): Activation function. n_levels (int): Number of feature levels. n_points (int): Number of sampling points. )rN)r(r)r,r- self_attnr5r6r2r3r cross_attnr7r4r/linear1r dropout3linear2dropout4norm3) r8rrrrr rrr9s r:r)z*DeformableTransformerDecoderLayer.__init__^s* .wQQQ 7++ \'** 'w'8LL 7++ \'** y%00  7++ y00  7++ \'** r;r<r=r>r?r@c||n||zS)z;Add positional embeddings to the input tensor, if provided.rBrCs r:rDz0DeformableTransformerDecoderLayer.with_pos_embedrEr;tgtc |||||}|||z}||S)zPerform forward pass through the Feed-Forward Network part of the layer. Args: tgt (torch.Tensor): Input tensor. Returns: (torch.Tensor): Output tensor after FFN. )r r r r r r )r8rtgt2s r: forward_ffnz-DeformableTransformerDecoderLayer.forward_ffns_||DMM$((4<<3D3D*E*EFFGGDMM$'''zz#r;Nembedrfeatsshapesr padding_maskrL query_poscn|||x}} ||dd| dd|dd|ddd} ||| z}||}|||||d|||} ||| z}||}| |S)a?Perform the forward pass through the entire decoder layer. Args: embed (torch.Tensor): Input embeddings. refer_bbox (torch.Tensor): Reference bounding boxes. feats (torch.Tensor): Feature maps. shapes (list): Feature shapes. padding_mask (torch.Tensor, optional): Padding mask. attn_mask (torch.Tensor, optional): Attention mask. query_pos (torch.Tensor, optional): Query position embeddings. Returns: (torch.Tensor): Output tensor after decoder layer. rr)rLrf) rDr transposer6r3r unsqueezer7r4r) r8rrrrrrLrrNrOrs r:rUz)DeformableTransformerDecoderLayer.forwards$2##E9555AnnQ[[A.. Aq0A0A5??STVWCXCXdmnnn  )Aq//  c*** 5!!oo   y 1 1:3G3G3J3JESY[g   c*** 5!!&&&r;)rrrrrrrrr r!rrrrrX)rr=r@r=rY)rr=rr=rr=rrrr?rLr?rr?r@r=) rZr[r\r]r,rr)r_rDrrUr`ras@r:rrEs4 '+'+'+'+'+'+'+R777\7    &-1)-)-('('('('('('('('('r;rc4eZdZdZddfd Z dddZxZS)raqDeformable Transformer Decoder based on PaddleDetection implementation. This class implements a complete deformable transformer decoder with multiple decoder layers and prediction heads for bounding box regression and classification. Attributes: layers (nn.ModuleList): List of decoder layers. num_layers (int): Number of decoder layers. hidden_dim (int): Hidden dimension. eval_idx (int): Index of the layer to use during evaluation. References: https://github.com/PaddlePaddle/PaddleDetection/blob/develop/ppdet/modeling/transformers/deformable_transformer.py rirr decoder_layerr!reval_idxctt|||_||_||_|dkr|n||z|_dS)aLInitialize the DeformableTransformerDecoder with the given parameters. Args: hidden_dim (int): Hidden dimension. decoder_layer (nn.Module): Decoder layer module. num_layers (int): Number of decoder layers. eval_idx (int): Index of the layer to use during evaluation. rN)r(r)r rrrr)r8rrrrr9s r:r)z%DeformableTransformerDecoder.__init__sV !-<< $$$,MMzH7L r;Nrr=rrrr bbox_head score_headpos_mlprLr?rc (|} g} g} d} |}t|jD]9\}}|| |||| |||} ||| }tj|t |z}|jrx| ||| |dkr| |n~| tj|t | znF||jkr;| ||| | |n!|} |jr|n|};tj | tj | fS)aPerform the forward pass through the entire decoder. Args: embed (torch.Tensor): Decoder embeddings. refer_bbox (torch.Tensor): Reference bounding boxes. feats (torch.Tensor): Image features. shapes (list): Feature shapes. bbox_head (nn.Module): Bounding box prediction head. score_head (nn.Module): Score prediction head. pos_mlp (nn.Module): Position MLP. attn_mask (torch.Tensor, optional): Attention mask. padding_mask (torch.Tensor, optional): Padding mask. Returns: dec_bboxes (torch.Tensor): Decoded bounding boxes. dec_cls (torch.Tensor): Decoded classification scores. Nr) rrrrr trainingappendrdetachr)r8rrrrrr r!rLrr dec_bboxesdec_clslast_refined_bboxrrbbox refined_bboxs r:rUz$DeformableTransformerDecoder.forwards:  '')) !$+.. R RHAuU6:uflIW^W^_iWjWjkkF9Q<''D = 0K0K)KLLL} }z!}V4455566%%l3333%%emD?K\;];]4]&^&^____dm##}z!}V44555!!,/// , 26-Q,,...\JJ{:&& G(<(<<r3s"""""" 44444444444444TTTTTTTTTT HKHKHKHKHKbiHKHKHKVElElElElEl "ElElElP)))))ry)))>+V+V+V+V+Vry+V+V+V\11111ry1118@E@E@E@E@E")@E@E@EF(I(I(I(I(I")(I(I(IVx(x(x(x(x(29x(x(x(v|'|'|'|'|' |'|'|'~U=U=U=U=U=29U=U=U=U=U=r;