/j} ddlZddlZddlZddlZddlmZddlmcmZ ddl m Z dZ dZ ddZdZdd Zd ejd ejd ejd ejdejf dZdS)N)uniform_)inverse_sigmoid#multi_scale_deformable_attn_pytorchc^tjfdt|DS)aCreate a list of cloned modules from the given module. Args: module (nn.Module): The module to be cloned. n (int): Number of clones to create. Returns: (nn.ModuleList): A ModuleList containing n clones of the input module. Examples: >>> import torch.nn as nn >>> layer = nn.Linear(10, 10) >>> clones = _get_clones(layer, 3) >>> len(clones) 3 c8g|]}tjS)copydeepcopy).0_modules a/home/longshao/multi-rider-rag/.venv/lib/python3.11/site-packages/ultralytics/nn/modules/utils.py z_get_clones.. s#BBBA$-//BBB)nn ModuleListrange)r ns` r _get_clonesrs0" =BBBBqBBB C CCr{Gz?cRttjd|z |z  S)aInitialize conv/fc bias value according to a given probability value. This function calculates the bias initialization value based on a prior probability using the inverse sigmoid (logit) function. It's commonly used in object detection models to initialize classification layers with a specific positive prediction probability. Args: prior_prob (float, optional): Prior probability for bias initialization. Returns: (float): Bias initialization value calculated from the prior probability. Examples: >>> bias = bias_init_with_prob(0.01) >>> print(f"Bias initialization value: {bias:.4f}") Bias initialization value: -4.5951 )floatnplog) prior_probs rbias_init_with_probr#s(& "&!j.J6777 8 88rcdtj|jjdz }t |j| |t |dr |jt |j| |dSdSdS)aInitialize the weights and biases of a linear module. This function initializes the weights of a linear module using a uniform distribution within bounds calculated from the output dimension. If the module has a bias, it is also initialized. Args: module (nn.Module): Linear module to initialize. Examples: >>> import torch.nn as nn >>> linear = nn.Linear(10, 5) >>> linear_init(linear) rrbiasN)mathsqrtweightshaperhasattrr)r bounds r linear_initr&9s}  &--a011 1E V]UFE***vv-6;#:ufe,,,,,--#:#:rh㈵>c|dd}||}d|z |}tj||z S)aCalculate the inverse sigmoid function for a tensor. This function applies the inverse of the sigmoid function to a tensor, which is useful in various neural network operations, particularly in attention mechanisms and coordinate transformations. Args: x (torch.Tensor): Input tensor with values in range [0, 1]. eps (float, optional): Small epsilon value to prevent numerical instability. Returns: (torch.Tensor): Tensor after applying the inverse sigmoid function. Examples: >>> x = torch.tensor([0.2, 0.5, 0.8]) >>> inverse_sigmoid(x) tensor([-1.3863, 0.0000, 1.3863]) rr)minmax)r))clamptorchr)xepsx1x2s rrrMsW$ A1A S  B a%3  B 9R"W  rvaluevalue_spatial_shapessampling_locationsattention_weightsreturnc~|j\}}}}|j\}}}} } }|d|Dd} d|zdz } g} t|D]\}\}}| |ddd||z|||}| dddddd|fdddd}t j||ddd }| ||dd||zd|| | z}tj | d d |z d  |||z|}|dd S) aImplement multi-scale deformable attention in PyTorch. This function performs deformable attention across multiple feature map scales, allowing the model to attend to different spatial locations with learned offsets. Args: value (torch.Tensor): The value tensor with shape (bs, num_keys, num_heads, embed_dims). value_spatial_shapes (torch.Tensor): Spatial shapes of the value tensor with shape (num_levels, 2). sampling_locations (torch.Tensor): The sampling locations with shape (bs, num_queries, num_heads, num_levels, num_points, 2). attention_weights (torch.Tensor): The attention weights with shape (bs, num_queries, num_heads, num_levels, num_points). Returns: (torch.Tensor): The output tensor with shape (bs, num_queries, num_heads * embed_dims). References: https://github.com/IDEA-Research/detrex/blob/main/detrex/layers/multi_scale_deform_attn.py cg|] \}}||z Srr)r H_W_s rrz7multi_scale_deformable_attn_pytorch..s III&"bb2gIIIrr)dimNrbilinearzerosF)mode padding_mode align_corners)r#split enumerateflatten transposereshapeF grid_sampleappendr,stacksumview contiguous)r1r2r3r4bsr num_heads embed_dims num_queries num_levels num_points value_listsampling_gridssampling_value_listlevelr8r9value_l_sampling_grid_l_sampling_value_l_outputs rrres2$); B9j;M;S8A{Iz:qII4HIIIqQQJ++a/N$%9::66xB e$,,Q//99!Q??GGYXbdfhjkk*!!!QQQ5.9CCAqIIQQRSUVWWM &Zg]b    ""#45555*33Aq99AA Y; Z(? (b 1 1 1 9 9" = =@Q Q R b)j(+ 6 6   Aq ! ! , , . ..r)r)r')r r numpyrr,torch.nnrtorch.nn.functional functionalrH torch.nn.initr__all__rrr&rTensorrrrrrds  """""" BDDD(9999,---(08/ <8/,8/ 8/| 8/  \ 8/8/8/8/8/8/r