j>[UddlmZddlZddlmZmZddlmZddlm Z m Z ddl m Z m Z mZddlmZddlmZmZdd lmZdd lmZdd lmZmZmZdd lmZdd lm Z m!Z!ddl"m#Z#ddl$m%Z%e rPddl&Z&ddlm'Z'm(Z(ddl)m*Z*ddl+Z,ddlm-Z-e&j.dkrddl m/Z/nddl0m/Z/ddl1m2Z2ddl3m4Z4ddlm5Z5ddl6m7Z7m8Z8m9Z9hdZ:de;d<GddZ<GddZ=dS) ) annotationsN) CoroutineSequence)suppress) Parameter signature) TYPE_CHECKINGAnyFinal) functions) parse_versionqualified_type_name) from_arrow)N_INFER_DEFAULT)DuplicateErrorModuleUpgradeRequiredErrorUnsuitableSQLError)ARROW_DRIVER_REGISTRY)ODBCCursorProxySurrealDBCursorProxy)dtype_from_cursor_description) _run_async)IterableIterator) TracebackType)ArrowDriverProperties) )Self TextClause) Selectable DataFrame)ConnectionOrCursorCursor SchemaDict> USEDROPALTERGRANTCREATEDELETEINSERTREVOKEUPDATEUPSERTVACUUMANALYZEREPLACEzFinal[set[str]]_INVALID_QUERY_TYPESc"eZdZdZd dZd d Zd S)CloseAfterFrameIterzDAllows cursor close to be deferred until the last batch is returned.framesr cursorr&returnNonec"||_||_dSN) _iter_frames_cursor)selfr8r9s a/home/longshao/multi-rider-rag/.venv/lib/python3.11/site-packages/polars/io/database/_executor.py__init__zCloseAfterFrameIter.__init__?s" Iterator[DataFrame]c#K|jEd{Vt|jdr|jdSdS)Nclose)r>hasattrr?rFr@s rA__iter__zCloseAfterFrameIter.__iter__CsY$$$$$$$$ 4< ) ) ! L    ! !rCN)r8r r9r&r:r;)r:rD)__name__ __module__ __qualname____doc__rBrIrCrAr7r7<sBNN!!!!!!rCr7cneZdZUdZdZded<dLd ZdMd ZdNdZdOdZ e dPdZ dQdZ e dRdZ dSd$Ze dTd)ZdUd*ZdVd.ZdVd/ZdWd2Ze dXd5Ze dXd6Ze dXd7Ze dXd8Ze dYd9ZdZd:Zd[d>Zd\dBZdCdDdEd]dHZddCdCedId^dKZdCS)_ConnectionExecutorzIAbstraction for querying databases with user-supplied connection objects.Fboolcan_close_cursor connectionr%r:r;c6t|trdn?t|jddd|_|jdkrt|}|||_ d|_ dS)Narrow_odbc_proxy.r surrealdb)client) isinstancertyperKsplitlower driver_namer_normalise_cursorr9result)r@rSs rArBzConnectionExecutor.__init__Qs*o66 F  j!!,223::1=CCEE   { * *-Z@@@J,,Z88  rCrc|Sr=rNrHs rA __enter__zConnectionExecutor.__enter__]s rCexc_typetype[BaseException] | Noneexc_valBaseException | Noneexc_tbTracebackType | Nonec,||jr@ddlm}t |j|r#t |dSdS|jr0t|jdr|j dSdSdS)Nr)AsyncConnectionrF) _is_alchemy_asyncr9sqlalchemy.ext.asynciorjrZr_close_async_cursorrRrGrF)r@rcrergrjs rA__exit__zConnectionExecutor.__exit__`s  ! !$+ . . > > > > > >$+77 74335566666 7 7  " wt{G'D'D K           rCstrcBdt|jd|jdS)N)r[rJr^rHs rA__repr__zConnectionExecutor.__repr__ps'E4::&EE0@EEEErCdfr$schema_overridesr'c|jfd|Dx}r||}|S)z&Apply schema overrides to a DataFrame.cg|]<\}}|v ||ktj||=SrN)Fcolcast).0rydtypeexisting_schemas rA z7ConnectionExecutor._apply_overrides..wsU   Uo%%%?33G*G*G E#JJOOE " "*G*G*GrC)schemaitems with_columns)rtru cast_colsr}s @rA_apply_overridesz#ConnectionExecutor._apply_overridesssb)    .4466    9 , ++B rCcK|jrdt|jdrQddlm}t |5|jd{VddddS#1swxYwYdSdSdS)NrFr)AsyncContextNotStarted)rRrGr9sqlalchemy.ext.asyncio.excrrrF)r@rs rArmz&ConnectionExecutor._close_async_cursors   *WT['%B%B * I I I I I I011 * *k''))))))))) * * * * * * * * * * * * * * * * * * * * * *s A!!A%(A% module_nameminimum_versionc Rt|}tt5d}dD]9}tt ||dx}t rt |}n:|r*|t |krd|d|}t|ddddS#1swxYwYdS)z.s---%**---rC)fetchallrZlistrdict)r`rrowss rA_fetchall_rowsz!ConnectionExecutor._fetchall_rowss\   .# .'1$q'D%;N'O'O .DD----- rCc#K ||}|sdS|s(t|dtttfr|Vnd|DVW)zDFetch row data incrementally, yielding over the complete result set.Trc,g|]}t|SrNrrs rAr~z6ConnectionExecutor._fetchmany_rows..s222cuSzz222rCN) fetchmanyrZrrr)r@r`rrrs rA_fetchmany_rowsz"ConnectionExecutor._fetchmany_rowssy 3##J//D 3 3z$q'D%3FGG 3 22T222222 3rCSchemaDict | Noneinfer_schema_length&DataFrame | Iterator[DataFrame] | Nonecddlm jdrdnj}t j|g}t |dD]\}}|dx} rK|dkrdnj} | | n$#t$r|t|krYUwxYw|r$|d r|sd jd } t| fd  ||| D} |r| nt| cSn5#t$r(d} tfd| DsYdndwwxYwdS)z5Return resultset data in Arrow format for frame init.rr#adbc_adbcrW)startradbc_driver_managerrzCannot set `iter_batches` for z- without also setting a non-zero `batch_size`c3K|]=}t|r|pint|V>dS))ruN)rZrr)r{batchr$rur@s rA z1ConnectionExecutor._from_arrow..su  "%33ND))%2B2HbJJJ#E.s-BBqCHH}BBBBBBrCN)polarsr$r^ startswithrget enumeraterrlen ValueErrorrnext Exceptionany)r@rrrurdriverdriver_properties_listirrdriver_to_checkrr8arrow_not_supportedr$rs` ` @@rA _from_arrowzConnectionExecutor._from_arrowsA %$$$$$2 *55g>>TDDT &;%>vr%J%J "(12HPQ(R(R(R# @# @$$+,=>>3 2861A1A--tGW$ 22?CHHHH5s#9::::$H  *%&89*BL*G9IGGG%S//)      "&!2!2)%1#-"3""   ".?vv4<<???G# @H   # BBBB.ABBBBB       ts=A'D4B  D B,(D*B,,A#D ED>>Ec ddlm |r|sd}t|t|jx}t x}rt |j|_ t|jdrQ|jdkx}r{t|jdrd|jj j D} nxt|jdrd |jj j D} nGd |j}t|t|jd rd |jj D} ng} | | pi d| D  fd|r||j||n||j|gD} |r| nt!|  |r|SS |r|dSdS#|r|wwxYw)z.Return resultset data row-wise for frame init.rr#zFCannot set `iter_batches` without also setting a non-zero `batch_size`r sqlalchemyr9c2g|]}|d|ddfSrrWNrNr{ds rAr~z1ConnectionExecutor._from_rows..s5'''./QqT1QRR5M'''rC _metadatacg|]}|dfSr=rN)r{ks rAr~z1ConnectionExecutor._from_rows.. s&U&U&UQ4y&U&U&UrCz0Unable to determine metadata from query result; descriptionc2g|]}|d|ddfSrrNrs rAr~z1ConnectionExecutor._from_rows..&s("R"R"RQAaD!ABB%="R"R"RrC)rrucg|]\}}|SrNrN)r{nm_s rAr~z1ConnectionExecutor._from_rows...s!>!>!>Q"!>!>!>rCc3<K|]}|pddVdS)Nr)datarrurorientrN)r{rr$rresult_columnsrus rArz0ConnectionExecutor._from_rows../s^I!-5)9,?$ rC)rr)rN)rr$rrZr`rrrGr^r9rrkeys_inject_type_overridesrrrrF) r@rrrurroriginal_resultis_asyncr cursor_descr8r$rs `` @@rA _from_rowszConnectionExecutor._from_rowss %$$$$$  "  "X S// !!T["@/)LL L8 2$T[11DK- (t{J//( @"&"2l"BC:%t{H55.''37;3E3Q''' !k::.&U&U$+:O:T&U&U&U `QUQ\``(oo-T[-88%"R"R$+:Q"R"R"RKK"$K#'#>#> +&6&<"$?$$ "?!>+!>!>!>( W,, K'1'1- #11$+*1UUV$".?vv4<<? (%%'''' ( (%%''''' ( (x (%%'''' (s EGG3rlist[tuple[str, Any]]ct}|D]O\}}||vrd|d}t||||vrt|}||||<||P|S)a Attempt basic dtype inference from a cursor description. Notes ----- This is limited; the `type_code` description attr may contain almost anything, from strings or python types to driver-specific codes, classes, enums, etc. We currently only do the additional inference from string/python type values. (Further refinement will require per-driver module knowledge and lookups). zcolumn z2 appears more than once in the query/result cursor)setrradd)r@rru dupe_checkrdescrr|s rArz)ConnectionExecutor._inject_type_overridesGsUU #  HBZXXXX$S)))!b0@&@&@5d;;$+0$R( NN2    rCconnr c` ddlm}m}m}t ||||fS#t $rYdSwxYw)z2Check if the given connection is SQLALchemy async.r)rj AsyncSessionasync_sessionmakerF)rlrjrrrZ ImportError)rrjrrs rArkz$ConnectionExecutor._is_alchemy_asynccsu            d_lDV$WXX X   55 s  --cddlm}t||rdS ddlm}t||S#t $rYdSwxYw)z5Check if the given connection is a SQLAlchemy Engine.r)EngineT) AsyncEngineF)sqlalchemy.enginerrZrlrr)rrrs rA_is_alchemy_enginez%ConnectionExecutor._is_alchemy_engineqsv -,,,,, dF # # 4  : : : : : :dK00 0   55 s 0 >>cft|jddddkS)zCCheck if the given connection is a SQLAlchemy object (of any kind).rVrWrr)r[rKr\)rs rA_is_alchemy_objectz%ConnectionExecutor._is_alchemy_objects-Dzz$**32215EErCcddlm}ddlm}m}t ||||frdS ddlm}t ||S#t$rYdSwxYw)z=Check if the given connection is a SQLAlchemy Session object.r)r)Session sessionmakerT)rF)rlrsqlalchemy.ormrrrZrr)rrrrrs rA_is_alchemy_sessionz&ConnectionExecutor._is_alchemy_sessions 87777788888888 d\7LA B B 4  A A A A A Ad$677 7   55 s; A A c ddlm}t||rdSddlm}t||S#t $rYdSwxYw)z8Check if the given result is a SQLAlchemy Result object.r) CursorResultT) AsyncResultF)rrrZrlrr)r`rrs rA_is_alchemy_resultz%ConnectionExecutor._is_alchemy_resultst  6 6 6 6 6 6&,// t : : : : : :fk22 2   55 s00 >>c>|jdkr||r|S|jjdkr2d|_|jS|jjdkr d|_|S||rd|_|S|St|dr+t|jx}r |n|}d|_|St|dr|Sd t|d }t|) zCNormalise a connection object such that we have the query executor.rzdatabricks-sql-python databricks duckdb_engineduckdbTr9executezUnrecognised connection type z!; no 'execute' or 'cursor' method) r^renginerraw_connectionr9rrRconnectrGcallabler TypeError)r@rr9rs rAr_z$ConnectionExecutor._normalise_cursors?  | + +''--  ;%)@@@'3D$;5577>>@@@['?::'/D$K,,T22 ,0D)<<>>)K T8 $ $ !)DK*?&!@!@LVVXXXfF$(D !M T9 % % K +,?,E,E + + + nnrCqueryr!optionscK||j}|r|jn|j}t|dd|j|fi|d{VS|4d{V}|rt |ds|j}|j|fi|d{V}|cdddd{VS#1d{VswxYwYdS)z5Execute a query using an async SQLAlchemy connection.sync_connectionNr )rr9beginrr rGsession)r@rr is_sessionr9rr`s rA_sqlalchemy_async_executez,ConnectionExecutor._sqlalchemy_async_executes--dk:: (2C""$$$  6,d 3 3 ?'9999999999 9       T $'$ ":": $|'4<9999999999F                               s,/B.. B8;B8str | TextClause | Selectabledict[str, Any]9tuple[Any, dict[str, Any], str | TextClause | Selectable]cddlm}ddlm}ddlm}d}d}t |j|r6d|vr2d|vr.|}| d|d<d}| |}| |j} | st |trt|jdrl|jj}t ||rt|}t |t r+t#d |Dst%|||<n t |tr ||}|| r|jn |jj}|||fS) z.sA4412 1tUm,,444444rC)rrsqlalchemy.sqlrsqlalchemy.sql.elementsr!rZr9copypoprrkrrGrrorallrrr ) r@rrrrr! param_keycursor_executerrs rA_sqlalchemy_setupz$ConnectionExecutor._sqlalchemy_setups +*****''''''666666  t{G , , !''''llnnG ' L 9 9GH  IY''))$+66 68,,  %677 "[8N%,, #E &$'' 3446<44411 3&+6]] " s # # DKKE  !2:S.. @S w--rCNT)rselect_queries_onlydict[str, Any] | Noner*c|rtt|tr_tjdtjdd|}|sdn|dx}t vr|d}t|pi||j r| |\}}n |j j } t|j }n#t$ri}YnwxYwr+td|Dr ||fi} nfd|pD} ||g| R} | | d ur|j n| } |jd kr|| r| j } | |_|S) z-Execute a query and reference the result set.z\w{3,}z/\*(.|[\r\n])*?\*/rz( statements are not valid 'read' queriesc3TK|]#}|jtjtjfvV$dSr=)kindr KEYWORD_ONLYPOSITIONAL_OR_KEYWORDr s rArz-ConnectionExecutor.execute..sH   Fy-y/NO O      rCc34K|]}r|v|VdSr=rN)r{ors rArz-ConnectionExecutor.execute..$s="" "KLPW<< <<<<""rCNTr)rZroresearchsubgroupr5rrr9r)r rrrrvaluesr^rr`) r@rrr*q query_typerr(rr`positional_optionss ` rAr zConnectionExecutor.executes  .:eS#9#9 . )RV,A2u%M%MNNA()9bbqwwqzz9 >RRR#MMM(----R  " "4; / / 1-1-C-CE7-S-S *NGUU![0N ~..9FF   FFF  @#  ]]__     @$^E55W55FF""""%+%6w""" $^E?,>???F"(6T>>  x ' 'D,C,CF,K,K ']F  s>C C"!C")rrrurDataFrame | Iterator[DataFrame]c,|jd}t||j}|o|x}rd|_|j|jfD]9}|||||} | &|r dt | |jD} | cS:d|jd|j}t|) z Convert the result set to a DataFrame. Wherever possible we try to return arrow-native data directly; only fall back to initialising with row-level data if no other option. Nz.cannot return a frame before executing a queryF)rrrurc3K|]}|VdSr=rN)r{rts rArz/ConnectionExecutor.to_polars..Ts6rC)r9zCurrently no support for z connection ) r` RuntimeErrorrRrrr7r^r9NotImplementedError) r@rrrurr can_closedefer_cursor_close frame_initframes rA to_polarszConnectionExecutor.to_polars1s  ; BCs## #) "."<9 =  *$)D !   O   JJ%)!1$7 E  %"5!#';###E ! X(8 W W W W "#&&&rC)rSr%r:r;)r:r)rcrdrerfrgrhr:r;)r:ro)rtr$rur'r:r$)r:r;)rrorror:r;)rrrrrrQr:r)r`r&rrQr:r)r`r&rrrrQr:r) rrrrQrurrrr:r)rrrur'r:r')rr r:rQ)r`r r:rQ)rr r:r&)rr!rr r:r )rrrrr:r)rrrr+r*rQr:r) rrQrrrurrrr:r<)rJrKrLrMrR__annotations__rBrbrnrs staticmethodrrmrrrrrrrrkrrrrr_rr)r rrErNrCrArPrPJsSS#""""         FFFF   \ **** 6 6 6\ 6    2   \  3 3 3 3????B@(@(@(@(D    8   \    \ FFF\F   \    \ """"H ).).).).^*.$( ------d#!%.2*9 /'/'/'/'/'/'/'/'rCrP)> __future__rr4collections.abcrr contextlibrinspectrrtypingr r r rr rxpolars._utils.variousr rpolars.convertrpolars.datatypesrpolars.exceptionsrrr"polars.io.database._arrow_registryr"polars.io.database._cursor_proxiesrrpolars.io.database._inferencerpolars.io.database._utilsrsysrrtypesrpyarrowpar version_infortyping_extensionsr#r!sqlalchemy.sql.expressionr"r$polars._typingr%r&r'r5rFr7rPrNrCrAr]s""""""" ////////((((((((,,,,,,,,,,!!!!!!DDDDDDDD%%%%%%,,,,,, EDDDDDTTTTTTTTGGGGGG000000FJJJ22222222######HHHHHH 7""******222222444444      EEEEEEEEEE)))" ! ! ! ! ! ! ! !V'V'V'V'V'V'V'V'V'V'rC