jj d dl mZ d dlZd dlmZ d dlmZ d dlmZmZ d dl m
Z
m Z d dl m
Z
d dlmZmZ d d lmZ d d
lmZmZ erd d l mZ d d lmZmZ d d
lmZm Z ddd/dZ d0d1d Z d2d"Z d3d%Z d4d.Z!dS )5 ) annotationsN)datetime)Path)
TYPE_CHECKINGAny)_DELTALAKE_AVAILABLE deltalake)eprint)NullTime)
unpack_dtypes)POLARS_STORAGE_CONFIG_KEYS_get_path_scheme)
DeltaTable) DataFrameDataType)
SchemaDictStorageOptionsDictT)strict table_uri
str | Pathr boolreturnstrc t t | 4t | | n| }|S )N)r r r
expanduserresolve)r r resolved_uris [/home/longshao/multi-rider-rag/.venv/lib/python3.11/site-packages/polars/io/delta/_utils.py_resolve_delta_lake_urir sR I & & .
Y""$$,,V444
L
table_pathstr | Path | DeltaTableversionint | str | datetime | Nonestorage_optionsStorageOptionsDict | Nonedelta_table_optionsdict[str, Any] | Nonedeltalake.DeltaTablec t | d | D }t | t j r6t |du|du|dug r t
j dt d | S |i }t | }t |t t f st j |f||d|}n)t j | fd|i|}| | |S )z
Initialize a Delta lake table for use in read and scan operations.
Notes
-----
Make sure to install deltalake>=0.8.0. Read the documentation
`here `_.
Nc , i | ]\ }}|t v||S )r ).0kvs r
z)_get_delta_lake_table..1 s4
1222
q222r! zWhen supplying a DeltaTable directly, `version`, `storage_options`, and `delta_table_options` are ignored.
To silence this warning, don't supply those parameters. )
stacklevel)r$ r& r& )
_check_if_delta_availableitems
isinstancer r anywarningswarnRuntimeWarningr r r load_as_version)r" r$ r& r( r dl_tbls r _get_delta_lake_tabler= s^ "
'--//
*i 233 t# t+#4/
MK
" *:66L gX / /
(%
+
"
%
+
"
w''' Mr! Nonec 6 t sd} t | d S )Nz=deltalake is not installed
Please run: pip install deltalake)r ModuleNotFoundError)msgs r r4 r4 [ s' 'O!#&&&' 'r! dtypeslist[DataType]c j t | }t t h}||z x}rd|}t | d S )Nz+dataframe contains unsupported data types: )r
r r TypeError)rB
schema_dtypesunsupported_typesoverlaprA s r _check_for_unsupported_typesrI a sN !6*M t
"333w GGGGnn r! add_actions_dfr filter_columns list[str]schemar verboseDataFrame | Nonec dd l }d| vr|rt d d S d| d i}d| v r3| d j d ni }d| v r3| d j d ni }d | v r3| d j d ni }|D ]} | | x}
C| d g|
d| j }
| | x} 7| d g||
d| j } | | x} 7| d g||
d| j } |
|| d < | || d < | || d
< |
|| j S )Nr num_recordszDscan_delta: statistics load failed: 'num_records' column not presentlen
null_countT) as_seriesminmax)dtype_nc_min_max)height) polarsr
structunnestto_dictgetSeriesget_index_typenew_from_indexr[ r )
rJ rK rM rN ploutnull_count_colsmin_colsmax_colscol_namecol_nccol_mincol_maxs
r 0_extract_table_statistics_from_delta_add_actionsrm m sj N** V
t!&}(E FC > ) ) |$+2244<->-@-@YAAPP>( F ||H-- -G 6iifX.>i??NN>( G ||H-- -G 6iifX.>i??NN>( G !'x !(x !(x
<<N$9< : ::r! )r r r r r r )NNN)
r" r# r$ r% r& r' r( r) r r* )r r> )rB rC r r> )
rJ r rK rL rM r rN r r rO )"
__future__r r8 r pathlibr typingr r polars._dependenciesr r polars._utils.loggingr
polars.datatypesr r polars.datatypes.convertr
polars.io.cloud._utilsr r r r\ r r polars._typingr r r r= r4 rI rm r- r! r rw s " " " " " " % % % % % % % % @ @ @ @ @ @ @ @ ( ( ( ( ( ( ' ' ' ' ' ' ' ' 2 2 2 2 2 2 O O O O O O O O >$$$$$$********======== FJ ,01515 9 9 9 9 9x' ' ' ' 6; 6; 6; 6; 6; 6;r!