o -ѹg@sXddlmZddlZddlZddlZddlZddlm Z m Z m Z m Z m Z ddlmZmZddlZddlmZeejddZeejjdZeejjdZed gd ZejdddZej dddddfZ!ej"dddZ#ej Z$e%ed ed fZ&ejdddZ'ej(d d e)dDejdZ*ej+ej,-ejdddejdddee$fejdddddfejdddej.dddej/ej/ejdddejejej/ej/ej/ej/d ddddddZ0ej+ej,-ejdddejddde#e$fej"dddddfejdddej.dddej/ej/ej"dddejejej/ej/ej/ej/d ddddddZ1ej+ej,-ejdddejdddee$fejdddddfejdddej.dddej/ej/ejdddejejej/ej/ej/ej/d ddddddZ2ej+dddej,jdddej,jdddej,jej,j/ddddZ3ej+ddddddZ4ej+dej,jej,jd d! " #dmd$d%Z5ej+dej,6e&ej,jej,jd&d! " #dmd'd(Z7ej+dej,6e&ej,jej,jd&d! " #dmd)d*Z8ej+dej,jej,jd d! " #dmd+d,Z9ej+dej,jej,jd d! " #dmd-d.Z:ej+dd/dnd1d2Z;ej+dd/ " 0 #dnd3d4Zej,j?ej,jdddd8ej,jej,j?ej,jdddd8ej,j?ej,j.ddd0d8gdej,jej,j@ej,jAd9dd:d;d<ZBej+d=ej,>ej,j?ej,j"dddd8ej,jej,j?ej,j"dddd8ej,j?ej,j.ddd0d8gdej,jej,j@ej,jAd9dd:d>d?ZCej+d@ej,j?ej,jdddd8ej,j?ej,jdddd8ej,j?ej,jdAddd8ej,j?ej,jdddd8ej,j?ej,jdAddd8ej,j?ej,jdddd8ej,j?ej,j.ddd0d8gej,j/ej,j>dBddCdDdEZDej+dFej,j?ej,jdddd8ej,j?ej,j"dddd8ej,j?ej,j"dAddd8ej,j?ej,jdddd8ej,j?ej,jdAddd8ej,j?ej,jdddd8ej,j?ej,j.ddd0d8gej,j/ej,j>dBddCdGdHZEej+dddIdJdKZFej+dLej,j/iddCdMdNZG  0 0 #dodOdPZHej+dd/dQdRZIdSdTZJdUdVZKdWdXZLe+dYdZZMej+dd[d\d]ZNd^d_ZOdZPdZQdAZRd`ZSdaZTdbdcZUdddeZVej+dej.dddejej/dfd0dgdhdiZWej+ddLejid0djdkdlZXdS)p)warnN) sparse_mul sparse_diff sparse_sum arr_intersectsparse_dot_product) tau_rand_intnorm) namedtupleCg:0yE>FlatTree hyperplanesoffsetschildrenindices leaf_sizecCsg|] }t|dqS)1)bincount.0irX/Users/admin/.pyenv/versions/3.10.0/lib/python3.10/site-packages/pynndescent/rp_trees.py *srdtype) n_leftn_righthyperplane_vectorhyperplane_offsetmargindr left_index right_indexT)localsfastmathnogilcachecCs|jd}t||jd}t||jd}|||k7}||jd}||}||}t||}t||} t|tkr@d}t| tkrHd} tj|tjd} t|D]} ||| f|||| f| | | <qTt| } t| tkrud} t|D] } | | | | | <qyd} d}t|jdtj }t|jdD]L}d}t|D]} || | |||| f7}qt|tkrt|d||<||dkr| d7} q|d7}q|dkrd||<| d7} qd||<|d7}q| dks|dkrd} d}t|jdD]}t|d||<||dkr| d7} q|d7}qtj| tj d}tj|tj d}d} d}t|jdD] }||dkrE|||| <| d7} q0||||<|d7}q0||| dfS)MGiven a set of ``graph_indices`` for graph_data points from ``graph_data``, create a random hyperplane to split the graph_data, returning two arrays graph_indices that fall on either side of the hyperplane. This is the basis for a random projection tree, which simply uses this splitting recursively. This particular split uses cosine distance to determine the hyperplane and which side each graph_data sample falls on. Parameters ---------- data: array of shape (n_samples, n_features) The original graph_data to be split indices: array of shape (tree_node_size,) The graph_indices of the elements in the ``graph_data`` array that are to be split in the current operation. rng_state: array of int64, shape (3,) The internal state of the rng Returns ------- indices_left: array The elements of ``graph_indices`` that fall on the "left" side of the random hyperplane. indices_right: array The elements of ``graph_indices`` that fall on the "left" side of the random hyperplane. r r?rN) shaperr absEPSnpemptyfloat32rangeint8int32)datar rng_statedimr'r(leftright left_norm right_normr#r&hyperplane_normr!r"siderr% indices_left indices_rightrrrangular_random_projection_split.sv ,                       rEcCs|jd}t||jd}t||jd}|||k7}||jd}||}||}d}d} tj|dtjd} | d|} | |d} t|D]"} ||| f||| fA}|||| f@| | <|||| f@| | <qJd}t|D]} |t| | 7}|t||| f7}| t||| f7} qsd}d}t|jdtj}t|jdD]^}d}t|D]"} |t| | |||| f@7}|t| | |||| f@8}qt|t krt|d||<||dkr|d7}q|d7}q|dkrd||<|d7}qd||<|d7}q|dks|dkr8d}d}t|jdD]}t|d||<||dkr2|d7}q|d7}qtj|tj d}tj|tj d}d}d}t|jdD] }||dkrh||||<|d7}qS||||<|d7}qS||| dfS)r-r rr/r0rN) r1rr4r5uint8r7popcntr8r2r3r9)r:rr;r<r'r(r=r>r?r@r#Zpositive_hyperplane_componentZnegative_hyperplane_componentr&Z xor_vectorrAr!r"rBrr%rCrDrrr)angular_bitpacked_random_projection_splitst ,       "             rHcCsd|jd}t||jd}t||jd}|||k7}||jd}||}||}d}tj|tjd} t|D]$} ||| f||| f| | <|| | ||| f||| fd8}q:d} d} t|jdtj} t|jdD]N}|}t|D]} || | |||| f7}q|t|tkrtt|d| |<| |dkr| d7} qt| d7} qt|dkrd| |<| d7} qtd| |<| d7} qt| dks| dkrd} d} t|jdD]}t|d| |<| |dkr| d7} q| d7} qtj| tj d}tj| tj d}d} d} t| jdD] }| |dkr |||| <| d7} q |||| <| d7} q ||| |fS)aPGiven a set of ``graph_indices`` for graph_data points from ``graph_data``, create a random hyperplane to split the graph_data, returning two arrays graph_indices that fall on either side of the hyperplane. This is the basis for a random projection tree, which simply uses this splitting recursively. This particular split uses euclidean distance to determine the hyperplane and which side each graph_data sample falls on. Parameters ---------- data: array of shape (n_samples, n_features) The original graph_data to be split indices: array of shape (tree_node_size,) The graph_indices of the elements in the ``graph_data`` array that are to be split in the current operation. rng_state: array of int64, shape (3,) The internal state of the rng Returns ------- indices_left: array The elements of ``graph_indices`` that fall on the "left" side of the random hyperplane. indices_right: array The elements of ``graph_indices`` that fall on the "left" side of the random hyperplane. r rr/r@r0N) r1rr4r5r6r7r8r2r3r9)r:rr;r<r'r(r=r>r$r#r&r!r"rBrr%rCrDrrr!euclidean_random_projection_split5sd ,  "               rJ)normalized_left_datanormalized_right_datarAr)r*r+r,r)c"CsFt||jd}t||jd}|||k7}||jd}||}||}|||||d} |||||d} |||||d} |||||d} t| } t| }t| tkrgd} t|tkrod}| | tj}| |tj}t| || |\}}t|}t|tkrd}t |jdD] }|||||<qd}d}t |jdtj }t |jdD]l}d}|||||||d}|||||||d}t ||||\}}|D]}||7}qt|tkrt|d||<||dkr |d7}q|d7}q|dkrd||<|d7}qd||<|d7}q|dks2|dkrZd}d}t |jdD]}t|d||<||dkrT|d7}q=|d7}q=tj |tj d}tj |tj d} d}d}t |jdD] }||dkr||||<|d7}qu||| |<|d7}qut||f}!|| |!dfS)Given a set of ``graph_indices`` for graph_data points from a sparse graph_data set presented in csr sparse format as inds, graph_indptr and graph_data, create a random hyperplane to split the graph_data, returning two arrays graph_indices that fall on either side of the hyperplane. This is the basis for a random projection tree, which simply uses this splitting recursively. This particular split uses cosine distance to determine the hyperplane and which side each graph_data sample falls on. Parameters ---------- inds: array CSR format index array of the matrix indptr: array CSR format index pointer array of the matrix data: array CSR format graph_data array of the matrix indices: array of shape (tree_node_size,) The graph_indices of the elements in the ``graph_data`` array that are to be split in the current operation. rng_state: array of int64, shape (3,) The internal state of the rng Returns ------- indices_left: array The elements of ``graph_indices`` that fall on the "left" side of the random hyperplane. indices_right: array The elements of ``graph_indices`` that fall on the "left" side of the random hyperplane. rr r.r/r0rN)rr1r r2r3astyper4r6rr7r5r8rr9vstack)"indsindptrr:rr;r'r(r=r> left_inds left_data right_inds right_datar?r@rKrLhyperplane_indshyperplane_datarAr&r!r"rBrr%i_indsi_data_mul_datavalrCrD hyperplanerrr&sparse_angular_random_projection_splits*                   r^)r*r+r,cCstt||jd}tt||jd}|||k7}||jd}||}||}|||||d} |||||d} |||||d} |||||d} d} t| | | | \}}t| | | | \}}|d}t||||tj\}}|D]}| |8} qd}d}t |jdtj }t |jdD]l}| }|||||||d}|||||||d}t||||\}}|D]}||7}qt|t krtt|d||<||dkr|d7}q|d7}q|dkrd||<|d7}qd||<|d7}q|dks|dkrAd}d}t |jdD]}tt|d||<||dkr;|d7}q"|d7}q"tj |tj d}tj |tj d}d}d}t |jdD] }||dkrq||||<|d7}q\||||<|d7}q\t||f}|||| fS)rMrr r/rIr0rN)r4r2rr1rrrrNr6r5r8r7r3r9rO)rPrQr:rr;r'r(r=r>rRrSrTrUr$rVrWZ offset_inds offset_datar\r!r"rBrr%rXrYrZr[rCrDr]rrr(sparse_euclidean_random_projection_split6sz                    r`) left_node_numright_node_num)r+r)c  C|jd|krb|dkrbt|||\} } } } t|| |||||||d t|d} t|| |||||||d t|d}|| || |t| t|f|tjdgtjddS|tjdgtjd|tj |tdtdf||dSNrr rrg) r1rJmake_euclidean_treelenappendr4r9arrayr6infr:rrrr point_indicesr;r max_depth left_indices right_indicesr]offsetrarbrrrrgsR     rg)rrarbc  Crerf) r1rEmake_angular_treerhrir4r9rjr6rkrlrrrrrR     rrc  Cre)Nrr rr) r1rH make_bit_treerhrir4r9rjrFrkrlrrrru0rsruc  C$|jd| krh| dkrht|||||\} } } }t|||| |||||| | d t|d}t|||| |||||| | d t|d}|| |||t|t|f|tjdgtjddS|tjdgdggtjd|tj |tdtdf||dSrf) r1r`make_sparse_euclidean_treerhrir4r9rjfloat64rkrPrQr:rrrrrmr;rrnrorpr]rqrarbrrrrwts^      rwc  Crvrf) r1r^make_sparse_angular_treerhrir4r9rjrxrkryrrrrzsZ     rz)r+Fc Cst|jdtj}tjjt }tjjt }tjjt }tjjt } |r8t |||||| |||d n t|||||| |||d |} | D]} t| | krXtt| } qIt|||| | } | S)Nrrn)r4aranger1rNr9numbatypedList empty_listdense_hyperplane_type offset_type children_typepoint_indices_typerrrgrhr r:r;rangularrnrrrrrm max_leaf_sizepointsresultrrrmake_dense_treesF   rc Cst|jddtj}tjjt }tjjt } tjjt } tjjt } |rTs  zmake_forest..c3,|]}tt|dVqdSr)rrrrrrrra c3rr)rrrrrrrrlrzRandom Projection forest initialisation failed due to recursionlimit being reached. Something is a little strange with your graph_data, and this may take longer than normal to compute.)maxr4r9randint INT32_MIN INT32_MAXrNint64scipysparseZisspmatrix_csrrParallelr7 RuntimeErrorRecursionError SystemErrorrtuple) r:Z n_neighborsZn_treesrr;Z random_staterrZbit_treernrrrr make_forest,s>      )rcCsd}tt|jD]}|j|ddkr!|j|ddkr!|d7}q tj||fdtjd}d}tt|jD]+}|j|ddksJ|j|ddkra|j|jd}|j|||d|f<|d7}q6|S)Nrrr r)r7rhrr4fullr9rr1)treern_leavesrrZ leaf_indexrrrrget_leaves_from_trees$$rcs8tdd|Dtjdddfdd|D}|S)NcSsg|]}|jqSr)rrZrp_treerrrrsz.rptree_leaf_array_parallel..rrrc3s |] }tt|VqdSN)rrrrrrrrs z-rptree_leaf_array_parallel..)r4rrr) rp_forestrrrrrptree_leaf_array_parallels rcCs(t|dkr tt|StdggS)Nrr)rhr4rOrrj)rrrrrptree_leaf_arrays rc Cs|j|ddkr-|t|j|}| ||df<| ||df<|j||||<||fS|j|||<|j|||<|d||df<|} t||||||d||j|d\}}|d|| df<t||||||d||j|d\}}||fSr)rrhrrrrecursive_convert rrrrrZnode_numZ leaf_startZ tree_nodeZleaf_endZ old_node_numrrrrs@    rc Cs |j|ddkr-|t|j|}| ||df<| ||df<|j||||<||fS|j|||ddd|j|jdf<|j|||<|d||df<|} t||||||d||j|d\}}|d|| df<t||||||d||j|d\}}||fSr)rrhrrr1rrecursive_convert_sparserrrrrsF    r)r,cCsPd}d}tt|jD]}|j|ddkr|d7}|d7}q |d7}q ||fSr)r7rhr)rn_nodesrrrrrnum_nodes_and_leavess  rc Cs0t|\}}d}|jdjdkr.|jdjtjkr|d}n|}tj||f|jdjd}nd}|}tj|d|ftjd}d|dddddf<tj|tjd}tdtj |dftjd} tdtj |tjd} |rt |||| | ddt |j dnt |||| | ddt |j dt||| | |jS)NFrr r0rTr)rrndimr r4rFzerosr6r9Zonesrrhrrr r) r data_sizeZdata_dimrrZ is_sparseZhyperplane_dimrrrrrrrconvert_tree_formats,  rrcCs|j|j|j|j|jf}|Srrrrrrrdenumbaify_tree&srcCs(t|t|t|t|t|t}|Sr)r FLAT_TREE_HYPERPLANESFLAT_TREE_OFFSETSFLAT_TREE_CHILDRENFLAT_TREE_INDICESFLAT_TREE_LEAF_SIZErrrrrenumbaify_tree2sr) intersectionrr)parallelr)r,cCsrd}t|jdD]$}t|||j|j|j|j|}t|||}|t |jddk7}q |t |jdS)Nr/rr ) r}Zpranger1rrrrrrr6)rneighbor_indicesr:r;rrZ leaf_indicesrrrr score_tree>s r)r+r)r,c Csd}t|j}t|D]G}t|}|j|d}|j|d}|dkrR|dkrRt|j|jdD]}|j||} t|| |j|} |t| jddk7}q2q |t|jdS)Nr/rr r) rhrr7r}r9rr1rr6) rrrrrrZ left_childZ right_childjidxrrrrscore_linked_treeWs   r)rcrd)rcFrd)NFFrd)YwarningsrlocaleZnumpyr4r}Z scipy.sparserZpynndescent.sparserrrrrZpynndescent.utilsrr r collectionsr setlocale LC_NUMERICr3Ziinfor9minrrrr r6rrxrrFrrZtypeofrrrjr7rGZnjittypesTuplerZuint32rErHrJr^r`rgZListTyperrrurwrzrrrbooleanArrayZintpZuint16rrrrrrrrrrrrrrrrrrrrrrrrrrrs  "2  r"2  o"2  d  v  <  ;  < D B  ) 5             T &  (