a g gb@sdZddlZddlZddlmZmZddlmZddlm Z ddl m Z m Z m Z mZmZmZmZmZmZmZmZmZmZmZejdkrddlmZmZndd l mZmZdd lmZdd lm Z dd l!m"Z"m#Z#dd l$m%Z%m&Z&ddl'm(Z(m)Z)m*Z*m+Z+m,Z,m-Z-m.Z.m/Z/m0Z0m1Z1ddl2m3Z3m4Z4m5Z5ddl6m7Z7m8Z8m9Z9m:Z:m;Z;mZ>ddl?m@Z@mAZAGdddeBZCedZDee@eAfZEe e"ee%e&gee"fZFeGZHeGZIeGZJeGZKe5eDeCfZLeLeeHZMeNgdZOePe3eCdddZQe"ee%e&ee"dddZRGdddeZSe Gdd d ZTeGd!d"d"ZUGd#d$d$eSeUZVGd%d&d&eSZWGd'd(d(eSZXePeeeGeGfd)d*d+ZYePeZd)d,d-Z[ePePePd.d/d0Z\Gd1d2d2eXeUZ]Gd3d4d4eXeUZ^Gd5d6d6Z_e@e eEgdfd7d8d9Z`ee e eGgeZfd:d;d<ZadS)=z7 String transformers that can split and merge strings. N)ABCabstractmethod) defaultdict) dataclass)AnyCallableClassVar CollectionDictIterableIteratorListOptionalSequenceSetTupleTypeVarUnion))FinalLiteral)rr)trait)contains_pragma_comment)Line append_leaves)FeatureMode) CLOSING_BRACKETSOPENING_BRACKETSSTANDALONE_COMMENT is_empty_lpar is_empty_par is_empty_rparis_part_of_annotation parent_type replace_childsyms)ErrOkResult)assert_is_leaf_stringcount_chars_in_widthget_string_prefixhas_triple_quotesnormalize_string_quotes str_width)token)LeafNodec@seZdZdZdS)CannotTransformz-Base class for errors raised by Transformers.N)__name__ __module__ __qualname____doc__r9r9N/var/www/html/llm_bihealth/app/venv/lib/python3.9/site-packages/black/trans.pyr4=sr4T)u、u。u，)err_msgreturncCst|}t|S)zW(T)ransform Err Convenience function used when working with the TResult type. )r4r()r<cant_transformr9r9r:TErrOsr?)linefeaturesmoder=c #s4jD]}|jtjkrq$qtdttdtdfdd ttdtdfdd }}d }t jD]\}}|}|rd |_ d }d |kot jd kno|jtjko||d ddoj|d j dko||d dd}|rd |_ |j |dd|D]} |j | ddqqp|VdS)z>A transformer which normalizes spacing around power operators.z*No doublestar token was found in the line.)indexstepr=cs|dkrtjtjh}n tjtjh}d|kr.is_simple_lookup)baseexponent)rFkindr=cszj|}|jtjtjhvr4||dkr,dnddS|jtjtjtjhvrvj|djtjtjhvrv|dddSdS)NrVrDrE)rGF)rNrOr1rPNUMBERPLUSMINUSTILDE)rFrWstartrTr@r9r:is_simple_operand|s z'hug_power_op..is_simple_operandFrrDrU)rWlambdarVTZ preformattedN)rNrOr1 DOUBLESTARr4intrboolclone enumerateprefixrMrRappendcomments_after) r@rArBleafr^new_lineZ should_hugidxZnew_leaf comment_leafr9r]r: hug_power_opXs8  " rnc@seZdZUdZdZeed<eeddddZe e e ddd Z e e e eeee d d d Ze eeeee d ddZdS)StringTransformera An implementation of the Transformer protocol that relies on its subclasses overriding the template methods `do_match(...)` and `do_transform(...)`. This Transformer works exclusively on strings (for example, by merging or splitting them). The following sections can be found among the docstrings of each concrete StringTransformer subclass. Requirements: Which requirements must be met of the given Line for this StringTransformer to be applied? Transformations: If the given Line meets all of the above requirements, which string transformations can you expect to be applied to it by this StringTransformer? Collaborations: What contractual agreements does this StringTransformer have with other StringTransfomers? Such collaborations should be eliminated/minimized as much as possible. r5N) line_lengthnormalize_stringsr=cCs||_||_dSN)rprq)selfrprqr9r9r:__init__szStringTransformer.__init__r@r=cCsdS)a Returns: * Ok(string_indices) such that for each index, `line.leaves[index]` is our target string if a match was able to be made. For transformers that don't result in more lines (e.g. StringMerger, StringParenStripper), multiple matches and transforms are done at once to reduce the complexity. OR * Err(CannotTransform), if no match could be made. Nr9rsr@r9r9r:do_matchszStringTransformer.do_matchr@string_indicesr=cCsdS)a& Yields: * Ok(new_line) where new_line is the new transformed line. OR * Err(CannotTransform) if the transformation failed for some reason. The `do_match(...)` template method should usually be used to reject the form of the given Line, but in some cases it is difficult to know whether or not a Line meets the StringTransformer's requirements until the transformation is already midway. Side Effects: This method should NOT mutate @line directly, but it MAY mutate the Line's underlying Node structure. (WARNING: If the underlying Node structure IS altered, then this method should NOT be allowed to yield an CannotTransform after that point.) Nr9)rsr@ryr9r9r: do_transformszStringTransformer.do_transform)r@ _features_moder=ccstdd|jDstd||}t|trN|}td|jjd|| }| ||D].}t|tr|}td|| }|VqbdS)z StringTransformer instances have a call signature that mirrors that of the Transformer type. Raises: CannotTransform(...) if the concrete StringTransformer class is unable to transform @line. css|]}|jtjkVqdSrr)rOr1STRING.0rjr9r9r: z-StringTransformer.__call__..z"There are no strings in this line.zThe string transformer z; does not recognize this line as one that it can transform.z>StringTransformer failed while attempting to transform string.N) anyrNr4rw isinstancer(err __class__r5okrz)rsr@r{r| match_resultr>ryZ line_resultr9r9r:__call__s*    zStringTransformer.__call__)r5r6r7r8r__annotations__rcrdrtrr TMatchResultrwr r TResultrzr rrrr9r9r9r:ros     roc@s"eZdZUdZeed<eed<dS) CustomSplitaA custom (i.e. manual) string split. A single CustomSplit instance represents a single substring. Examples: Consider the following string: ``` "Hi there friend." " This is a custom" f" string {split}." ``` This string will correspond to the following three CustomSplit instances: ``` CustomSplit(False, 16) CustomSplit(False, 17) CustomSplit(True, 16) ``` has_prefix break_idxN)r5r6r7r8rdrrcr9r9r9r:rs rc@seZdZUdZeeefZee d<e e Z ee eeedffe d<eeddddZeeed d d d Zeeedd dZeedddZd S)CustomSplitMapMixinz This mixin class is used to map merged strings to a sequence of CustomSplits, which will then be used to re-split the strings iff none of the resultant substrings go over the configured max line length. _Key._CUSTOM_SPLIT_MAPzCustomSplitMapMixin._Keystringr=cCs t||fS)z Returns: A unique identifier that is used internally to map @string to a group of custom splits. )id)rr9r9r:_get_key9szCustomSplitMapMixin._get_keyN)r custom_splitsr=cCs||}t||j|<dS)zCustom Split Map Setter Method Side Effects: Adds a mapping from @string to the custom splits @custom_splits. N)rtupler)rsrrkeyr9r9r:add_custom_splitsBs z%CustomSplitMapMixin.add_custom_splitscCs$||}|j|}|j|=t|S)aaCustom Split Map Getter Method Returns: * A list of the custom splits that are mapped to @string, if any exist. OR * [], otherwise. Side Effects: Deletes the mapping between @string and its associated custom splits (which are returned to the caller). )rrlist)rsrrrr9r9r:pop_custom_splitsMs  z%CustomSplitMapMixin.pop_custom_splitscCs||}||jvS)zb Returns: True iff @string is associated with a set of custom splits. )rr)rsrrr9r9r:has_custom_splitsas z%CustomSplitMapMixin.has_custom_splits)r5r6r7r8rStringIDstrrrrrrrr r staticmethodrr rr rrdrr9r9r9r:r,s    rc@seZdZdZeedddZeeee e edddZ e eeee eddd Z eeee edd d Zeeeeegefeeefd d dZe eee ddddZdS) StringMergeraStringTransformer that merges strings together. Requirements: (A) The line contains adjacent strings such that ALL of the validation checks listed in StringMerger._validate_msg(...)'s docstring pass. OR (B) The line contains a string which uses line continuation backslashes. Transformations: Depending on which of the two requirements above where met, either: (A) The string group associated with the target string is merged. OR (B) All line-continuation backslashes are removed from the target string. Collaborations: StringMerger provides custom split information to StringSplitter. rucCs|j}t|}g}d}||r||}|jtjkr||dr||djtjkrt|sd|||d7}||r||jtjkr|d7}qlq|jtjkrd|jvr|||d7}||r||jtjkr|d7}qq|d7}q|rt|St dSdS)NrrD\ z+This line has no strings that need merging.) rNis_valid_index_factoryrOr1r}r$rhrRr)r?)rsr@LLis_valid_indexryrlrjr9r9r:rw~s4       zStringMerger.do_matchrxc cs|}|||}t|tr"|}|||}t|tr@|}t|trt|tr|}|}td}||_||_t|Vn t|VdS)Nz6StringMerger failed to merge any strings in this line.) )_remove_backslash_line_continuation_charsrr)r_merge_string_groupr(rr4 __cause__) rsr@ryrkZ rblc_resultZ msg_resultZmsg_cant_transformZrblc_cant_transformr>r9r9r:rzs&    zStringMerger.do_transformcCs|j}g}|D]6}||}|jtjkrd|jvrt|js||q|sRtdS|}|j |_ t ||||D]}|j|}|j dd|_qvt |S)a$ Merge strings that were split across multiple lines using line-continuation backslashes. Returns: Ok(new_line), if @line contains backslash line-continuation characters. OR Err(CannotTransform), otherwise. rzKFound no string leaves that contain backslash line continuation characters.r_)rNrOr1r}rRr.rhr?recommentscopyrreplacer))r@ryrZindices_to_transform string_idx string_leafrkZnew_string_leafr9r9r:rs,     z6StringMerger._remove_backslash_line_continuation_charscCs|j}t|}i}|D].}|||}t|tr2q||||||<q|sRtdS|}d} d} t|D]x\} } | |vr| } || \} } | | | | kr| | krnn$| || D]}|j |ddqqjt ||| gqjt |S)am Merges string groups (i.e. set of adjacent strings). Each index from `string_indices` designates one string group's first leaf in `line.leaves`. Returns: Ok(new_line), if ALL of the validation checks found in _validate_msg(...) pass. OR Err(CannotTransform), otherwise. zNo string group is mergedrETra) rNr _validate_msgrr(_merge_one_string_groupr?rerfrhrirr))rsr@ryrrZmerged_string_idx_dictrvresultrkZprevious_merged_string_idxZprevious_merged_num_of_stringsirjrrmr9r9r:rs<      z StringMerger._merge_string_group)rrrr=csd||j}d}||jdtttdfdd }g}g}|} d} | sz|| rz|| jtjkrzt|| j} | d7} qBd} d} d} |} || r*|| jtjkr*| d7} || j}t|}d | vrd |vrt d d |}|||}t |}| || | ||} || | } | d7} q| }t tj| }|j rNt|j|_|jt| dd}|D]`}||}|dksJd ||t|d }||rt| ndd}| t||qht tj|j|d}|d urN||t|jkrD||}t|d|D]}||q|d urN|||n t||||j|| |fS)aA Merges one string group where the first string in the group is `LL[string_idx]`. Returns: A tuple of `(num_of_strings, leaf)` where `num_of_strings` is the number of strings merged and `leaf` is the newly merged string to be replaced in the new line. z#@@@@@ BLACK BREAKPOINT MARKER @@@@@rE)r string_prefixr=csTt|d|vrt|}d}|t|dd}td|dd|}|S)aStrip @string (i.e. make it a "naked" string) Pre-conditions: * assert_is_leaf_string(@string) Returns: A string that is identical to @string except that @string_prefix has been stripped, the surrounding QUOTE characters have been removed, and any remaining QUOTE characters have been escaped. fz(?:(?.make_nakedr_rDrrz(\{|\})z\1\1z=Logic error while filling the custom string breakpoint cache.N)parentrRrrOr1r}r-lowerrrrdrhr2rqr/rMfindrrchildrenremoverange insert_childr&r)rsrrrZ atom_nodeZ BREAK_MARKrrZprefix_trackerZ next_str_idxrgSZNSnum_of_stringsZSSZ next_prefixZNSSrZnon_string_idxZS_leafZ temp_stringZmark_idxZbreakpoint_idxrZfirst_child_idxrlr9rr:r"sr               z$StringMerger._merge_one_string_groupNr@rr=c CsdD]d}|}d}t|j}||r|j|jtjtfvr|j|jtkrNd}n|r^tdS||7}qqd}t}d}|j|dD]} | jtjkr| jtjkrt | |j vr|d7}q>t | j rtdS|d7}t | j } d | vrtd S|| t | |j vr|d7}t|j t | rtd Sq|d krXtd |dS|dkrrtd|dSt|dkr|ddhkrtd|dStdS)aValidate (M)erge (S)tring (G)roup Transform-time string validation logic for _merge_string_group(...). Returns: * Ok(None), if ALL validation checks (listed below) pass. OR * Err(CannotTransform), if any of the following are true: - The target string group does not contain ANY stand-alone comments. - The target string is not in a string group (i.e. it has no adjacent strings). - The string group has more than one inline comment. - The string group has an inline comment that appears to be a pragma. - The set of all string prefixes in the string group is of length greater than one and is not equal to {"", "f"}. - The string group consists of raw strings. - The string group is stringified type annotations. We don't want to process stringified type annotations since pyright doesn't support them spanning multiple string values. (NOTE: mypy, pytype, pyre do support them, so we can change if pyright also gains support in the future. See https://github.com/microsoft/pyright/issues/4359.) rCFTzMStringMerger does NOT merge string groups which contain stand-alone comments.rNrDz.StringMerger does NOT merge multiline strings.rz(StringMerger does NOT merge raw strings.z0Cannot merge strings which have pragma comments.rz,Not enough strings to merge (num_of_strings=z).z!Too many inline string comments (r_rzToo many different prefixes ()rrNrOr1r}r r?setCOMMArrr.rRr-raddrrMr)) r@rincrZfound_sa_commentrZnum_of_inline_string_commentsZset_of_prefixesrrjrgr9r9r:rsZ           zStringMerger._validate_msg)r5r6r7r8rrrwr rcr rrzrrrr2rrdrrrr9r9r9r:rjs$& * 8 rc@sTeZdZdZeedddZeeee e edddZ eeeedd d Z d S) StringParenStrippera)StringTransformer that strips surrounding parentheses from strings. Requirements: The line contains a string which is surrounded by parentheses and: - The target string is NOT the only argument to a function call. - The target string is NOT a "pointless" string. - If the target string contains a PERCENT, the brackets are not preceded or followed by an operator with higher precedence than PERCENT. Transformations: The parentheses mentioned in the 'Requirements' section are stripped. Collaborations: StringParenStripper has its own inherent usefulness, but it is also relied on to clean up the parentheses created by StringParenWrapper (in the event that they are no longer needed). ruc Cs*|j}t|}g}d}|d7}|t|kr.q||}|jtjkrDq|jrd|jjrd|jjjtjkrdq||dr||djtj kst ||drq||dr||djtj ks||djt vrq|}t }|||} ||dr|||d} tjdd||d| Dvr|| jtjtjtjtjtjtjtjtjtjtj h vs| jr|| jjtjkr|| jtjtjhvr|q|| r|| jtjkrt|| s|| dr|| djtjtjtj tjhvrq|||}|t|dkr||djtjkr|d7}qq|r"t|St dS)NrErDrcSsh|] }|jqSr9rOr~r9r9r: Rrz/StringParenStripper.do_match..z+This line has no strings wrapped in parens.)!rNrrMrOr1r}rr'Z simple_stmtrKr!rPr StringParserparsePERCENTSTARATSLASH DOUBLESLASHr[rbAWAITrLZfactorrYrZrIr#rQrhr)r?) rsr@rrryrlrjr string_parserZnext_idxZ before_lparr9r9r:rws      "   $zStringParenStripper.do_matchrxc cs|j}g}|D]X}t}|||}d}||d||fD]} || rrz0BaseStringSplitter._validate..z This string (z/) appears to be pointless (i.e. has no parent).ziLine appears to end with an inline pragma comment. Splitting the line could modify the pragma's behavior.z"We cannot split multiline strings.N)rN_get_max_string_lengthrMrRr?rrr1r}NEWLINErrrr.r))rsr@rrrmax_string_lengthr9r9r:rs.    zBaseStringSplitter._validatecCs*|j}t|}|jd}||dr|d}||djtjkrb||djdkrb|dkrb|d8}||}|j|jvr|tt |d7}|jtj kr|d7}|jtj tj tj tjfvr|d7}t|d|dD]"}|tt |7}|jtvrqq||dr||d} | jtjkrJ| jdkrJt||dkrJ||d} | jtj kr`|d7}||dr||d} | jtjkr| jtjkr|d7}||dr||djtjkr|d7}|t| j7}d} |||D]&} | sd} |d7}|t| j7}qtt ||j|} | S) az Calculates the max string length used when attempting to determine whether or not the target string is responsible for causing the line to go over the line length limit. WARNING: This method is tightly coupled to both StringSplitter and (especially) StringParenWrapper. There is probably a better way to accomplish what is being done here. Returns: max_string_length: such that `line.leaves[string_idx].value > max_string_length` implies that the target string IS responsible for causing this line to exceed the line length limit. rDr_rrNFT)rNrdepthrOr1rKrRrrMrrCOLONEQUAL PLUSEQUALrPreversedrrIrQrir,rp)rsr@rrroffsetZp_idxPrjNZNNZ has_commentsrmrr9r9r:r)s^      ,   z)BaseStringSplitter._get_max_string_lengthrr=cCs|djtjkrdS|dj}|dj}|sd|sdt|dtjkrd|dj}|dusXJ|j}|j}|rt|jtj kr|r|jtj krdSdS)ac Returns: string_idx such that @LL[string_idx] is equal to our target (i.e. matched) string, if this line matches the "prefer paren wrap" statement requirements listed in the 'Requirements' section of the StringParenWrapper class's docstring. OR None, otherwise. rN) rOr1r} prev_sibling next_siblingr%r'Zatomrr)rrrrr9r9r:_prefer_paren_wrap_matchs      z+BaseStringSplitter._prefer_paren_wrap_match)r5r6r7r8r1EQEQUALGREATER GREATEREQUALLESS LESSEQUALNOTEQUALrrYrrrrrrrrrwrcrrrrr r2rrr9r9r9r:rs$   +mr)sr=ccsFg}d}|t|krB||dkrd|sP|dt|krP||ddkrP|d7}q|||d7}q||dkr|s~|d7}q|}|s||dfV|d7}q|r8d}|||ddvr|||d}n||d vr||}|r8|t|7}|t|kr*|||t||kr*|d7}q|t|7}q|d7}qdS) z Yields spans corresponding to expressions in a given f-string. Spans are half-open ranges (left inclusive, right exclusive). Assumes the input string is a valid f-string, but will not crash if the input string is invalid. r{rDr}Nr)z'''z""")'")rMrhpop)rstackrjdelimr9r9r:iter_fexpr_spanss@ $    (  rcCs tt|Srr)rr)rr9r9r:fstring_contains_exprsr)fstring old_quoter=cCsv|dkr dnd}g}d}t|D]8\}}|||||||||||}q |||dd|S)a; Toggles quotes used in f-string expressions that are `old_quote`. f-string expressions can't contain backslashes, so we need to toggle the quotes if the f-string itself will end up using the same quote. We can simply toggle without escaping because, quotes can't be reused in f-string expressions. They will fail to parse. NOTE: If PEP 701 is accepted, above statement will no longer be true. Though if quotes can be reused, we can simply reuse them without updates or escaping, once Black figures out how to parse the new grammar. rrrNr_)rrhrjoin)rrZ new_quotepartsZprevious_indexr\endr9r9r:rs rc@seZdZUdZdZeed<eedddZ ee e e e eddd Zee eeefd d d Zee eeefd d dZeeed ddZee ee dddZeddddZeeedddZeee edddZdS)StringSplittera^ StringTransformer that splits "atom" strings (i.e. strings which exist on lines by themselves). Requirements: * The line consists ONLY of a single string (possibly prefixed by a string operator [e.g. '+' or '==']), MAYBE a string trailer, and MAYBE a trailing comma. AND * All of the requirements listed in BaseStringSplitter's docstring. Transformations: The string mentioned in the 'Requirements' section is split into as many substrings as necessary to adhere to the configured line length. In the final set of substrings, no substring should be smaller than MIN_SUBSTR_SIZE characters. The string will ONLY be split on spaces (i.e. each new substring should start with a space). Note that the string will NOT be split on a space which is escaped with a backslash. If the string is an f-string, it will NOT be split in the middle of an f-expression (e.g. in f"FooBar: {foo() if x else bar()}", {foo() if x else bar()} is an f-expression). If the string that is being split has an associated set of custom split records and those custom splits will NOT result in any line going over the configured line length, those custom splits are used. Otherwise the string is split as late as possible (from left-to-right) while still adhering to the transformation rules listed above. Collaborations: StringSplitter relies on StringMerger to construct the appropriate CustomSplit objects and add them to the custom split map. MIN_SUBSTR_SIZErucCs|j}||durtdSt|}d}||r||dr||j||djgtjtjgkrt||t||ddkr|d7}n@||r||j|jvs||jtjkrt||dkr|d7}||rt ||r|d7}||r||jtj kr tdS|}t }| ||}||r@t ||r@|d7}||rd||jtjkrd|d7}||rvtdSt|gS) Nz)Line needs to be wrapped in parens first.rrDznot inrinz"Line does not start with a string.z%This line does not end with a string.)rNrr?rrOr1rPrrr!r}rrr#rr))rsr@rrrlrrr9r9r:r)sH "   z StringSplitter.do_splitter_matchrxc#sjt|dks,Jjjdt||d}|jd}t}t|}t|j}d|vo|t |j}d  rt dd Ddnd t dd  fd d } ||do|dj tjktd  fd d jd8jd8dkrBtd|jdjVdS|jtontfddD |jtd  fdd } g} | rΈ rd} | j} nLt }|}|durrΈ|jg} dd qq|} d| |} rZ| jsZ|||ksB|||krZ| d7} d| |}|rl||}ttj|}||| }| ||!|| !t"|||| ddq| EdH|r|ttj}||| }| |||dr}|ddD]&}|t#|7}|j tj$kr:qbq:t%|ks|dj tjkr|!|t&||ddt"|Vn>|!|t"|V }t&||ddt"|Vn |!|j'(|_'t"|VdS)NrDrrrErTcss|]}tt|VqdSrr)rMr)r prefix_leafr9r9r:r|rz.StringSplitter.do_transform..)rkr=cs:rng}t|D] \}}t||||qdS)a\ Side Effects: If @line starts with a string operator and this is the first line we are constructing, this function appends the string operator to @new_line and replaces the old string operator leaf in the node structure. Otherwise this function does nothing. N)rfr&rh)rkZmaybe_prefix_leavesrr )rfirst_string_linestring_op_leavesr9r:maybe_append_string_operatorss zBStringSplitter.do_transform..maybe_append_string_operatorsr=cs0j}|jd8}|rdnd8}|8}|S)a; Returns: The max allowed width of the string value used for the last line we will construct. Note that this value means the width rather than the number of characters (e.g., many East Asian characters expand to two columns). rrDr)rpr)result)ends_with_commar@rsstring_op_leaves_lengthr9r:max_last_string_columns z;StringSplitter.do_transform..max_last_string_columnrzUnable to split z at such high of a line depth: c3s|]}|jkVqdSrr)r)rcsplit)max_break_widthr9r:rrcs"rtdkStkSdS)z Returns: True iff `rest_value` (the remaining string value from the last split), should be split again. rDN)rMr0r9)rr rest_valueuse_custom_breakpointsr9r:more_splits_should_be_mades z?StringSplitter.do_transform..more_splits_should_be_madeF))rNrMrr5rRrinsert_str_child_factoryr-rr_get_string_operator_leavessumrrOr1rrcrprr?rrdallrrr,_get_break_idxr_normalize_f_stringr2r}_maybe_normalize_string_quotesrerhr)rrKr0rrr)rsr@ryrrrinsert_str_childrgZdrop_pointless_f_prefixrrZstring_line_resultsrrZmax_bidxZmaybe_break_idxZ next_valueZ next_leaf next_lineZ rest_leaf last_lineZ temp_valuerjZnon_string_liner9) rrrr r@rrrrsrrrr:rz_s                               zStringSplitter.do_transformrccsd}tt|}|D]p\}}|dkr,| }q|r8|dkr>d}qd}|d}|D]\}}|dkrN|}qzqNt|jjd||fVqdS)z Yields: All ranges of @string which, if @string were to be split there, would result in the splitting of an \N{...} expression (which is NOT allowed). F\rrDr LOGIC ERROR!N)iterrf RuntimeErrorrr5)rsrZ previous_was_unescaped_backslashitrlcbeginrr9r9r:_iter_nameescape_slicesGs"     z&StringSplitter._iter_nameescape_slicesccs&dt|vrdSt|EdHdS)z Yields: All ranges of @string which, if @string were to be split there, would result in the splitting of an f-expression (which is NOT allowed). rN)r-rr)rsrr9r9r:_iter_fexpr_slicesfsz!StringSplitter._iter_fexpr_slicescCsJt}||||g}|D]&}|D]\}}|t||dq&q|SNrD)rr,r+updater)rsrZillegal_indices iteratorsr(r*rr9r9r:_get_illegal_split_indicesqs z)StringSplitter._get_illegal_split_indices)r max_break_idxr=cst|sJtttdfdd ttdfdd }|}|drv||sv|d8}qX||s|d}|dr||s|d7}q|r||sdS|S)a This method contains the algorithm that StringSplitter uses to determine which character to split each string at. Args: @string: The substring that we are attempting to split. @max_break_idx: The ideal break index. We will return this value if it meets all the necessary conditions. In the likely event that it doesn't we will try to find the closest index BELOW @max_break_idx that does. If that fails, we will expand our search by also considering all valid indices ABOVE @max_break_idx. Pre-Conditions: * assert_is_leaf_string(@string) * 0 <= @max_break_idx < len(@string) Returns: break_idx, if an index is able to be found that meets all of the conditions listed in the 'Transformations' section of this classes' docstring. OR None, otherwise. )rr=cs|vS)z Returns: True iff returning @i would result in the splitting of an unsplittable expression (which is NOT allowed). r9)r)_illegal_split_indicesr9r:breaks_unsplittable_expressionszEStringSplitter._get_break_idx..breaks_unsplittable_expressioncs|dk}|do&|dtv}d}|d}|rX|dkrX| }|d8}q4t|djkotd|jk}|s|o|o|o| S)z Returns: True iff ALL of the conditions listed in the 'Transformations' section of this classes' docstring would be be met by returning @i.  rDTr$N)SPLIT_SAFE_CHARSrMr )rZis_spaceZ is_split_safeZis_not_escapedrZ is_big_enough)r3rrsrr9r:passes_all_checkss"  z8StringSplitter._get_break_idx..passes_all_checksrDN)rr+r0Indexrd)rsrr1r6rr9)r2r3rrsrr:r|s     zStringSplitter._get_break_idxNrjr=cCs|jrt|j|_dSrr)rqr/rR)rsrjr9r9r:r sz-StringSplitter._maybe_normalize_string_quotes)rrgr=cCsht|d|vr`t|s`|dd}|t|d}tdd|}tdd|}|}||S|SdS)a Pre-Conditions: * assert_is_leaf_string(@string) Returns: * If @string is an f-string that contains no f-expressions, we return a string identical to @string except that the 'f' prefix has been stripped and all double braces (i.e. '{{' or '}}') have been normalized (i.e. turned into '{' or '}'). OR * Otherwise, we return @string. rr_Nz\{\{rz\}\}r)r+rrrMrr)rsrrg new_prefixtempZ new_stringr9r9r:rs   z"StringSplitter._normalize_f_string)rNr=cCs\t|}g}d}||j|jtjgvrXt||jt||}|||d7}q|S)NrrD) rrOrr1rPr2rstriprh)rsrNrrrr r9r9r:rs  z*StringSplitter._get_string_operator_leaves)r5r6r7r8r rrrrrr rcr rrzrrr7r+r,rr0rrr2r rr rr9r9r9r:rs % 7 i  Urc@seZdZdZeedddZeee e e dddZ eee e e ddd Z eee e e dd d Zeee e e dd d Zeee e e dddZeee eeedddZdS)StringParenWrappera StringTransformer that wraps strings in parens and then splits at the LPAR. Requirements: All of the requirements listed in BaseStringSplitter's docstring in addition to the requirements listed below: * The line is a return/yield statement, which returns/yields a string. OR * The line is part of a ternary expression (e.g. `x = y if cond else z`) such that the line starts with `else `, where is some string. OR * The line is an assert statement, which ends with a string. OR * The line is an assignment statement (e.g. `x = ` or `x += `) such that the variable is being assigned the value of some string. OR * The line is a dictionary key assignment where some valid key is being assigned the value of some string. OR * The line is an lambda expression and the value is a string. OR * The line starts with an "atom" string that prefers to be wrapped in parens. It's preferred to be wrapped when the string is surrounded by commas (or is the first/last child). Transformations: The chosen string is wrapped in parentheses and then split at the LPAR. We then have one line which ends with an LPAR and another line that starts with the chosen string. The latter line is then split again at the RPAR. This results in the RPAR (and possibly a trailing comma) being placed on its own line. NOTE: If any leaves exist to the right of the chosen string (except for a trailing comma, which would be placed after the RPAR), those leaves are placed inside the parentheses. In effect, the chosen string is not necessarily being "wrapped" by parentheses. We can, however, count on the LPAR being placed directly before the chosen string. In other words, StringParenWrapper creates "atom" strings. These can then be split again by StringSplitter, if necessary. Collaborations: In the event that a string line split by StringParenWrapper is changed such that it no longer needs to be given its own line, StringParenWrapper relies on StringParenStripper to clean up the parentheses it created. For "atom" strings that prefers to be wrapped in parens, it requires StringSplitter to hold the split until the string is wrapped in parens. rucCs|j}|jdjtvrtdS||pX||pX||pX||pX||pX| |}|dur|j|j }t dd|Ds|j |j dd}t||kr||stdSt|gStdS) NrEzACannot wrap parens around a line that ends in an opening bracket.css|]}|dkp|tvVqdS)r4N)r5)rcharr9r9r:rIsz7StringParenWrapper.do_splitter_match..rDrzWe do not wrap long strings in parentheses when the resultant line would still be over the specified line length and can't be split further by StringSplitter.z2This line does not contain any non-atomic strings.)rNrOrr? _return_match _else_match _assert_match _assign_match_dict_or_lambda_matchrrRrrprr0rr))rsr@rr string_valueZmax_string_widthr9r9r:r5s:      z$StringParenWrapper.do_splitter_matchrcCsjt|dtjtjfvrf|djdvrft|}|drFt|drFdnd}||rf||jtj krf|SdS)aK Returns: string_idx such that @LL[string_idx] is equal to our target (i.e. matched) string, if this line matches the return/yield statement requirements listed in the 'Requirements' section of this classes' docstring. OR None, otherwise. r)r=yieldrDrN) r%r'Z return_stmtZ yield_exprrRrr"rOr1r}rrrlr9r9r:r>[s z StringParenWrapper._return_matchcCstt|dtjkrp|djtjkrp|djdkrpt|}|drPt|drPdnd}||rp||jtj krp|SdS)aG Returns: string_idx such that @LL[string_idx] is equal to our target (i.e. matched) string, if this line matches the ternary expression requirements listed in the 'Requirements' section of this classes' docstring. OR None, otherwise. relserDrN) r%r'testrOr1rPrRrr"r}rEr9r9r:r?ts zStringParenWrapper._else_matchcCst|dtjkr|djdkrt|}t|D]r\}}|jtjkr0t ||dr\|dn|d}||r0||jtj kr0|}t }| ||}||s0|Sq0dS)aE Returns: string_idx such that @LL[string_idx] is equal to our target (i.e. matched) string, if this line matches the assert statement requirements listed in the 'Requirements' section of this classes' docstring. OR None, otherwise. rassertrDrN) r%r'Z assert_stmtrRrrfrOr1rr"r}rrrrrrjrlrrr9r9r:r@s     z StringParenWrapper._assert_matchcCst|dtjtjtjfvr|djtjkrt|}t |D]\}}|jtj tj fvr|t |dkr>t ||drz|dn|d}||r>||jtj kr>|}t }|||}||r||jtjkr|d7}||s>|Sq>dS)am Returns: string_idx such that @LL[string_idx] is equal to our target (i.e. matched) string, if this line matches the dictionary key assignment statement or lambda expression requirements listed in the 'Requirements' section of this classes' docstring. OR None, otherwise. rrDrN)r%rr'Z dictsetmakerZlambdefrrfrOr1rrMr"r}rrr)rZ parent_typesrrrjrlrrr9r9r:rBs    z(StringParenWrapper._dict_or_lambda_matchrxccs|j}t|dks,J|jjdt||d}t|}t||}d}d}||jtjkrdd}||g} |r| ||| } |d|} d} | r| djtj krd} | | d| t | || ttj d} | rt||d| n|| | | | D]&}||D]}| j |ddqq t| V||j}t|j|jdd|j|jd }ttj|}||| |d}||dr||dd}|r| | r|r|djtjksJd | d |d | }n|r|djtjkr|dj}|dur|| vr| |}|dkr|t| dkr| |djtjkr| |djd kr| t |||t|V| }| j|_ttjd }|durt||n||| ||rttjd}t|||| |t|VdS)NrDrrrEFTrra)rBrZinside_bracketsshould_split_rhsmagic_trailing_commaz8Apparently, old parentheses do NOT exist?! (left_leaves=z, right_leaves=rr`,)rNrMrr5rrrOr1rrhrerKrrr2r&rir)rRrrBrrLrMr}rIopening_bracketrFrZbracket_tracker)rsr@ryrrrr!Z comma_idxrZleaves_to_steal_comments_from first_lineZ left_leavesZold_parens_existZ lpar_leafrjrmrCZ string_linerZ old_rpar_leafZ right_leavesrOrFr#Z new_rpar_leafZ comma_leafr9r9r:rzs                      zStringParenWrapper.do_transformN)r5r6r7r8rrrrr r2rrcr>r?r@rArBr rrzr9r9r9r:r<s8&".' r<c@s,eZdZUdZdZeed<dZeed<dZeed<dZ eed <d Z eed <d Z eed <dZ eed<dZ eed<dZeed<eejfeeej fe eefeeej fe e ej fe e efee ej fe e efe e efee efei Zeeeeefefed<ddddZeeeedddZeedddZdS)ra A state machine that aids in parsing a string's "trailer", which can be either non-existent, an old-style formatting sequence (e.g. `% varX` or `% (varX, varY)`), or a method-call / attribute access (e.g. `.format(varX, varY)`). NOTE: A new StringParser object MUST be instantiated for each string trailer we need to parse. Examples: We shall assume that `line` equals the `Line` object that corresponds to the following line of python code: ``` x = "Some {}.".format("String") + some_other_string ``` Furthermore, we will assume that `string_idx` is some index such that: ``` assert line.leaves[string_idx].value == "Some {}." ``` The following code snippet then holds: ``` string_parser = StringParser() idx = string_parser.parse(line.leaves, string_idx) assert line.leaves[idx].type == token.PLUS ``` ic4 DEFAULT_TOKENrDSTARTrrQrrPrrSINGLE_FMT_ARGr rKrIrDONE_gotoNrcCs|j|_d|_dS)Nr)rR_state_unmatched_lpars)rsr9r9r:rtszStringParser.__init__)rNrr=cCsD||jtjksJ|d}|t|kr@|||r@|d7}q|S)a6 Pre-conditions: * @leaves[@string_idx].type == token.STRING Returns: The index directly after the last leaf which is apart of the string trailer, if a "trailer" exists. OR @string_idx + 1, if no string "trailer" exists. rD)rOr1r}rM _next_state)rsrNrrlr9r9r:rs  zStringParser.parser8cCst|r dS|j}|tjkr*|jd7_|j}||jkrf|tjkr|jd8_|jdkr|j|_nf||f|jvr|j||f|_n6||jf|jvr|j||jf|_nt |j j d|j|j krdSdS)a Pre-conditions: * On the first call to this function, @leaf MUST be the leaf that was directly after the string leaf in question (e.g. if our target string is `line.leaves[i]` then the first call to this method must be `line.leaves[i + 1]`). * On the next call to this function, the leaf parameter passed in MUST be the leaf directly following @leaf. Returns: True iff @leaf is apart of the string's trailer. TrDrr%F) r"rOr1rKrYrXrIrWrQr'rr5rV)rsrjZ next_tokenZ current_stater9r9r:rZs&      zStringParser._next_state)r5r6r7r8rQrrrRrQrPrrTrKrIrVr1rWr r ParserStateNodeTypertr r2rcrrdrZr9r9r9r:rs0                r)rr=cs(|j|tddfdd }|S)a Factory for a convenience function that is used to orphan @string_leaf and then insert multiple new leaves into the same part of the node structure that @string_leaf had originally occupied. Examples: Let `string_leaf = Leaf(token.STRING, '"foo"')` and `N = string_leaf.parent`. Assume the node `N` has the following original structure: Node( expr_stmt, [ Leaf(NAME, 'x'), Leaf(EQUAL, '='), Leaf(STRING, '"foo"'), ] ) We then run the code snippet shown below. ``` insert_str_child = insert_str_child_factory(string_leaf) lpar = Leaf(token.LPAR, '(') insert_str_child(lpar) bar = Leaf(token.STRING, '"bar"') insert_str_child(bar) rpar = Leaf(token.RPAR, ')') insert_str_child(rpar) ``` After which point, it follows that `string_leaf.parent is None` and the node `N` now has the following structure: Node( expr_stmt, [ Leaf(NAME, 'x'), Leaf(EQUAL, '='), Leaf(LPAR, '('), Leaf(STRING, '"bar"'), Leaf(RPAR, ')'), ] ) N)childr=cs0dus JdusJ|d7dSr-)r)r]Zstring_child_idxZ string_parentr9r:r!D s   z2insert_str_child_factory..insert_str_child)rrLN)rr!r9r^r:r s. r)seqr=csttdfdd }|S)a Examples: ``` my_list = [1, 2, 3] is_valid_index = is_valid_index_factory(my_list) assert is_valid_index(0) assert is_valid_index(2) assert not is_valid_index(3) assert not is_valid_index(-1) ``` )rlr=csd|kotkSS)zx Returns: True iff @idx is positive AND seq[@idx] does NOT raise an IndexError. r)rM)rlr`r9r:r` sz.is_valid_index_factory..is_valid_index)rcrd)r`rr9rar:rP sr)br8rsysabcrr collectionsrZ dataclassesrtypingrrrr r r r r rrrrrr version_infoZtyping_extensionsrrZmypy_extensionsrZblack.commentsrZ black.linesrrZ black.moderrZ black.nodesrrr r!r"r#r$r%r&r'Z black.rustyr(r)r*Z black.stringsr+r,r-r.r/r0Zblib2to3.pgen2r1Zblib2to3.pytreer2r3 Exceptionr4r;r_Z Transformerrcr7r\r[rrr frozensetr5rr?rnrorrrrrrrdrrrr<rrrr9r9r9r:sz  @   0         Pj=!9w/~ =