Skip to content

Minor request: \v for vertical spacing #477

Closed

Description

@dchaplinsky

Hi!

I'm using the regex lib to make a port of language tool libs (originally java) for sentence and word tokenization.
Those are relying on \v\h heavily. Some of those rules are shipped in the xml files full of regexes and I'm willing not to alter those to not to maintain a separate copy. I can kind of workaround it by replacing \v with VERTICAL_SPACE: str = "\u000a\u000b\u000c\u000d\u0085\u2028\u2029" but it's another tiny nightmare, as those regexes can come in different fashions: \v*, [\v\t]*, etc.

Please review the possibility to add the \v flag

Activity

  1. dchaplinsky commented on Aug 15, 2022

    @dchaplinsky
    Author

    I can see that code suggest \v pseudo, but I cannot understand why it doesn't work then:

    In [3]: import regex as re
    
    In [8]: re.search(r"\v", "\n") is None
    Out[8]: True
    
    In [9]: re.search(r"\v", "\n", flags=re.M | re.U | re.V1) is None
    Out[9]: True
    
  2. mrabarnett commented on Aug 16, 2022

    @mrabarnett
    Owner

    \v already exists in Python as being short for \x0b (LINE TABULATION):

    >>> '\v'
    '\x0b'
    >>> '\v' == '\N{LINE TABULATION}'
    True
  3. dchaplinsky commented on Aug 16, 2022

    @dchaplinsky
    Author

    Thanks for the prompt reply!

    Any ideas on the matching of vertical space?

  4. mrabarnett commented on Aug 16, 2022

    @mrabarnett
    Owner

    There are far fewer characters that need to match: [\x0A\x0B\x0C\x0D\x85\u2028\u2029] or [\x0A-x0D\x85\u2028\u2029].

    Maybe it could be added as \V, although that would be inconsistent with \h, and there are pairs of lowercase/uppercase escape codes where the uppercase one is the negative of the lowercase one, e.g. \d and \D. On the other hand, those implementations that have \h and \v don't have \H and \V.

    Also, I don't want to add something that the re module might do differently if it were added later.

    That's why it hasn't been added already.

  5. dchaplinsky commented on Aug 16, 2022

    @dchaplinsky
    Author
  6. mrabarnett commented on Aug 16, 2022

    @mrabarnett
    Owner

    I've come across a mention of \H and \V, so using \V would be a bad idea.

  7. dchaplinsky commented on Aug 16, 2022

    @dchaplinsky
    Author
  8. mrabarnett commented on Aug 16, 2022

    @mrabarnett
    Owner

    Now I'm thinking about \y and \Y, which look a little like \v and \V. ProgressSQL uses them instead of \b and \B, which every other implementation that I know of uses, possibly because \b normally represents \x08 outside regex, and does still within characters classes.

    I want the regex module to remain compatible with the re module, and just in case they ever get added there in the future, I'm soliciting opinions on python-dev.

  9. mrabarnett commented on Aug 17, 2022

    @mrabarnett
    Owner

    I've added \p{HorizSpace} (\p{H}) and \p{VertSpace} (\p{V}) in regex 2022.8.17, which is currently being built on GitHub and should arrive on PyPI soon.

  10. dchaplinsky commented on Aug 17, 2022

    @dchaplinsky
    Author

    Wow, many thanks!

  11. dchaplinsky commented on Oct 11, 2022

    @dchaplinsky
    Author
  12. mrabarnett commented on Oct 11, 2022

    @mrabarnett
    Owner

    Given the feedback on python-dev, I won't be adding \y and \Y. What I've already added should suffice.

  13. dchaplinsky commented on Oct 11, 2022

    @dchaplinsky
    Author
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions