Avoid "str | bytes" where names or text are meant

[matangover]

Right now _Element.text is typed as Optional[Union[str, bytes]]. However according to the [lxml FAQ]:(https://lxml.de/3.6/FAQ.html):

In Python 3, lxml always returns Unicode strings for text and names, as does ElementTree.

Having to deal with bytes makes all code accessing _Element.text unnecessarily cumbersome.

[matangover]

This is also relevant to _Element.attrib, whose __getitem__ (among others) is typed as returning Union[str, bytes]. I'm not sure whether it can actually contain bytes values (couldn't find info in docs). However right now there is a discrepancy: _Element.get is typed as returning str. So one of those must be wrong.

[scoder]

The thing is, as long as users are still running their code in Py2, and writing code that works with Py2 and Py3, both need to be supported. Users who only target Py2 and not Py3 at all would probably also not use static type checkers, but legacy Py2/3 code should be supported.

That said, yes, the current declarations are inconsistent.

And the fact that legacy code needs to be supported doesn't mean that str wouldn't be sufficient. lxml does not use bytes as in "sequence of bytes" but only as byte encoded textual data, i.e. Py2 str, not Py3 bytes.

I think it's worth trying a switch to plain str and seeing if that really breaks anything.

PR welcome.

[lovetox]

This seems to be changed in d95ca46

Issue can be closed in my opinion

[scoder]

I changed the title to reflect the general issue here.

[matangover]

I think it's worth trying a switch to plain str and seeing if that really breaks anything.

It would break correctness on Python 2 because on Python 2 these attributes may return either str or unicode. Can we drop Python 2 compatibility of the typings? If yes, we can switch to str.

If you want to keep Python 2 compatibility, we can use Union[typing.Text, str] or use a version-dependent type alias.

[abelcheung]

@matangover Agreed it's a good idea to drop py2 support for annotations. Personally I don't know anybody who's writing new code on py2 now, lest using annotation on py2 project.

However, the removal of bytes is mostly for method / function return type and object attributes. bytes as input argument is still in use, I'm not sure dropping that is good idea or not.

[matangover]

Agreed, str should be used for places where "names or text" are returned / accepted.

[macro1]

Per python/typing#208 an instance of Python 2 str is a subtype of unicode. Are we over thinking this, and would typing.Text just work for 2.7 type checking? It seems to me the only thing we need to do is indicate for Python 3 where bytes are acceptable as an input as well.

[5j9]

The latest release of lxml only supports 3.6+. Support for Python 2.7 and Python versions < 3.6 was removed in v5.1.0 (2024-01-05).