I recently rediscovered this strange behaviour in Python’s Unicode handling.—Evan

Yes, but these are not file-like objects. In the IncrementalParser, it is not the case that a read operation returns an empty string. Instead, the application repeatedly feeds data explicitly. For a file-like object, returning "" indicates EOF.

Regards, Martin—"Martin

True, on the outside there are no file-like objects. But the IncrementalParser gets passed the XML bytes in chunks, so it has to use a stateful decoder for decoding. Unfortunately this means that is has to use a stream API. (See <python.org> for a patch that somewhat fixes that.)

(Another option would be to completely ignore the stateful API and handcraft stateful decoding (or only support stateless decoding), like most XML parsers for Python do now.)—Walter

Not neccassarily. In the example above the IncrementalParser gets fed a chunk of data, it stuffs this data into the Queue, so that the StreamReader can decode it. Once the data from the Queue is exhausted, there won’t any further data until the user calls feed() on the IncrementalParser again.

Bye,—Walter