In the UTF-16 encoding scheme, U+FEFF at the very beginning of a file or stream explicitly signals the byte order.
So it is perfectly permissible to perform this type of processing if you consider a string to be equivalent to a stream.—Evan
The programmer or the application might, but Python’s codecs don’t. The point is that this is also true of rawstrings that happen to contain UTF-16 or UTF-32 data. The UTF-16 ("auto-endian") codec shouldn’t strip leading BOMs either, unless it has been told it has the beginning of the string.—Stephen