HG problem: abort: decoding near '9: error: ‘sse2::l': 'ascii' codec can't decode byte 0xe2 in position 213: ordinal not in range(128)

Discussion:

ISHIKAWA,chiaki

2018-12-02 07:53:16 UTC

Permalink

Hi,

I am developing mozilla patches locally under my local PC running linux.

I often include the GCC's error/warning message in the mercurial
commit message (in the second and subsequent lines, the first line is
for usage by mozilla's bugzilla procedure.) when my local patch is
created to address build error or suppress compiler warning.

It worked just fine for months/years.

After an upgrade of local tools under Debian GNU/Linux last month or
so, I just noticed that such a patch with GCC warning line could not
be re-applied after it was qpopped. (I use mercurial mqueue
extension.)

This happened yesterday. This is the error message.

note: commit message saved in .hg/last-message.txt
transaction abort!
rollback completed
abort: decoding near '9: error: ‘sse2::l': 'ascii' codec can't decode
byte 0xe2 in position 213: ordinal not in range(128)!

I was puzzled. After looking at the patch and the message above, I
finally figured out that that hg barfed on the proper quote character
just before "sse2::l" above. Can you spot the slightly tilted OPENING
quote character?

So I removed the GCC warning line from the commit message and the
patch could be applied. This is problematic because I have about a few
dozen patches (actually more) and some patches could not be re-applied
due to the error above. I had to temporarily removed the offending
patch(es). Luckily the offending patches are relatively independent
and taken out for the moment.
But this could not be done if the patches are interdependent and
must be applied in a certain order.

Either
- GCC began producing such a non-ascii character lately (it is v7 on
my linux PC)
[but I think I must have seen such characters long time before that],
- the "ascii" codec or whatever used by python has become very strict
(But why ascii? Should we not use utf-8 after all?)
[this could be an issue with setup of hg under my Debian GNU/Linux
distribution?] , or
- something in my locale/character setting under Debian GNU/Linux
broke lately due to the upgrade of local packages.

My point is that I have NOT changed in my personal setting at all
(unless |./mach bootstrap| I occasionally run to make sure that I have
the latest rust and other tools did something funny to my .hgrc).
Only the tools and system-wide setting could have changed mysteriously
from my point of view.

I found a similar bug report in the following open source project:
https://github.com/schacon/hg-git/issues/27

I have not tried the HGENCODING=utf-8 fix mentioned in the above URL
for fear of breaking existing patches against my local copy of the
mozilla-central and comm-central source tree.

I wonder what would be the ultimate correct fix or
procedure to avoid/suppress this error.

I am posting this to warn developers who *MAY* see similar issues,
and to figure out where I should contact and report the issue:
- mercurial developers?
- python developers?
- who else?

I understand that mozilla is a big user of mercurial and so it should have
some clout to figure out the correct answer is quickly. :-)

TIA

Happy Hacking in the good old sense of the phrase...

from a very perplexed linux user.

Dave Townsend

2018-12-02 16:18:43 UTC

Permalink

I would report this to mercurial.

Post by ISHIKAWA,chiaki
Hi,
I am developing mozilla patches locally under my local PC running linux.
I often include the GCC's error/warning message in the mercurial
commit message (in the second and subsequent lines, the first line is
for usage by mozilla's bugzilla procedure.) when my local patch is
created to address build error or suppress compiler warning.
It worked just fine for months/years.
After an upgrade of local tools under Debian GNU/Linux last month or
so, I just noticed that such a patch with GCC warning line could not
be re-applied after it was qpopped. (I use mercurial mqueue
extension.)
This happened yesterday. This is the error message.
note: commit message saved in .hg/last-message.txt
transaction abort!
rollback completed
abort: decoding near '9: error: ‘sse2::l': 'ascii' codec can't decode
byte 0xe2 in position 213: ordinal not in range(128)!
I was puzzled. After looking at the patch and the message above, I
finally figured out that that hg barfed on the proper quote character
just before "sse2::l" above. Can you spot the slightly tilted OPENING
quote character?
So I removed the GCC warning line from the commit message and the
patch could be applied. This is problematic because I have about a few
dozen patches (actually more) and some patches could not be re-applied
due to the error above. I had to temporarily removed the offending
patch(es). Luckily the offending patches are relatively independent
and taken out for the moment.
But this could not be done if the patches are interdependent and
must be applied in a certain order.
Either
- GCC began producing such a non-ascii character lately (it is v7 on
my linux PC)
[but I think I must have seen such characters long time before that],
- the "ascii" codec or whatever used by python has become very strict
(But why ascii? Should we not use utf-8 after all?)
[this could be an issue with setup of hg under my Debian GNU/Linux
distribution?] , or
- something in my locale/character setting under Debian GNU/Linux
broke lately due to the upgrade of local packages.
My point is that I have NOT changed in my personal setting at all
(unless |./mach bootstrap| I occasionally run to make sure that I have
the latest rust and other tools did something funny to my .hgrc).
Only the tools and system-wide setting could have changed mysteriously
from my point of view.
https://github.com/schacon/hg-git/issues/27
I have not tried the HGENCODING=utf-8 fix mentioned in the above URL
for fear of breaking existing patches against my local copy of the
mozilla-central and comm-central source tree.
I wonder what would be the ultimate correct fix or
procedure to avoid/suppress this error.
I am posting this to warn developers who *MAY* see similar issues,
- mercurial developers?
- python developers?
- who else?
I understand that mozilla is a big user of mercurial and so it should have
some clout to figure out the correct answer is quickly. :-)
TIA
Happy Hacking in the good old sense of the phrase...
from a very perplexed linux user.
_______________________________________________
dev-platform mailing list
https://lists.mozilla.org/listinfo/dev-platform

ISHIKAWA,chiaki

2018-12-04 09:05:30 UTC

Permalink

Post by Dave Townsend
I would report this to mercurial.

Thank you very much.

Looking forward to hearing the answer (maybe this could be
HGENCODING=utf-8 after all. )

I have a suspicion that python's locale handler, especially the one that
is involved in iconv-like code conversion has become either stricter or
buggier in the transition from v2 to v3.

This is based on my observation of another program broken in subtle manner:
I have used a very useful visual merge tool called "meld" for more than
a year until this spring.
That runs under linux and is written in python.
This fall, when I tried to use it again to manage the merge of local
changes to mozilla TB so that they can be applied to newer TB source
tree I fetched from comm-central, I notice that "meld" fail to handle a
few local files that contain non-ASCII character. They are local memo
files. It is very annoying and again the problem was traced to
encode/decode in python runtime.

Thank you again.

Post by Dave Townsend

_______________________________________________
dev-platform mailing list
https://lists.mozilla.org/listinfo/dev-platform