[bug] Character display problem in mumi and my mail client

[Arun Isaac <arunisaac@systemreboot.net>]

[-- Attachment #1.1: Type: message/rfc822, Size: 4765 bytes --]

[-- Attachment #1.1.1.1: Type: text/plain, Size: 1620 bytes --]

Hi Guix,

It appears that mumi (or at least the instance of it running on 
issues.guix.gnu.org) has problems displaying some non-ASCII characters. I 
noticed it with '’'. Compare

https://issues.guix.gnu.org/issue/36207

with

https://debbugs.gnu.org/cgi/bugreport.cgi?bug=36207

In the former, Ludo's name displays as 'Ludo¢' while it the latter, it 
displays correctly as 'Ludo’'.

However, in Ludo's reply the character is displayed correctly.

Does this indicate that my mail client (alpine on Debian Stretch) is doing 
the wrong thing? It seems that mumi could handle this situation better 
since it debbugs appears to handle it correctly.

Looking at the raw mail downloaded from debbugs, I see that I'm sending 
mail as with the following encoding:

```
Content-Type: text/plain; FORMAT=flowed; CHARSET=ISO-8859-7
Content-Transfer-Encoding: 8BIT
```

while Ludo's mail is:

```
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
```

Interestingly, when I download the mbox file for my mail from debbugs, and 
looking at it Emacs with my en_us.UTF-8 locale, the cents-symbol appears. 
Using iconv to convert the file from ISO-8859-7 to UTF8, causes the 
correct character to display. So what looks to be happening is that mumi 
is interpreting my messages using the wrong encoding.

Thoughts? Is this something we want to fix?

This reminds me of rjbs's talk, "Email Hates the Living". [0]

Best,
Jack

[0] http://yapcasia.org/2011/talk/59

P.S. How are we tracking issues and patches for mumi? bug-guix@ and 
guix-patches?

[-- Attachment #1.2: Type: message/rfc822, Size: 6190 bytes --]

From: Ricardo Wurmus <rekado@elephly.net>
To: Jack Hill <jackhill@jackhill.us>
Cc: guix-devel@gnu.org
Subject: Re: Character display problem in mumi and my mail client
Date: Thu, 18 Jul 2019 10:00:55 +0200
Message-ID: <87zhlbg4aw.fsf@elephly.net>


Hi Jack,

thanks for the report.

> It appears that mumi (or at least the instance of it running on
> issues.guix.gnu.org) has problems displaying some non-ASCII
> characters. I noticed it with '’'. Compare
>
> https://issues.guix.gnu.org/issue/36207
>
> with
>
> https://debbugs.gnu.org/cgi/bugreport.cgi?bug=36207

The problem here is that the Debbugs SOAP service (which we use in
guile-debbugs and thus mumi) serves up a base64-encoded blob for the
email body without any information about encodings, so we just try UTF-8
and fall back to ISO 8859-1 if there’s an error.  If we could operate on
the actual email that would be different.

Unfortunately, the SOAP service does not provide access to the actual
emails.  That’s only available through … the official Debbugs web
interface.

I’m currently in the process of moving away from the SOAP service for
fetching message contents, because it’s just too painful.  There are too
many truncated or otherwise mangled messages, and in the end we are just
assembling them back to a good old email to parse them with Arun’s
guile-email.  Going forward mumi will only use the SOAP service to get
bug status information and pointers to messages.  The actual emails will
be fetched through the Debbugs web interface with much cursing.

> P.S. How are we tracking issues and patches for mumi? bug-guix@ and
> guix-patches?

Either of them would be fine.  Please add [mumi] to the subject line so
that it’s easier to distinguish them.

--
Ricardo



[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 487 bytes --]