From mboxrd@z Thu Jan 1 00:00:00 1970 From: Arun Isaac To: guile-email@systemreboot.net Subject: [bug] Character display problem in mumi and my mail client References: <87zhlbg4aw.fsf@elephly.net> Date: Thu, 18 Jul 2019 15:46:57 +0530 Message-ID: MIME-Version: 1.0 Content-Type: multipart/signed; boundary="=====-=-="; micalg=pgp-sha256; protocol="application/pgp-signature" List-Id: --=====-=-= Content-Type: multipart/mixed; boundary="=-=-=" --=-=-= Content-Type: message/rfc822 Content-Disposition: inline Received: from mail.systemreboot.net (mugam.systemreboot.net [139.59.75.54]) by localhost (mpop-1.4.4) with POP3 for ; Thu, 18 Jul 2019 14:21:57 +0530 Return-path: Envelope-to: arunisaac@systemreboot.net Delivery-date: Thu, 18 Jul 2019 02:54:05 +0530 Received: from [209.51.188.17] (helo=lists.gnu.org) by systemreboot.net with esmtps (TLSv1.2:ECDHE-RSA-AES256-GCM-SHA384:256) (Exim 4.92) (envelope-from ) id 1hnrPB-0000S0-Br for arunisaac@systemreboot.net; Thu, 18 Jul 2019 02:54:05 +0530 Received: from localhost ([::1]:60540 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.86_2) (envelope-from ) id 1hnrP8-0005tQ-EN for arunisaac@systemreboot.net; Wed, 17 Jul 2019 17:24:02 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:48808) by lists.gnu.org with esmtp (Exim 4.86_2) (envelope-from ) id 1hnrP6-0005t8-3J for guix-devel@gnu.org; Wed, 17 Jul 2019 17:24:01 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1hnrP4-0005Vn-Sn for guix-devel@gnu.org; Wed, 17 Jul 2019 17:24:00 -0400 Received: from minsky.hcoop.net ([104.248.1.95]:43260) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1hnrP4-0005VR-P6 for guix-devel@gnu.org; Wed, 17 Jul 2019 17:23:58 -0400 Received: from marsh.hcoop.net ([45.55.52.66]) by minsky.hcoop.net with esmtpsa (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.89) (envelope-from ) id 1hnrP4-0004pL-38 for guix-devel@gnu.org; Wed, 17 Jul 2019 17:23:58 -0400 Date: Wed, 17 Jul 2019 17:23:57 -0400 (EDT) From: Jack Hill X-X-Sender: jackhill@marsh.hcoop.net To: guix-devel@gnu.org Subject: Character display problem in mumi and my mail client Message-ID: User-Agent: Alpine 2.20 (DEB 67 2015-01-07) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 104.248.1.95 X-BeenThere: guix-devel@gnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "Development of GNU Guix and the GNU System distribution." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-devel-bounces+arunisaac=systemreboot.net@gnu.org Sender: "Guix-devel" MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="====-=-=" --====-=-= Content-Type: text/plain; charset=utf-8; format=flowed Content-Disposition: inline Content-Transfer-Encoding: quoted-printable Hi Guix, It appears that mumi (or at least the instance of it running on=20 issues.guix.gnu.org) has problems displaying some non-ASCII characters. I=20 noticed it with '=E2=80=99'. Compare https://issues.guix.gnu.org/issue/36207 with https://debbugs.gnu.org/cgi/bugreport.cgi?bug=3D36207 In the former, Ludo's name displays as 'Ludo=C2=A2' while it the latter, it= =20 displays correctly as 'Ludo=E2=80=99'. However, in Ludo's reply the character is displayed correctly. Does this indicate that my mail client (alpine on Debian Stretch) is doing= =20 the wrong thing? It seems that mumi could handle this situation better=20 since it debbugs appears to handle it correctly. Looking at the raw mail downloaded from debbugs, I see that I'm sending=20 mail as with the following encoding: ``` Content-Type: text/plain; FORMAT=3Dflowed; CHARSET=3DISO-8859-7 Content-Transfer-Encoding: 8BIT ``` while Ludo's mail is: ``` Content-Type: text/plain; charset=3Dutf-8 Content-Transfer-Encoding: quoted-printable ``` Interestingly, when I download the mbox file for my mail from debbugs, and= =20 looking at it Emacs with my en_us.UTF-8 locale, the cents-symbol appears.=20 Using iconv to convert the file from ISO-8859-7 to UTF8, causes the=20 correct character to display. So what looks to be happening is that mumi=20 is interpreting my messages using the wrong encoding. Thoughts? Is this something we want to fix? This reminds me of rjbs's talk, "Email Hates the Living". [0] Best, Jack [0] http://yapcasia.org/2011/talk/59 P.S. How are we tracking issues and patches for mumi? bug-guix@ and=20 guix-patches? --====-=-=-- --=-=-= Content-Type: message/rfc822 Content-Disposition: inline Received: from mail.systemreboot.net (mugam.systemreboot.net [139.59.75.54]) by localhost (mpop-1.4.4) with POP3 for ; Thu, 18 Jul 2019 14:21:59 +0530 Return-path: Envelope-to: arunisaac@systemreboot.net Delivery-date: Thu, 18 Jul 2019 13:46:20 +0530 Received: from [209.51.188.17] (helo=lists.gnu.org) by systemreboot.net with esmtps (TLSv1.2:ECDHE-RSA-AES256-GCM-SHA384:256) (Exim 4.92) (envelope-from ) id 1ho1aN-0001Gk-Df for arunisaac@systemreboot.net; Thu, 18 Jul 2019 13:46:19 +0530 Received: from localhost ([::1]:35398 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.86_2) (envelope-from ) id 1ho1aK-0004D5-3H for arunisaac@systemreboot.net; Thu, 18 Jul 2019 04:16:16 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:47816) by lists.gnu.org with esmtp (Exim 4.86_2) (envelope-from ) id 1ho1aH-0004Cf-Nw for guix-devel@gnu.org; Thu, 18 Jul 2019 04:16:14 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1ho1aG-00086s-F7 for guix-devel@gnu.org; Thu, 18 Jul 2019 04:16:13 -0400 Received: from sender4-of-o53.zoho.com ([136.143.188.53]:21350) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1ho1aF-00085k-Sz for guix-devel@gnu.org; Thu, 18 Jul 2019 04:16:12 -0400 ARC-Seal: i=1; a=rsa-sha256; t=1563436859; cv=none; d=zoho.com; s=zohoarc; b=BXL33K5o4UUkZYwKgDSNYU4rFspUQH9gz4feN08LTTCCOIc1zv25rK6ffZRlcsD32AdgxP1gGnLIDH7OXJfgt2fCaK6RBpxsXclCU6Nsr4NT+ZdMyEAc8pB/ySO8XJazKBOc5mKHx6jJ8em3NvEFQjVriVxZEO6Y0uQaKs5vOYE= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com; s=zohoarc; t=1563436859; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:MIME-Version:Message-ID:References:Subject:To:ARC-Authentication-Results; bh=8ZNVzdPgaBZ6pEylpBe94cspT279ETCoC542PlqP838=; b=awQaIYRtCCyXofeLcGx0rHBKTDYNcF7ENRC4B0VlOnB12JKFYlkTmFyXffkJI5z846jYIFPdD3uqf0A+sGbcvvLTfFzNPZ799vsSpC7+7Nm6wrPyBLGvfWtBE72m1ZVER63QCU3VbTvo3y6nMHY2GUcejSAlPvqldRNEDg9o4c0= ARC-Authentication-Results: i=1; mx.zoho.com; dkim=pass header.i=elephly.net; spf=pass smtp.mailfrom=rekado@elephly.net; dmarc=pass header.from= header.from= DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; t=1563436859; s=zoho; d=elephly.net; i=rekado@elephly.net; h=References:From:To:Cc:Subject:In-reply-to:Date:Message-ID:MIME-Version:Content-Type:Content-Transfer-Encoding; l=1605; bh=8ZNVzdPgaBZ6pEylpBe94cspT279ETCoC542PlqP838=; b=OQT9yvKkG4mjFfUVVV3omIhLl1KA7UmsiwMaBs6bkppGRZM6/hGgoWCt1phrPTYh aDbJWgtQM5bqYenCSvEdpe6IH0JxS9GvTMn/lOLOqNP+fC+3lQA/Cu+9OlwYFQidPLs HfiRNe607I9RFhu5ApoZtc7o8lG7RQwfqHz9wo80= Received: from localhost (p54AD4F1E.dip0.t-ipconnect.de [84.173.79.30]) by mx.zohomail.com with SMTPS id 1563436858307927.8126500387494; Thu, 18 Jul 2019 01:00:58 -0700 (PDT) References: User-agent: mu4e 1.2.0; emacs 26.2 From: Ricardo Wurmus To: Jack Hill Subject: Re: Character display problem in mumi and my mail client In-reply-to: X-URL: https://elephly.net X-PGP-Key: https://elephly.net/rekado.pubkey X-PGP-Fingerprint: BCA6 89B6 3655 3801 C3C6 2150 197A 5888 235F ACAC Date: Thu, 18 Jul 2019 10:00:55 +0200 Message-ID: <87zhlbg4aw.fsf@elephly.net> X-ZohoMailClient: External X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 136.143.188.53 X-BeenThere: guix-devel@gnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "Development of GNU Guix and the GNU System distribution." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: guix-devel@gnu.org Errors-To: guix-devel-bounces+arunisaac=systemreboot.net@gnu.org Sender: "Guix-devel" MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Hi Jack, thanks for the report. > It appears that mumi (or at least the instance of it running on > issues.guix.gnu.org) has problems displaying some non-ASCII > characters. I noticed it with '=E2=80=99'. Compare > > https://issues.guix.gnu.org/issue/36207 > > with > > https://debbugs.gnu.org/cgi/bugreport.cgi?bug=3D36207 The problem here is that the Debbugs SOAP service (which we use in guile-debbugs and thus mumi) serves up a base64-encoded blob for the email body without any information about encodings, so we just try UTF-8 and fall back to ISO 8859-1 if there=E2=80=99s an error. If we could opera= te on the actual email that would be different. Unfortunately, the SOAP service does not provide access to the actual emails. That=E2=80=99s only available through =E2=80=A6 the official Debbu= gs web interface. I=E2=80=99m currently in the process of moving away from the SOAP service f= or fetching message contents, because it=E2=80=99s just too painful. There ar= e too many truncated or otherwise mangled messages, and in the end we are just assembling them back to a good old email to parse them with Arun=E2=80=99s guile-email. Going forward mumi will only use the SOAP service to get bug status information and pointers to messages. The actual emails will be fetched through the Debbugs web interface with much cursing. > P.S. How are we tracking issues and patches for mumi? bug-guix@ and > guix-patches? Either of them would be fine. Please add [mumi] to the subject line so that it=E2=80=99s easier to distinguish them. =2D- Ricardo --=-=-=-- --=====-=-= Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQEzBAEBCAAdFiEEf3MDQ/Lwnzx3v3nTLiXui2GAK7MFAl0wRyIACgkQLiXui2GA K7OKPgf/S4C40d35Y839D/CpS7Ryp8+oSeCE6b7GJGdxiIV8OZ/335aSMPKWHfYG KG03TEz7PJgMv6Yxq4wzLUbFGr7WHNrk2LacSBIJWZh4I4LL4xVCb3rdORYyt3oi 0GfkSzaHrp0MVyBM3HtcHa0qGyOvFq+YUik2LQxTV0e/K4PjnObmJH1VRV5tQhjk ikxmHXqtdDzP3N8EA+XWcJE8MAlDyyUsVAsVvjRZLqsV92DXBTtG0yV2GNU4N7Kw uqAW0fMvUJbqd918bGHkpVfa3u6FYmqu2cQSnJPkqzWNzNemlxqvNYyIBeTZ6une raPP0EuPyaitgG726PunnCGjfWUBTQ== =TVP8 -----END PGP SIGNATURE----- --=====-=-=--