guile-email discussion
 help / color / Atom feed
* [bug] Character display problem in mumi and my mail client
       [not found] ` <87zhlbg4aw.fsf@elephly.net>
@ 2019-07-18 10:16   ` Arun Isaac
  0 siblings, 0 replies; 8+ messages in thread
From: Arun Isaac @ 2019-07-18 10:16 UTC (permalink / raw)
  To: guile-email

[-- Attachment #1.1: Type: message/rfc822, Size: 4765 bytes --]

[-- Attachment #1.2: Type: message/rfc822, Size: 6190 bytes --]

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 487 bytes --]

^ permalink raw reply	[flat|nested] 8+ messages in thread

* Re: [guile-email] Character display problem in mumi and my mail client
       [not found] <alpine.DEB.2.20.1907171636590.9756@marsh.hcoop.net>
       [not found] ` <87zhlbg4aw.fsf@elephly.net>
@ 2019-07-18 10:23 ` Arun Isaac
  2019-07-18 10:23   ` Arun Isaac
  2019-07-18 10:23   ` Arun Isaac
  2019-07-28  7:33 ` [guile-email] " Arun Isaac
  2 siblings, 2 replies; 8+ messages in thread
From: Arun Isaac @ 2019-07-18 10:23 UTC (permalink / raw)
  To: Jack Hill, guix-devel; +Cc: Ricardo Wurmus, guile-email

[-- Attachment #1.1: Type: text/plain, Size: 1746 bytes --]


Hi,

Thanks for the bug report!

> It appears that mumi (or at least the instance of it running on 
> issues.guix.gnu.org) has problems displaying some non-ASCII characters. I 
> noticed it with '’'. Compare
>
> https://issues.guix.gnu.org/issue/36207
>
> with
>
> https://debbugs.gnu.org/cgi/bugreport.cgi?bug=36207
>
> In the former, Ludo's name displays as 'Ludo¢' while it the latter, it 
> displays correctly as 'Ludo’'.
>
> However, in Ludo's reply the character is displayed correctly.
>
> Does this indicate that my mail client (alpine on Debian Stretch) is doing 
> the wrong thing? It seems that mumi could handle this situation better 
> since it debbugs appears to handle it correctly.
>
> Looking at the raw mail downloaded from debbugs, I see that I'm sending 
> mail as with the following encoding:
>
> ```
> Content-Type: text/plain; FORMAT=flowed; CHARSET=ISO-8859-7
> Content-Transfer-Encoding: 8BIT
> ```
>
> while Ludo's mail is:
>
> ```
> Content-Type: text/plain; charset=utf-8
> Content-Transfer-Encoding: quoted-printable
> ```
>
> Interestingly, when I download the mbox file for my mail from debbugs, and 
> looking at it Emacs with my en_us.UTF-8 locale, the cents-symbol appears. 
> Using iconv to convert the file from ISO-8859-7 to UTF8, causes the 
> correct character to display. So what looks to be happening is that mumi 
> is interpreting my messages using the wrong encoding.
>
> Thoughts? Is this something we want to fix?

I think this is a bug in guile-email. On brief examination, I found that
guile-email is assuming charset of UTF-8 when the
Content-Transfer-Encoding is 8bit. This is incorrect behaviour. I will
fix this soon.

Regards,
Arun.

[-- Attachment #1.2: signature.asc --]
[-- Type: application/pgp-signature, Size: 487 bytes --]

[-- Attachment #2: Type: text/plain, Size: 110 bytes --]

-- 
guile-email mailing list
guile-email@systemreboot.net
https://lists.systemreboot.net/listinfo/guile-email

^ permalink raw reply	[flat|nested] 8+ messages in thread

* Re: Character display problem in mumi and my mail client
  2019-07-18 10:23 ` [guile-email] " Arun Isaac
@ 2019-07-18 10:23   ` Arun Isaac
  2019-07-18 10:23   ` Arun Isaac
  1 sibling, 0 replies; 8+ messages in thread
From: Arun Isaac @ 2019-07-18 10:23 UTC (permalink / raw)
  To: Jack Hill, guix-devel; +Cc: guile-email, Ricardo Wurmus


[-- Attachment #1: Type: text/plain, Size: 1746 bytes --]


Hi,

Thanks for the bug report!

> It appears that mumi (or at least the instance of it running on 
> issues.guix.gnu.org) has problems displaying some non-ASCII characters. I 
> noticed it with '’'. Compare
>
> https://issues.guix.gnu.org/issue/36207
>
> with
>
> https://debbugs.gnu.org/cgi/bugreport.cgi?bug=36207
>
> In the former, Ludo's name displays as 'Ludo¢' while it the latter, it 
> displays correctly as 'Ludo’'.
>
> However, in Ludo's reply the character is displayed correctly.
>
> Does this indicate that my mail client (alpine on Debian Stretch) is doing 
> the wrong thing? It seems that mumi could handle this situation better 
> since it debbugs appears to handle it correctly.
>
> Looking at the raw mail downloaded from debbugs, I see that I'm sending 
> mail as with the following encoding:
>
> ```
> Content-Type: text/plain; FORMAT=flowed; CHARSET=ISO-8859-7
> Content-Transfer-Encoding: 8BIT
> ```
>
> while Ludo's mail is:
>
> ```
> Content-Type: text/plain; charset=utf-8
> Content-Transfer-Encoding: quoted-printable
> ```
>
> Interestingly, when I download the mbox file for my mail from debbugs, and 
> looking at it Emacs with my en_us.UTF-8 locale, the cents-symbol appears. 
> Using iconv to convert the file from ISO-8859-7 to UTF8, causes the 
> correct character to display. So what looks to be happening is that mumi 
> is interpreting my messages using the wrong encoding.
>
> Thoughts? Is this something we want to fix?

I think this is a bug in guile-email. On brief examination, I found that
guile-email is assuming charset of UTF-8 when the
Content-Transfer-Encoding is 8bit. This is incorrect behaviour. I will
fix this soon.

Regards,
Arun.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 487 bytes --]

^ permalink raw reply	[flat|nested] 8+ messages in thread

* Re: Character display problem in mumi and my mail client
  2019-07-18 10:23 ` [guile-email] " Arun Isaac
  2019-07-18 10:23   ` Arun Isaac
@ 2019-07-18 10:23   ` Arun Isaac
  1 sibling, 0 replies; 8+ messages in thread
From: Arun Isaac @ 2019-07-18 10:23 UTC (permalink / raw)
  To: Jack Hill, guix-devel; +Cc: guile-email


[-- Attachment #1: Type: text/plain, Size: 1746 bytes --]


Hi,

Thanks for the bug report!

> It appears that mumi (or at least the instance of it running on 
> issues.guix.gnu.org) has problems displaying some non-ASCII characters. I 
> noticed it with '’'. Compare
>
> https://issues.guix.gnu.org/issue/36207
>
> with
>
> https://debbugs.gnu.org/cgi/bugreport.cgi?bug=36207
>
> In the former, Ludo's name displays as 'Ludo¢' while it the latter, it 
> displays correctly as 'Ludo’'.
>
> However, in Ludo's reply the character is displayed correctly.
>
> Does this indicate that my mail client (alpine on Debian Stretch) is doing 
> the wrong thing? It seems that mumi could handle this situation better 
> since it debbugs appears to handle it correctly.
>
> Looking at the raw mail downloaded from debbugs, I see that I'm sending 
> mail as with the following encoding:
>
> ```
> Content-Type: text/plain; FORMAT=flowed; CHARSET=ISO-8859-7
> Content-Transfer-Encoding: 8BIT
> ```
>
> while Ludo's mail is:
>
> ```
> Content-Type: text/plain; charset=utf-8
> Content-Transfer-Encoding: quoted-printable
> ```
>
> Interestingly, when I download the mbox file for my mail from debbugs, and 
> looking at it Emacs with my en_us.UTF-8 locale, the cents-symbol appears. 
> Using iconv to convert the file from ISO-8859-7 to UTF8, causes the 
> correct character to display. So what looks to be happening is that mumi 
> is interpreting my messages using the wrong encoding.
>
> Thoughts? Is this something we want to fix?

I think this is a bug in guile-email. On brief examination, I found that
guile-email is assuming charset of UTF-8 when the
Content-Transfer-Encoding is 8bit. This is incorrect behaviour. I will
fix this soon.

Regards,
Arun.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 487 bytes --]

^ permalink raw reply	[flat|nested] 8+ messages in thread

* Re: [guile-email] Character display problem in mumi and my mail client
       [not found] <alpine.DEB.2.20.1907171636590.9756@marsh.hcoop.net>
       [not found] ` <87zhlbg4aw.fsf@elephly.net>
  2019-07-18 10:23 ` [guile-email] " Arun Isaac
@ 2019-07-28  7:33 ` Arun Isaac
  2019-07-28  7:33   ` Arun Isaac
                     ` (2 more replies)
  2 siblings, 3 replies; 8+ messages in thread
From: Arun Isaac @ 2019-07-28  7:33 UTC (permalink / raw)
  To: Jack Hill, Ricardo Wurmus; +Cc: guix-devel, guile-email

[-- Attachment #1.1: Type: text/plain, Size: 1631 bytes --]


> It appears that mumi (or at least the instance of it running on 
> issues.guix.gnu.org) has problems displaying some non-ASCII characters. I 
> noticed it with '’'. Compare
>
> https://issues.guix.gnu.org/issue/36207
>
> with
>
> https://debbugs.gnu.org/cgi/bugreport.cgi?bug=36207

I have fixed this bug. See
https://git.systemreboot.net/guile-email/commit/?id=ac83c2a00c13702bc365cd0f3074239fa63d743f
and
https://git.systemreboot.net/guile-email/commit/?id=1f7c45fa0b71bd137e4b661e0d473c3eb9c98f48

guile-email's parse-email and parse-email-body functions now prefer to
operate on bytevectors, rather than on strings. Likewise, mbox->emails
now returns a list of bytevectors, not a list of strings.

I have updated the API documentation at
https://guile-email.systemreboot.net/manual/Parsing-e_002dmail.html#Parsing-e_002dmail

@Ricardo:

I think you are using a git checkout of guile-email for the mumi hosted
at issues.guix.info. Could you use the latest guile-email commit on
master (specifically, c85e6917ea21631857d93f58e60d910e07317131)? That
should fix this bug. No other changes are required in mumi. I will
release guile-email 0.2.0 in another week's time.

> This reminds me of rjbs's talk, "Email Hates the Living". [0]
>
> [0] http://yapcasia.org/2011/talk/59

Indeed, email drags along a lot of backward compatibility baggage from
the past. In the future, I'll add some of the pathological examples
mentioned in this talk as test cases.

Also, if you know of any corpus of email parser test cases, please let
me know. I will use them to expand guile-email's test suite.

[-- Attachment #1.2: signature.asc --]
[-- Type: application/pgp-signature, Size: 487 bytes --]

[-- Attachment #2: Type: text/plain, Size: 110 bytes --]

-- 
guile-email mailing list
guile-email@systemreboot.net
https://lists.systemreboot.net/listinfo/guile-email

^ permalink raw reply	[flat|nested] 8+ messages in thread

* Re: Character display problem in mumi and my mail client
  2019-07-28  7:33 ` [guile-email] " Arun Isaac
  2019-07-28  7:33   ` Arun Isaac
@ 2019-07-28  7:33   ` Arun Isaac
  2019-07-28  8:36   ` Ricardo Wurmus
  2 siblings, 0 replies; 8+ messages in thread
From: Arun Isaac @ 2019-07-28  7:33 UTC (permalink / raw)
  To: Jack Hill, Ricardo Wurmus; +Cc: guile-email, guix-devel


[-- Attachment #1: Type: text/plain, Size: 1631 bytes --]


> It appears that mumi (or at least the instance of it running on 
> issues.guix.gnu.org) has problems displaying some non-ASCII characters. I 
> noticed it with '’'. Compare
>
> https://issues.guix.gnu.org/issue/36207
>
> with
>
> https://debbugs.gnu.org/cgi/bugreport.cgi?bug=36207

I have fixed this bug. See
https://git.systemreboot.net/guile-email/commit/?id=ac83c2a00c13702bc365cd0f3074239fa63d743f
and
https://git.systemreboot.net/guile-email/commit/?id=1f7c45fa0b71bd137e4b661e0d473c3eb9c98f48

guile-email's parse-email and parse-email-body functions now prefer to
operate on bytevectors, rather than on strings. Likewise, mbox->emails
now returns a list of bytevectors, not a list of strings.

I have updated the API documentation at
https://guile-email.systemreboot.net/manual/Parsing-e_002dmail.html#Parsing-e_002dmail

@Ricardo:

I think you are using a git checkout of guile-email for the mumi hosted
at issues.guix.info. Could you use the latest guile-email commit on
master (specifically, c85e6917ea21631857d93f58e60d910e07317131)? That
should fix this bug. No other changes are required in mumi. I will
release guile-email 0.2.0 in another week's time.

> This reminds me of rjbs's talk, "Email Hates the Living". [0]
>
> [0] http://yapcasia.org/2011/talk/59

Indeed, email drags along a lot of backward compatibility baggage from
the past. In the future, I'll add some of the pathological examples
mentioned in this talk as test cases.

Also, if you know of any corpus of email parser test cases, please let
me know. I will use them to expand guile-email's test suite.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 487 bytes --]

^ permalink raw reply	[flat|nested] 8+ messages in thread

* Re: Character display problem in mumi and my mail client
  2019-07-28  7:33 ` [guile-email] " Arun Isaac
@ 2019-07-28  7:33   ` Arun Isaac
  2019-07-28  7:33   ` Arun Isaac
  2019-07-28  8:36   ` Ricardo Wurmus
  2 siblings, 0 replies; 8+ messages in thread
From: Arun Isaac @ 2019-07-28  7:33 UTC (permalink / raw)
  To: Jack Hill, Ricardo Wurmus; +Cc: guix-devel, guile-email


[-- Attachment #1: Type: text/plain, Size: 1631 bytes --]


> It appears that mumi (or at least the instance of it running on 
> issues.guix.gnu.org) has problems displaying some non-ASCII characters. I 
> noticed it with '’'. Compare
>
> https://issues.guix.gnu.org/issue/36207
>
> with
>
> https://debbugs.gnu.org/cgi/bugreport.cgi?bug=36207

I have fixed this bug. See
https://git.systemreboot.net/guile-email/commit/?id=ac83c2a00c13702bc365cd0f3074239fa63d743f
and
https://git.systemreboot.net/guile-email/commit/?id=1f7c45fa0b71bd137e4b661e0d473c3eb9c98f48

guile-email's parse-email and parse-email-body functions now prefer to
operate on bytevectors, rather than on strings. Likewise, mbox->emails
now returns a list of bytevectors, not a list of strings.

I have updated the API documentation at
https://guile-email.systemreboot.net/manual/Parsing-e_002dmail.html#Parsing-e_002dmail

@Ricardo:

I think you are using a git checkout of guile-email for the mumi hosted
at issues.guix.info. Could you use the latest guile-email commit on
master (specifically, c85e6917ea21631857d93f58e60d910e07317131)? That
should fix this bug. No other changes are required in mumi. I will
release guile-email 0.2.0 in another week's time.

> This reminds me of rjbs's talk, "Email Hates the Living". [0]
>
> [0] http://yapcasia.org/2011/talk/59

Indeed, email drags along a lot of backward compatibility baggage from
the past. In the future, I'll add some of the pathological examples
mentioned in this talk as test cases.

Also, if you know of any corpus of email parser test cases, please let
me know. I will use them to expand guile-email's test suite.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 487 bytes --]

^ permalink raw reply	[flat|nested] 8+ messages in thread

* Re: Character display problem in mumi and my mail client
  2019-07-28  7:33 ` [guile-email] " Arun Isaac
  2019-07-28  7:33   ` Arun Isaac
  2019-07-28  7:33   ` Arun Isaac
@ 2019-07-28  8:36   ` Ricardo Wurmus
  2 siblings, 0 replies; 8+ messages in thread
From: Ricardo Wurmus @ 2019-07-28  8:36 UTC (permalink / raw)
  To: Arun Isaac; +Cc: Jack Hill, guile-email, guix-devel


Arun Isaac <arunisaac@systemreboot.net> writes:

> I have fixed this bug. See
> https://git.systemreboot.net/guile-email/commit/?id=ac83c2a00c13702bc365cd0f3074239fa63d743f
> and
> https://git.systemreboot.net/guile-email/commit/?id=1f7c45fa0b71bd137e4b661e0d473c3eb9c98f48
[…]
> @Ricardo:
>
> I think you are using a git checkout of guile-email for the mumi hosted
> at issues.guix.info. Could you use the latest guile-email commit on
> master (specifically, c85e6917ea21631857d93f58e60d910e07317131)? That
> should fix this bug. No other changes are required in mumi. I will
> release guile-email 0.2.0 in another week's time.

Thank you.  I’ve updated the variant of guile-email that’s used by mumi.

-- 
Ricardo


^ permalink raw reply	[flat|nested] 8+ messages in thread

end of thread, back to index

Thread overview: 8+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
     [not found] <alpine.DEB.2.20.1907171636590.9756@marsh.hcoop.net>
     [not found] ` <87zhlbg4aw.fsf@elephly.net>
2019-07-18 10:16   ` [bug] Character display problem in mumi and my mail client Arun Isaac
2019-07-18 10:23 ` [guile-email] " Arun Isaac
2019-07-18 10:23   ` Arun Isaac
2019-07-18 10:23   ` Arun Isaac
2019-07-28  7:33 ` [guile-email] " Arun Isaac
2019-07-28  7:33   ` Arun Isaac
2019-07-28  7:33   ` Arun Isaac
2019-07-28  8:36   ` Ricardo Wurmus

guile-email discussion

Archives are clonable:
	git clone --mirror http://lists.systemreboot.net/guile-email/0 guile-email/git/0.git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V2 guile-email guile-email/ http://lists.systemreboot.net/guile-email \
		guile-email@systemreboot.net
	public-inbox-index guile-email

Example config snippet for mirrors


AGPL code for this site: git clone https://public-inbox.org/public-inbox.git