guile-email discussion
 help / color / mirror / Atom feed
* [guile-email] decoding-error crash
@ 2019-09-15 12:57 Christopher Baines
  2019-09-16  9:50 ` Arun Isaac
  0 siblings, 1 reply; 17+ messages in thread
From: Christopher Baines @ 2019-09-15 12:57 UTC (permalink / raw)
  To: guile-email


[-- Attachment #1.1.1: Type: text/plain, Size: 1311 bytes --]

Hey,

This issue comes from trying to using guile-email to read emails to the
guix-commits mailing list from the Guix Data Service.

I'm seeing crashes like the following:


Backtrace:
          14 (apply-smob/1 #<catch-closure e63940>)
In ice-9/boot-9.scm:
    705:2 13 (call-with-prompt _ _ #<procedure default-prompt-handler (k proc)>)
In ice-9/eval.scm:
    619:8 12 (_ #(#(#<directory (guile-user) ee4140>)))
In ice-9/boot-9.scm:
   2312:4 11 (save-module-excursion _)
  3832:12 10 (_)
In /home/chris/decoding-error.scm:
     98:1  9 (_)
In email/email.scm:
   817:34  8 (parse-email _)
In rnrs/io/ports.scm:
    334:0  7 (call-with-port #<input: string 106d930> _)
In email/email.scm:
   609:21  6 (_ #<input: string 106d930>)
In email/utils.scm:
    60:13  5 (read-while _ _ _)
In ice-9/ports.scm:
    550:4  4 (call-with-output-string _)
In email/utils.scm:
    52:13  3 (read-while-loop #<output: string 106d8c0>)
    91:14  2 (get-line-with-delimiter _)
In ice-9/rdelim.scm:
   195:24  1 (read-line _ _)
In unknown file:
           0 (%read-line #<input: string 106d930>)

ERROR: In procedure %read-line:
Throw to key `decoding-error' with args `("decode-char" "input decoding error" 84 #<input: string 106d930>)'.


I've attached a file which should reproduce the issue.

Any ideas?

Thanks,

Chris


[-- Attachment #1.1.2: decoding-error.scm --]
[-- Type: text/plain, Size: 4096 bytes --]

(use-modules (rnrs bytevectors)
             (email email))

(define mail
  "Return-Path: <guix-commits-bounces+mail=cbaines.net@gnu.org>
X-Original-To: cbaines@mira.cbaines.net
Delivered-To: cbaines@mira.cbaines.net
Received: by mira.cbaines.net (Postfix, from userid 113)
	id 5ADAB17307; Thu, 22 Aug 2019 16:21:39 +0100 (BST)
X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on mira.cbaines.net
X-Spam-Level: 
X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00,URIBL_BLOCKED
	autolearn=unavailable autolearn_force=no version=3.4.0
Received: from lists.gnu.org (lists.gnu.org [209.51.188.17])
	by mira.cbaines.net (Postfix) with ESMTP id 26E3A17305
	for <mail@cbaines.net>; Thu, 22 Aug 2019 16:21:39 +0100 (BST)
Received: from localhost ([::1]:44422 helo=lists1p.gnu.org)
	by lists.gnu.org with esmtp (Exim 4.90_1)
	(envelope-from <guix-commits-bounces+mail=cbaines.net@gnu.org>)
	id 1i0ouA-0001ex-KU
	for mail@cbaines.net; Thu, 22 Aug 2019 11:21:38 -0400
Received: from eggs.gnu.org ([2001:470:142:3::10]:46437)
 by lists.gnu.org with esmtp (Exim 4.90_1)
 (envelope-from <ludo@gnu.org>) id 1i0oso-0008KE-FG
 for guix-commits@gnu.org; Thu, 22 Aug 2019 11:20:15 -0400
Received: from vcs0.savannah.gnu.org ([209.51.188.201]:34251)
 by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from <ludo@gnu.org>)
 id 1i0oso-0003FN-2h
 for guix-commits@gnu.org; Thu, 22 Aug 2019 11:20:14 -0400
Received: by vcs0.savannah.gnu.org (Postfix, from userid 68006)
 id F323F21324; Thu, 22 Aug 2019 11:20:11 -0400 (EDT)
To: guix-commits@gnu.org
Subject: 02/03: website: Don't set 'GUIX_WEB_SITE_INFO'.
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Mail-Followup-To: guix-devel@savannah.gnu.org
In-Reply-To: <20190822152011.3696.62577@vcs0.savannah.gnu.org>
References: <20190822152011.3696.62577@vcs0.savannah.gnu.org>
X-Git-Repo: guix-artwork
X-Git-Refname: refs/heads/master
X-Git-Reftype: branch
X-Git-Rev: f9a9795a37cd292fc9aacb25af2aff2cdb43d483
Auto-Submitted: auto-generated
Message-Id: <20190822152011.F323F21324@vcs0.savannah.gnu.org>
Date: Thu, 22 Aug 2019 11:20:11 -0400 (EDT)
From: ludo@gnu.org (Ludovic Courtès)
Content-Transfer-Encoding: quoted-printable
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic]
X-BeenThere: guix-commits@gnu.org
X-Mailman-Version: 2.1.23
Precedence: list
List-Id: <guix-commits.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/guix-commits>,
 <mailto:guix-commits-request@gnu.org?subject=unsubscribe>
List-Archive: <https://lists.gnu.org/archive/html/guix-commits>
List-Post: <mailto:guix-commits@gnu.org>
List-Help: <mailto:guix-commits-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/guix-commits>,
 <mailto:guix-commits-request@gnu.org?subject=subscribe>
Reply-To: =?utf-8?q?Ludovic_Court=C3=A8s?= <ludo@gnu.org>
Errors-To: guix-commits-bounces+mail=cbaines.net@gnu.org
Sender: \"Guix-commits\" <guix-commits-bounces+mail=cbaines.net@gnu.org>
X-TUID: D4q5j3vAumD9

civodul pushed a commit to branch master
in repository guix-artwork.

commit f9a9795a37cd292fc9aacb25af2aff2cdb43d483
Author: Ludovic Court=C3=A8s <ludo@gnu.org>
Date:   Thu Aug 22 16:41:14 2019 +0200

    website: Don't set 'GUIX_WEB_SITE_INFO'.
   =20
    This is a followup to feb02faba58dd7f07bc5e8912c564b5ff0303920.
   =20
    * website/.guix.scm (build): Don't set \"GUIX_WEB_SITE_INFO\".
---
 website/.guix.scm | 3 ---
 1 file changed, 3 deletions(-)

diff --git a/website/.guix.scm b/website/.guix.scm
index 3aa4425..173f194 100644
--- a/website/.guix.scm
+++ b/website/.guix.scm
@@ -62,9 +62,6 @@
           ;; Use a sane default.
           (setenv \"XDG_CACHE_HOME\" \"/tmp/.cache\")
=20
-          ;; Choose the layout for guix.gnu.org.
-          (setenv \"GUIX_WEB_SITE_INFO\" \"t\")
-
           (invoke #+(file-append (specification->package \"haunt\")
                                  \"/bin/haunt\")
                   \"build\")
")

(display
 (parse-email
  (string->utf8 mail)))

[-- Attachment #1.2: signature.asc --]
[-- Type: application/pgp-signature, Size: 962 bytes --]

[-- Attachment #2: Type: text/plain, Size: 110 bytes --]

-- 
guile-email mailing list
guile-email@systemreboot.net
https://lists.systemreboot.net/listinfo/guile-email

^ permalink raw reply	[flat|nested] 17+ messages in thread

* Re: [guile-email] decoding-error crash
  2019-09-15 12:57 [guile-email] decoding-error crash Christopher Baines
@ 2019-09-16  9:50 ` Arun Isaac
  2019-09-16  9:50   ` Arun Isaac
  2019-09-16 16:32   ` Arun Isaac
  0 siblings, 2 replies; 17+ messages in thread
From: Arun Isaac @ 2019-09-16  9:50 UTC (permalink / raw)
  To: Christopher Baines; +Cc: guile-email

[-- Attachment #1: Type: text/plain, Size: 995 bytes --]


> I've attached a file which should reproduce the issue.

> From: ludo@gnu.org (Ludovic Courtès)

There is an encoding error in the email, specifically in the From header
above. Notice the è in Courtès. It should have been encoded as a
"MIME-encoded word" using Quoted-Printable encoding or some other
scheme. See https://en.wikipedia.org/wiki/MIME#Encoded-Word . Only ASCII
characters are allowed in email headers. utf-8 characters are not
allowed. guile-email sees this character whose encoding it does not
understand and crashes with a decoding error. I should probably make it
raise a more meaningful error.

Could you provide more context for your code snippet? How does your
program come by this email string? Ideally, you should handle emails
only as bytevectors (or binary ports), not as strings. An email can have
characters encoded using different schemes, whereas a string is
constrained to have all characters encoded using the same scheme.

Regards,
Arun.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 487 bytes --]

^ permalink raw reply	[flat|nested] 17+ messages in thread

* Re: [guile-email] decoding-error crash
  2019-09-16  9:50 ` Arun Isaac
@ 2019-09-16  9:50   ` Arun Isaac
  2019-09-16 16:32   ` Arun Isaac
  1 sibling, 0 replies; 17+ messages in thread
From: Arun Isaac @ 2019-09-16  9:50 UTC (permalink / raw)
  To: Christopher Baines; +Cc: guile-email


[-- Attachment #1.1: Type: text/plain, Size: 995 bytes --]


> I've attached a file which should reproduce the issue.

> From: ludo@gnu.org (Ludovic Courtès)

There is an encoding error in the email, specifically in the From header
above. Notice the è in Courtès. It should have been encoded as a
"MIME-encoded word" using Quoted-Printable encoding or some other
scheme. See https://en.wikipedia.org/wiki/MIME#Encoded-Word . Only ASCII
characters are allowed in email headers. utf-8 characters are not
allowed. guile-email sees this character whose encoding it does not
understand and crashes with a decoding error. I should probably make it
raise a more meaningful error.

Could you provide more context for your code snippet? How does your
program come by this email string? Ideally, you should handle emails
only as bytevectors (or binary ports), not as strings. An email can have
characters encoded using different schemes, whereas a string is
constrained to have all characters encoded using the same scheme.

Regards,
Arun.

[-- Attachment #1.2: signature.asc --]
[-- Type: application/pgp-signature, Size: 487 bytes --]

[-- Attachment #2: Type: text/plain, Size: 110 bytes --]

-- 
guile-email mailing list
guile-email@systemreboot.net
https://lists.systemreboot.net/listinfo/guile-email

^ permalink raw reply	[flat|nested] 17+ messages in thread

* Re: [guile-email] decoding-error crash
  2019-09-16  9:50 ` Arun Isaac
  2019-09-16  9:50   ` Arun Isaac
@ 2019-09-16 16:32   ` Arun Isaac
  2019-09-16 16:32     ` Arun Isaac
                       ` (2 more replies)
  1 sibling, 3 replies; 17+ messages in thread
From: Arun Isaac @ 2019-09-16 16:32 UTC (permalink / raw)
  To: Christopher Baines; +Cc: guile-email

[-- Attachment #1: Type: text/plain, Size: 435 bytes --]


>> From: ludo@gnu.org (Ludovic Courtès)

On the other hand, I just realized that this is a non-standard Emacs
message mode parens style address described at
https://www.gnu.org/software/emacs/manual/html_node/emacs/Mail-Headers.html
. guile-email aims to support these non-standard addresses. So, I think
we can relax the requirement that email headers be strictly ASCII. I'll
push a fix and let you know.

Thanks,
Arun.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 487 bytes --]

^ permalink raw reply	[flat|nested] 17+ messages in thread

* Re: [guile-email] decoding-error crash
  2019-09-16 16:32   ` Arun Isaac
@ 2019-09-16 16:32     ` Arun Isaac
  2019-09-16 20:20     ` Arun Isaac
  2019-09-16 20:55     ` Christopher Baines
  2 siblings, 0 replies; 17+ messages in thread
From: Arun Isaac @ 2019-09-16 16:32 UTC (permalink / raw)
  To: Christopher Baines; +Cc: guile-email


[-- Attachment #1.1: Type: text/plain, Size: 435 bytes --]


>> From: ludo@gnu.org (Ludovic Courtès)

On the other hand, I just realized that this is a non-standard Emacs
message mode parens style address described at
https://www.gnu.org/software/emacs/manual/html_node/emacs/Mail-Headers.html
. guile-email aims to support these non-standard addresses. So, I think
we can relax the requirement that email headers be strictly ASCII. I'll
push a fix and let you know.

Thanks,
Arun.

[-- Attachment #1.2: signature.asc --]
[-- Type: application/pgp-signature, Size: 487 bytes --]

[-- Attachment #2: Type: text/plain, Size: 110 bytes --]

-- 
guile-email mailing list
guile-email@systemreboot.net
https://lists.systemreboot.net/listinfo/guile-email

^ permalink raw reply	[flat|nested] 17+ messages in thread

* Re: [guile-email] decoding-error crash
  2019-09-16 16:32   ` Arun Isaac
  2019-09-16 16:32     ` Arun Isaac
@ 2019-09-16 20:20     ` Arun Isaac
  2019-09-16 20:20       ` Arun Isaac
  2019-09-22 13:19       ` Christopher Baines
  2019-09-16 20:55     ` Christopher Baines
  2 siblings, 2 replies; 17+ messages in thread
From: Arun Isaac @ 2019-09-16 20:20 UTC (permalink / raw)
  To: Christopher Baines; +Cc: guile-email


[-- Attachment #1.1: Type: text/plain, Size: 811 bytes --]


>>> From: ludo@gnu.org (Ludovic Courtès)
>
> On the other hand, I just realized that this is a non-standard Emacs
> message mode parens style address described at
> https://www.gnu.org/software/emacs/manual/html_node/emacs/Mail-Headers.html
> . guile-email aims to support these non-standard addresses. So, I think
> we can relax the requirement that email headers be strictly ASCII. I'll
> push a fix and let you know.

I've pushed a fix along with a test to prevent regression. See
https://git.systemreboot.net/guile-email/commit/?id=203c9c2ae39a4c64bfd9199bbc2deae03c347998

Could you use a git checkout of the master branch for your guile-email
deployment? Or would you rather I get a minor bugfix release out soon?

Thanks for reporting this bug and thanks for choosing guile-email! :-)

[-- Attachment #1.2: signature.asc --]
[-- Type: application/pgp-signature, Size: 487 bytes --]

[-- Attachment #2: Type: text/plain, Size: 110 bytes --]

-- 
guile-email mailing list
guile-email@systemreboot.net
https://lists.systemreboot.net/listinfo/guile-email

^ permalink raw reply	[flat|nested] 17+ messages in thread

* Re: [guile-email] decoding-error crash
  2019-09-16 20:20     ` Arun Isaac
@ 2019-09-16 20:20       ` Arun Isaac
  2019-09-22 13:19       ` Christopher Baines
  1 sibling, 0 replies; 17+ messages in thread
From: Arun Isaac @ 2019-09-16 20:20 UTC (permalink / raw)
  To: Christopher Baines; +Cc: guile-email

[-- Attachment #1: Type: text/plain, Size: 811 bytes --]


>>> From: ludo@gnu.org (Ludovic Courtès)
>
> On the other hand, I just realized that this is a non-standard Emacs
> message mode parens style address described at
> https://www.gnu.org/software/emacs/manual/html_node/emacs/Mail-Headers.html
> . guile-email aims to support these non-standard addresses. So, I think
> we can relax the requirement that email headers be strictly ASCII. I'll
> push a fix and let you know.

I've pushed a fix along with a test to prevent regression. See
https://git.systemreboot.net/guile-email/commit/?id=203c9c2ae39a4c64bfd9199bbc2deae03c347998

Could you use a git checkout of the master branch for your guile-email
deployment? Or would you rather I get a minor bugfix release out soon?

Thanks for reporting this bug and thanks for choosing guile-email! :-)

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 487 bytes --]

^ permalink raw reply	[flat|nested] 17+ messages in thread

* Re: [guile-email] decoding-error crash
  2019-09-16 16:32   ` Arun Isaac
  2019-09-16 16:32     ` Arun Isaac
  2019-09-16 20:20     ` Arun Isaac
@ 2019-09-16 20:55     ` Christopher Baines
  2019-09-17  5:37       ` Arun Isaac
  2 siblings, 1 reply; 17+ messages in thread
From: Christopher Baines @ 2019-09-16 20:55 UTC (permalink / raw)
  To: Arun Isaac; +Cc: guile-email

[-- Attachment #1: Type: text/plain, Size: 1251 bytes --]


Arun Isaac <arunisaac@systemreboot.net> writes:

>>> From: ludo@gnu.org (Ludovic Courtès)
>
> On the other hand, I just realized that this is a non-standard Emacs
> message mode parens style address described at
> https://www.gnu.org/software/emacs/manual/html_node/emacs/Mail-Headers.html
> . guile-email aims to support these non-standard addresses. So, I think
> we can relax the requirement that email headers be strictly ASCII. I'll
> push a fix and let you know.

Great, thanks.

In terms of where this message came from, I'm using getmail [1] to
recieve the messages, then using the external MTA delivery config to
pass the message off to a Guile script. I was processing the messages as
strings, but I'm switching to using bytevectors. I believe this issue
happens with strings and bytevectors.

I've checked my mailserver, and the file corresponding to the sample
email provided doesn't look to have a pure ASCII from header (in the
terminal, I see a ? instead of è, so I'm not sure what's going on
there).

Also, you can find examples (included the provided sample) in this mbox
file [2].

1: http://pyropus.ca/software/getmail/
2: https://lists.gnu.org/archive/mbox/guix-commits/2019-08

Thanks,

Chris

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 962 bytes --]

^ permalink raw reply	[flat|nested] 17+ messages in thread

* Re: [guile-email] decoding-error crash
  2019-09-16 20:55     ` Christopher Baines
@ 2019-09-17  5:37       ` Arun Isaac
  2019-09-17  5:37         ` Arun Isaac
  0 siblings, 1 reply; 17+ messages in thread
From: Arun Isaac @ 2019-09-17  5:37 UTC (permalink / raw)
  To: Christopher Baines; +Cc: guile-email

[-- Attachment #1: Type: text/plain, Size: 1112 bytes --]


>> we can relax the requirement that email headers be strictly ASCII. I'll
>> push a fix and let you know.
>
> Great, thanks.

Let me know if the fix works for you.

> In terms of where this message came from, I'm using getmail [1] to
> recieve the messages, then using the external MTA delivery config to
> pass the message off to a Guile script. I was processing the messages as
> strings, but I'm switching to using bytevectors.

Ok.

> I believe this issue happens with strings and bytevectors.

Yes, you are right. The issue of strings and bytevectors is
orthogonal. An email is a bag of bytes (not a bag of characters) -- a
binary file that needs to be parsed by guile-email.

> I've checked my mailserver, and the file corresponding to the sample
> email provided doesn't look to have a pure ASCII from header (in the
> terminal, I see a ? instead of è, so I'm not sure what's going on
> there).

Not sure which line you are referring to, but in Icecat, as far as I can
tell, I see all the 'è's correctly. Perhaps it's a decoding issue with
whichever viewer you are using.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 487 bytes --]

^ permalink raw reply	[flat|nested] 17+ messages in thread

* Re: [guile-email] decoding-error crash
  2019-09-17  5:37       ` Arun Isaac
@ 2019-09-17  5:37         ` Arun Isaac
  0 siblings, 0 replies; 17+ messages in thread
From: Arun Isaac @ 2019-09-17  5:37 UTC (permalink / raw)
  To: Christopher Baines; +Cc: guile-email


[-- Attachment #1.1: Type: text/plain, Size: 1112 bytes --]


>> we can relax the requirement that email headers be strictly ASCII. I'll
>> push a fix and let you know.
>
> Great, thanks.

Let me know if the fix works for you.

> In terms of where this message came from, I'm using getmail [1] to
> recieve the messages, then using the external MTA delivery config to
> pass the message off to a Guile script. I was processing the messages as
> strings, but I'm switching to using bytevectors.

Ok.

> I believe this issue happens with strings and bytevectors.

Yes, you are right. The issue of strings and bytevectors is
orthogonal. An email is a bag of bytes (not a bag of characters) -- a
binary file that needs to be parsed by guile-email.

> I've checked my mailserver, and the file corresponding to the sample
> email provided doesn't look to have a pure ASCII from header (in the
> terminal, I see a ? instead of è, so I'm not sure what's going on
> there).

Not sure which line you are referring to, but in Icecat, as far as I can
tell, I see all the 'è's correctly. Perhaps it's a decoding issue with
whichever viewer you are using.

[-- Attachment #1.2: signature.asc --]
[-- Type: application/pgp-signature, Size: 487 bytes --]

[-- Attachment #2: Type: text/plain, Size: 110 bytes --]

-- 
guile-email mailing list
guile-email@systemreboot.net
https://lists.systemreboot.net/listinfo/guile-email

^ permalink raw reply	[flat|nested] 17+ messages in thread

* Re: [guile-email] decoding-error crash
  2019-09-16 20:20     ` Arun Isaac
  2019-09-16 20:20       ` Arun Isaac
@ 2019-09-22 13:19       ` Christopher Baines
  2019-09-23  5:21         ` Arun Isaac
  1 sibling, 1 reply; 17+ messages in thread
From: Christopher Baines @ 2019-09-22 13:19 UTC (permalink / raw)
  To: Arun Isaac; +Cc: guile-email

[-- Attachment #1: Type: text/plain, Size: 1960 bytes --]


Arun Isaac <arunisaac@systemreboot.net> writes:

>>>> From: ludo@gnu.org (Ludovic Courtès)
>>
>> On the other hand, I just realized that this is a non-standard Emacs
>> message mode parens style address described at
>> https://www.gnu.org/software/emacs/manual/html_node/emacs/Mail-Headers.html
>> . guile-email aims to support these non-standard addresses. So, I think
>> we can relax the requirement that email headers be strictly ASCII. I'll
>> push a fix and let you know.
>
> I've pushed a fix along with a test to prevent regression. See
> https://git.systemreboot.net/guile-email/commit/?id=203c9c2ae39a4c64bfd9199bbc2deae03c347998
>
> Could you use a git checkout of the master branch for your guile-email
> deployment? Or would you rather I get a minor bugfix release out soon?

Thanks Arun, that definately fixes the issue with the email I sent
initially, although I think there might be other emails on the
guix-commits mailing list that still can't be read.

For testing, I downloaded
https://lists.gnu.org/archive/mbox/guix-commits/2019-08 and then tried
to read it with:

  (map parse-email (call-with-input-file "2019-08" mbox->emails))

And I get a decoding-error backtrace.

In email/email.scm:
   824:34  8 (parse-email _)
In rnrs/io/ports.scm:
    334:0  7 (call-with-port #<input: string 4a80770> _)
In email/email.scm:
   616:21  6 (_ #<input: string 4a80770>)
In email/utils.scm:
    60:13  5 (read-while _ _ _)
In ice-9/ports.scm:
    550:4  4 (call-with-output-string _)
In email/utils.scm:
    52:13  3 (read-while-loop #<output: string 4a80690>)
    91:14  2 (get-line-with-delimiter _)
In ice-9/rdelim.scm:
   195:24  1 (read-line _ _)
In unknown file:
           0 (%read-line #<input: string 4a80770>)

ERROR: In procedure %read-line:
Throw to key `decoding-error' with args `("peek-char" "input decoding error" 84 #<input: string 4a80770>)'.


Any ideas?

Thanks,

Chris

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 962 bytes --]

^ permalink raw reply	[flat|nested] 17+ messages in thread

* Re: [guile-email] decoding-error crash
  2019-09-22 13:19       ` Christopher Baines
@ 2019-09-23  5:21         ` Arun Isaac
  2019-09-23  5:21           ` Arun Isaac
  2019-09-23  7:30           ` Christopher Baines
  0 siblings, 2 replies; 17+ messages in thread
From: Arun Isaac @ 2019-09-23  5:21 UTC (permalink / raw)
  To: Christopher Baines; +Cc: guile-email

[-- Attachment #1: Type: text/plain, Size: 877 bytes --]


> I think there might be other emails on the guix-commits mailing list
> that still can't be read.
>
> For testing, I downloaded
> https://lists.gnu.org/archive/mbox/guix-commits/2019-08 and then tried
> to read it with:
>
>   (map parse-email (call-with-input-file "2019-08" mbox->emails))

These errors occur due to the occurrence of non-standard ISO-8859-1
characters in the headers due to Emacs message mode parens style
addresses. The earlier fix only allows UTF-8 or ASCII characters. I have
pushed a new fix that uses the substitute conversion strategy to
silently fail without an error in the event of something other than
UTF-8 or ASCII encoding.

https://git.systemreboot.net/guile-email/commit/?id=c10524726e0a91445a0329cc9aa72b7467258381

The alternative to this is to implement some complicated encoding
guessing scheme. I'm not sure it's worth the effort. WDYT?

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 487 bytes --]

^ permalink raw reply	[flat|nested] 17+ messages in thread

* Re: [guile-email] decoding-error crash
  2019-09-23  5:21         ` Arun Isaac
@ 2019-09-23  5:21           ` Arun Isaac
  2019-09-23  7:30           ` Christopher Baines
  1 sibling, 0 replies; 17+ messages in thread
From: Arun Isaac @ 2019-09-23  5:21 UTC (permalink / raw)
  To: Christopher Baines; +Cc: guile-email


[-- Attachment #1.1: Type: text/plain, Size: 877 bytes --]


> I think there might be other emails on the guix-commits mailing list
> that still can't be read.
>
> For testing, I downloaded
> https://lists.gnu.org/archive/mbox/guix-commits/2019-08 and then tried
> to read it with:
>
>   (map parse-email (call-with-input-file "2019-08" mbox->emails))

These errors occur due to the occurrence of non-standard ISO-8859-1
characters in the headers due to Emacs message mode parens style
addresses. The earlier fix only allows UTF-8 or ASCII characters. I have
pushed a new fix that uses the substitute conversion strategy to
silently fail without an error in the event of something other than
UTF-8 or ASCII encoding.

https://git.systemreboot.net/guile-email/commit/?id=c10524726e0a91445a0329cc9aa72b7467258381

The alternative to this is to implement some complicated encoding
guessing scheme. I'm not sure it's worth the effort. WDYT?

[-- Attachment #1.2: signature.asc --]
[-- Type: application/pgp-signature, Size: 487 bytes --]

[-- Attachment #2: Type: text/plain, Size: 110 bytes --]

-- 
guile-email mailing list
guile-email@systemreboot.net
https://lists.systemreboot.net/listinfo/guile-email

^ permalink raw reply	[flat|nested] 17+ messages in thread

* Re: [guile-email] decoding-error crash
  2019-09-23  5:21         ` Arun Isaac
  2019-09-23  5:21           ` Arun Isaac
@ 2019-09-23  7:30           ` Christopher Baines
  2019-09-23 19:28             ` Arun Isaac
  1 sibling, 1 reply; 17+ messages in thread
From: Christopher Baines @ 2019-09-23  7:30 UTC (permalink / raw)
  To: Arun Isaac; +Cc: guile-email

[-- Attachment #1: Type: text/plain, Size: 1203 bytes --]


Arun Isaac <arunisaac@systemreboot.net> writes:

>> I think there might be other emails on the guix-commits mailing list
>> that still can't be read.
>>
>> For testing, I downloaded
>> https://lists.gnu.org/archive/mbox/guix-commits/2019-08 and then tried
>> to read it with:
>>
>>   (map parse-email (call-with-input-file "2019-08" mbox->emails))
>
> These errors occur due to the occurrence of non-standard ISO-8859-1
> characters in the headers due to Emacs message mode parens style
> addresses. The earlier fix only allows UTF-8 or ASCII characters. I have
> pushed a new fix that uses the substitute conversion strategy to
> silently fail without an error in the event of something other than
> UTF-8 or ASCII encoding.
>
> https://git.systemreboot.net/guile-email/commit/?id=c10524726e0a91445a0329cc9aa72b7467258381
>
> The alternative to this is to implement some complicated encoding
> guessing scheme. I'm not sure it's worth the effort. WDYT?

Awesome, thanks Arun. Substituting invalid characters works for me.

Would you be up for doing a release with these changes at some point?
Then the 'patch-module-dir phase could be removed from the Guix package
definition as well.

Thanks,

Chris

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 962 bytes --]

^ permalink raw reply	[flat|nested] 17+ messages in thread

* Re: [guile-email] decoding-error crash
  2019-09-23  7:30           ` Christopher Baines
@ 2019-09-23 19:28             ` Arun Isaac
  2019-09-23 19:28               ` Arun Isaac
  2019-09-24  7:17               ` Christopher Baines
  0 siblings, 2 replies; 17+ messages in thread
From: Arun Isaac @ 2019-09-23 19:28 UTC (permalink / raw)
  To: Christopher Baines; +Cc: guile-email

[-- Attachment #1: Type: text/plain, Size: 445 bytes --]


> Would you be up for doing a release with these changes at some point?
> Then the 'patch-module-dir phase could be removed from the Guix package
> definition as well.

Indeed, I am up for it! :-) I have released a new bugfix release
0.2.1. I have also updated the Guix package.

https://lists.systemreboot.net/archives/guile-email/2019/000018.html
https://git.savannah.gnu.org/cgit/guix.git/commit/?id=2c33a849f7a9c2ba68f7e37aa8ba489bf5603594

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 487 bytes --]

^ permalink raw reply	[flat|nested] 17+ messages in thread

* Re: [guile-email] decoding-error crash
  2019-09-23 19:28             ` Arun Isaac
@ 2019-09-23 19:28               ` Arun Isaac
  2019-09-24  7:17               ` Christopher Baines
  1 sibling, 0 replies; 17+ messages in thread
From: Arun Isaac @ 2019-09-23 19:28 UTC (permalink / raw)
  To: Christopher Baines; +Cc: guile-email


[-- Attachment #1.1: Type: text/plain, Size: 445 bytes --]


> Would you be up for doing a release with these changes at some point?
> Then the 'patch-module-dir phase could be removed from the Guix package
> definition as well.

Indeed, I am up for it! :-) I have released a new bugfix release
0.2.1. I have also updated the Guix package.

https://lists.systemreboot.net/archives/guile-email/2019/000018.html
https://git.savannah.gnu.org/cgit/guix.git/commit/?id=2c33a849f7a9c2ba68f7e37aa8ba489bf5603594

[-- Attachment #1.2: signature.asc --]
[-- Type: application/pgp-signature, Size: 487 bytes --]

[-- Attachment #2: Type: text/plain, Size: 110 bytes --]

-- 
guile-email mailing list
guile-email@systemreboot.net
https://lists.systemreboot.net/listinfo/guile-email

^ permalink raw reply	[flat|nested] 17+ messages in thread

* Re: [guile-email] decoding-error crash
  2019-09-23 19:28             ` Arun Isaac
  2019-09-23 19:28               ` Arun Isaac
@ 2019-09-24  7:17               ` Christopher Baines
  1 sibling, 0 replies; 17+ messages in thread
From: Christopher Baines @ 2019-09-24  7:17 UTC (permalink / raw)
  To: Arun Isaac; +Cc: guile-email

[-- Attachment #1: Type: text/plain, Size: 380 bytes --]


Arun Isaac <arunisaac@systemreboot.net> writes:

>> Would you be up for doing a release with these changes at some point?
>> Then the 'patch-module-dir phase could be removed from the Guix package
>> definition as well.
>
> Indeed, I am up for it! :-) I have released a new bugfix release
> 0.2.1. I have also updated the Guix package.

Great, thanks so much for your help Arun!

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 962 bytes --]

^ permalink raw reply	[flat|nested] 17+ messages in thread

end of thread, other threads:[~2019-09-24 14:55 UTC | newest]

Thread overview: 17+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2019-09-15 12:57 [guile-email] decoding-error crash Christopher Baines
2019-09-16  9:50 ` Arun Isaac
2019-09-16  9:50   ` Arun Isaac
2019-09-16 16:32   ` Arun Isaac
2019-09-16 16:32     ` Arun Isaac
2019-09-16 20:20     ` Arun Isaac
2019-09-16 20:20       ` Arun Isaac
2019-09-22 13:19       ` Christopher Baines
2019-09-23  5:21         ` Arun Isaac
2019-09-23  5:21           ` Arun Isaac
2019-09-23  7:30           ` Christopher Baines
2019-09-23 19:28             ` Arun Isaac
2019-09-23 19:28               ` Arun Isaac
2019-09-24  7:17               ` Christopher Baines
2019-09-16 20:55     ` Christopher Baines
2019-09-17  5:37       ` Arun Isaac
2019-09-17  5:37         ` Arun Isaac

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox