From mboxrd@z Thu Jan  1 00:00:00 1970
From: Arun Isaac <arunisaac@systemreboot.net>
To: Ricardo Wurmus <rekado@elephly.net>
Cc: guile-email@systemreboot.net
Subject: Re: [guile-email] slow parse of email with huge attachment
In-Reply-To: <87y2ppezni.fsf@elephly.net>
References: <87a726gqsx.fsf@elephly.net> <cu7wo5aniz5.fsf@systemreboot.net> <87y2ppezni.fsf@elephly.net>
Date: Mon, 25 May 2020 20:22:54 +0530
Message-ID: <cu7a71wf4mx.fsf@systemreboot.net>
MIME-Version: 1.0
Content-Type: multipart/signed; boundary="=-=-=";
	micalg=pgp-sha256; protocol="application/pgp-signature"
List-Id: <guile-email.systemreboot.net>
Message-ID: <20200525145254.l3XSSRnYw4h_oOob2uLJKhrrKRvkuHL62BmyOjEv2y0@z>

--=-=-=
Content-Type: text/plain


Hi,

I've made some improvements. I got the following snippet down from
around 16s to 6s.

--8<---------------cut here---------------start------------->8---
(statprof (lambda ()
            (map parse-email
                 (call-with-input-file "large-base64-attachment.mbox"
                   mbox->emails))))
--8<---------------cut here---------------end--------------->8---

Specifically, I made the following improvements to guile-email.

- I rewrote the base64 decoder from scratch to be a little
  faster. Earlier, I was using the decoder I had copied from Guix.
- I eliminated several unneccessary bytevector<->string conversions.
- I rewrote read-bytes-till in (email utils) to process multiple bytes
  at a time, instead of byte by byte.

There is still scope for improvement, but do test and let me know if
this serves your purpose for now. Also, I am curious to see your new
benchmark of parse-email to compare with the 8s you reported earlier.

Cheers!

--=-=-=
Content-Type: application/pgp-signature; name="signature.asc"

-----BEGIN PGP SIGNATURE-----

iQEzBAEBCAAdFiEEf3MDQ/Lwnzx3v3nTLiXui2GAK7MFAl7L28wACgkQLiXui2GA
K7MITwgAu1v0hbs+s68vODX43XkP5uTa+P1GxxA+RgkaF/NhyIovpiktPYCaPDj5
vZwntDmHfHBNmgMMDXkx2ET3UzyLGJyHWSHdUM+YOkKj9/8lxsDWh05rQzT9HcYu
aakqFuw5e5ghCPGWIGEd6h+pP5k+5YAZjXDiefNBSCdXu0JE3ikWBSBRdowlgipY
5tVGqlc6eAoY5ag0KPSceb3Lzar6LG/RmutS29saXP5STlzFtat54nDxTyjGhFhD
Q6WGdctrYeMncd4nrDDRUxmD+so/yE5lj2r+p0ingC6ZCkztxtgy8a/7f6gEohr8
/Aetg1M4cOOi3C74Ii1gxBH8RjRolQ==
=p2Y+
-----END PGP SIGNATURE-----
--=-=-=--