Commit Graph

120 Commits

Author SHA1 Message Date
Andrew Stadler
87fcd03a1b Improve handling of content-disposition
* Check array returned by split("=")
* Add unit tests for this case
* Also add unit tests for quoting removal

Bug: 3040796
Change-Id: I170f3cd483fe35186194edeb0c3142fb0e2e9b75
2010-09-27 15:09:35 -07:00
Makoto Onuki
367ebd7fa5 Handle multiple IMAP SEARCH results.
Apparently IMAP servers may return multiple SEARCH responses for a
single SEARCH command, and we need to handle all of them.

Before the IMAP rework there was 3 methods that issued the SEARCH command.
Two of them ware doing it right, but the other wasn't, which was what
I copied from, unfortunately!

In case you're wondering, originally the test for this method was done through
upper methods, e.g. getMessage().

Bug 2911647

Change-Id: Ia50072944d5b01c1e59541c3a966067b13910cc4
2010-08-12 11:10:00 -07:00
Makoto Onuki
c14ff6ea45 Fix handling IOException in ImapStore
- mConnection.destroyResponses() should be protected with
  if (mConnection != null).
  When we get an IOException, we close the connection and null it out in
  ioExceptionHandler().  So mConnection can be null at any point after
  where ioExceptionHandler() first appears.

- ioExceptionHandler should close its parent ImapFolder only if the argument
  connection is mConnection.
  Methods like exists() may pass an ImapConnection which is not mConnection
  to ioExceptionHandler.  In which case we don't have to close the ImapFolder.

Bug 2898211

Change-Id: I8f9f45d91f596bb8da1a1575593e652d66deb643
2010-08-11 14:44:09 -07:00
Makoto Onuki
ae8fc30b41 Fix "disk I/O error" in LocalStoreUnitTests
The problem was that we didn't close the LocalStores created in these
testDbUpgradeXxx tests.

Also,
- Make sure to close databases.
- Make sure to close cursors.
- Cleard up warnings (unnecessary casts, etc.)

Bug 2859264

Change-Id: Ifaddbb6cf07794a7b5978564ea8fbb3cbf75b978
2010-07-28 13:07:33 -07:00
Marc Blank
80ee607a7c Send intro text with SmartForward
* We need to include the intro text (--Original Message--, etc.) to
  SmartForwards, and somehow this got in a past updat
* Add unit test for forwarding
* Fix unit test for reply so that it works localized

Bug: 2477988
Bug: 2685784
Change-Id: I8d92f00d37a434840ec3eb237f3901cd5dc7ad09
2010-07-09 09:39:58 -07:00
Makoto Onuki
34f29c8a74 Test for interaction between ImapStore and vendor policy.
Change-Id: I092b3a0f2f40d9aa19f2f61066362099c8b3f50b
2010-06-15 13:44:31 -07:00
Makoto Onuki
bf58053406 Follow-up to the new IMAP parser.
- Replace string literals in ImapStore with constants.
- Simplifies ImapStore.en/decodeFolderName
- Mix cases in the test data to test for case-insensitivity

Change-Id: I88424357227bcf78528df5e6a1c4ba45d54cc65b
2010-06-01 10:50:15 -07:00
Makoto Onuki
7e5ba0e1ea New IMAP parser to fix long-lasting problems.
- Almost completely re-wrote ImapResponseParser layer
- We no longer use simple ArrayList and String to represent
  imap response.  We have classes for that.  (Type safe!)
  These classes are also NPE-free.
  (which isn't necessarily a good thing, though)
- A lot of clean-ups and fixes in ImapStore.
- More tests for ImapStore.

Now ImapResponseParser moved to com.android.email.mail.store.imap.parser,
but inside, it's 99% new code.

This CL introduces many new classes, but most of them are small classes
to represent the IMAP response.

Problems that this CL fixes includes:
- Special characters in OK response
- Handling BYE response
- Case sensitivity
- ClassCast/ArrayIndexOutOfBound/NumberFormatException
- Handling NIL/literals at any position

Bug 2480227
Bug 2244049
Bug 2138981
Bug 1351896
Bug 2591435
Bug 2173061
Bug 2370627
Bug 2524881
Bug 2525902
Bug 2538076

Change-Id: I7116f57fba079b8a5ef8d5439a9b3d9a9af8e6ed
2010-05-28 15:59:09 -07:00
Makoto Onuki
32f8a49e01 Two new tests for the bottom half of ImapFolder.append().
They cover the cases:
- when OK response doesn't have APPENDUID.
- when APPEND fails.

Change-Id: I13a0fb8aaf2e1cbb5a1f055c9ce56e2891373ea0
2010-05-20 18:23:16 -07:00
Makoto Onuki
cbe4ae9291 More tests for IMAP, clean up, and a few bug fixes.
- A few new tests in ImapStoreUnitTests.
- Added TODOs to ImapStoreUnitTests (for mainly NO response handling)
- Renamed ImapStore.releaseConnection to poolConnection.
- Fixed a bug in getConnection where it'd return a closed connection.
- Now getConnection() hanles BYE response for NOOP correctly and treat the
  connection as closed.

Change-Id: I48e5b89049338f7d4f1ac77cd7ac7243945a9575
2010-05-19 13:13:18 -07:00
Makoto Onuki
2552b7b705 More test for ImapStore/ImapFolder.
- Also, fixed a potential crash in getMessages().
  It could happen when a client is gettign a message list while
  another client is removing messages.

Change-Id: I04b1de6bc384cffb7a5286bcec0a349a3d62a623
2010-05-18 17:14:30 -07:00
Makoto Onuki
7d3519151a Tests for IMAP FETCH
Adding regression test for the new IMAP parser.

Change-Id: Iac7f5c022e44ca5f06f735e145af15cc459eb61f
2010-05-18 10:48:11 -07:00
Makoto Onuki
128fb393e8 Relax MIME date parser.
Make the date parser accept invalid dates like
"Thu, 10 Dec 09 15:08:08 GMT-0700" which was observed in an email from eBay.

Per RFC, timezone must be either obs-zone (e.g. "GMT") or +/- with 4 digits.
The GMT+/-digits format is not permitted.

Bug 2367124

Change-Id: I59968274160aeadea70223208b463ee692660056
2010-05-17 14:14:08 -07:00
Makoto Onuki
0a8030562a Fix flaky tests
Follow up to I3bf7d340.  Make sure temp directory is set before running tests.

Turned out Application.onCreate doesn't seem to be guaranteed to be run
before unit tests.

Without this, some tests may fail saying: "TempDirectory not set.
Application hasn't started??", if onCreate runs too late.

Change-Id: Ic5aee939a2c21f9579a643d0729dd0e9ba81022e
2010-05-17 14:07:34 -07:00
Makoto Onuki
9c9e757917 Add static method to get temp dir in Email.
I need to be able to get the temp dir from anywhere without Context
for the new IMAP parser.

Change-Id: I3bf7d34059399a8253c0760ebc392804ea434412
2010-05-13 18:27:34 -07:00
Makoto Onuki
c563ce1b80 am b8970dab: am e46ac1a6: Cleanup imports. 2010-05-07 15:20:47 -07:00
Makoto Onuki
b8970dabb6 am e46ac1a6: Cleanup imports.
Merge commit 'e46ac1a61f32bd762a879457b1850087a1ad3a8c' into kraken

* commit 'e46ac1a61f32bd762a879457b1850087a1ad3a8c':
  Cleanup imports.
2010-05-07 15:17:35 -07:00
Makoto Onuki
e46ac1a61f Cleanup imports.
(Note it's for froyo-plus-asop)

Change-Id: Ia5b771cd41099ab486ba2e556b441fc7c0affe5f
2010-05-07 14:33:44 -07:00
Makoto Onuki
92db71e440 Merge "Send local IP address with EHLO instead of "localhost"." 2010-04-22 14:23:46 -07:00
Makoto Onuki
3844bbcaaf am 2ca8d734: am 6cfa8001: Merge "Fix failing unit tests" into froyo
Merge commit '2ca8d7347db3975746ad87347de57fd5800bf5d4' into kraken

* commit '2ca8d7347db3975746ad87347de57fd5800bf5d4':
  Fix failing unit tests
2010-04-22 13:24:40 -07:00
Makoto Onuki
f4dac9f266 Send local IP address with EHLO instead of "localhost".
Bug 1515345

Change-Id: I181c9f0d79fbdf62f7df77f72a1ec9653797b6dd
2010-04-22 11:23:08 -07:00
Makoto Onuki
25d55ed266 Fix failing unit tests
Bug 2614782

Change-Id: I7ed7bf61a567429017f64d9049577ec076331bb2
2010-04-22 09:18:42 -07:00
Marc Blank
fc08120213 am a0c71419: am 21c04fe3: Merge "Further harden getListOrNull" into froyo
Merge commit 'a0c71419cedaa0f84d12da61e415642b79cc0a9c' into kraken

* commit 'a0c71419cedaa0f84d12da61e415642b79cc0a9c':
  Further harden getListOrNull
2010-04-21 08:18:04 -07:00
Marc Blank
07fb9f8bea Further harden getListOrNull
* Add a bounds check
* Add a test for ImapList

Bug: 2611022
Change-Id: I3ad88fbc8d3145298731ab19ef7ff68d4011bb00
2010-04-20 19:31:29 -07:00
Andrew Stadler
ead786d9b5 am ef01261a: am 9cc6f46b: Merge "Try TOP even on POP servers that fail to report CAPA" into froyo
Merge commit 'ef01261a8909ea3fe3e40b06cc537e0032d47898' into kraken

* commit 'ef01261a8909ea3fe3e40b06cc537e0032d47898':
  Try TOP even on POP servers that fail to report CAPA
2010-04-12 11:33:04 -07:00
Andrew Stadler
645fc28301 Try TOP even on POP servers that fail to report CAPA
* Ignore the results of CAPA and always try TOP
* If TOP returns -ERR simply fall back to (bad old slow) RETR
* Unit tests for positive & failure cases

Bug: 2588432
Change-Id: Ife4b551217de1025e14efc46074f16ef4ae99c6f
2010-04-12 10:56:55 -07:00
Andrew Stadler
5f95d68288 am 3839dc98: Merge "Improve MIME & SMTP compliance on outbound messages" into froyo
Merge commit '3839dc981f15bb7da5e7500ee175485dc85a6f6a' into kraken

* commit '3839dc981f15bb7da5e7500ee175485dc85a6f6a':
  Improve MIME & SMTP compliance on outbound messages
2010-04-08 10:03:11 -07:00
Andrew Stadler
6bcccf6284 Improve MIME & SMTP compliance on outbound messages
* Write MIME-Version: 1.0 in all outbound messages, not just those
  with multiparts.  This is required by RFC 2045.
* Unit tests

Bug: 1678296
Change-Id: Icf37d93b8b0150f490791792499865a60744adea
2010-04-07 21:46:15 -07:00
Makoto Onuki
e2a076fdfd am af672452: Added a test for IMAP APPEND
Merge commit 'af6724527e564d35dd27ac35e24dbced554792e5' into kraken

* commit 'af6724527e564d35dd27ac35e24dbced554792e5':
  Added a test for IMAP APPEND
2010-04-06 14:43:22 -07:00
Makoto Onuki
af6724527e Added a test for IMAP APPEND
It's a preliminary change for IMAP bug fixes.

Also,
- Fixed a potential bug in ImapFolder.setFlags where it'd throw
  StringIndexOutOfBoundsException if flags is empty.

- Added a generic flag to proguard.flags so that now all methods with
  the "ForTest" sufix are automatically preserved.
  Turned out it wasn't needed for this CL, but it should come in handy
  someday.

Bug 2538076
Change-Id: I49a08afc196c7b7f1f30477dfc38ac5381045d84
2010-04-06 14:33:43 -07:00
Andrew Stadler
dc0753373e DO NOT MERGE. Handle STARTTLS when last line in EHLO response.
When receiving the EHLO response from the SMTP server, the multiline
answer has "-" prefix in all lines except the last line, where the
prefix is a blank. This is according to RFC 2821 section 4.2.1. This has
also been reported as issue 2309 at code.google.com.

Bug: 1744768

Change-Id: I3feccabed30767d2fa5b06352cd7d1c803e8d59c
2010-04-06 10:50:11 -07:00
Andrew Stadler
c4fcd852ba Eliminate duplication in Yahoo! sent mailbox
* Yahoo! is not supporting search by UID so I can't identify the new
  the UID after I upload.  This inability to correlate the local and
  remote messages means that we wind up syncing the same message back
  down, in a loop, which spawns more messages.
* Yahoo! has partial support for UIDPLUS, and reports the new UID when
  I append (upload) messages.
* Modify IMAP parser to parse response lists
* When APPENDUID is reported, use it (and skip the search)
* Modify the few other existing users of response lists to use the
  parsed versions instead.  Provided a couple of lightweight utilities
  to make it easier to work with ImapList.
* Unit tests for most of it.
* Optimization: share a static date/time parser for all IMAP connections

Bug: 2448220
Change-Id: Ic10fc1a195ccf4671a498188cc8b17848c8d9df7
2010-03-18 10:11:08 -07:00
Marc Blank
0ed690bfb4 Send meeting invite mail as multipart/alternative
* Turns out that Exchange 2003 requires the ics attachment to be in a
  multipart/alternative, rather than a multipart/mixed MIME message
* Exchange 2007 accepts both types
* Therefore, we change our output for this particular situation, i.e.
  a single attachment that is an ics file, to multipart/alternative
* Rename FLAG_SUPPRESS_CONTENT_DISPOSITION to FLAG_ICS_ALTERNATIVE_PART
  and make this flag do double duty - 1) suppress the Content-Disposition
  header (also required by Exchange) and 2) send the message as
  multipart/alternative
* Add unit tests for Rfc822Output to check that mime parts are composed
  properly

Bug: 2516394
Change-Id: I60e26f57b8ecaf01d0340e7828533334e0e7d45a
2010-03-17 09:59:24 -07:00
Andrew Stadler
319155a061 Merge "Follow-up to MimeMessage efficiency improvements." 2010-03-11 11:58:36 -08:00
Andrew Stadler
dfd53b0e82 Follow-up to MimeMessage efficiency improvements.
I missed a case where message-id should not be set locally, which is
the case where the Mime parser clears all headers *and* does not find
a message-id.  The parsed MimeMessage should accurately reflect this.

In the old code, the local id was created at construction time and then
immediately discarded by the parser (calling headers.clear()).

In the new code, I was generating a message-id any time I couldn't find
one.  Now, when explicitly cleared or removed, I set a boolean to inhibit
automatic generation of a new one.

I also missed the fact that a missing message-id no longer throws an
exception, it simply returns null, and so I changed the code that was
catching that exception to simply check for null.

(Note:  Clearly, modeling of legacy behavior is becoming annoying here;
It would be better to do away with all of the automatic logic, and simply
generate message-id locally when appropriate:  On locally-generated
messages.  I don't want to touch this for the current release, but I left
a note in the code to this effect.)

Bug: 2504774
Change-Id: Ibfcbd2363c7ae39ee6d44e4c3295f88258cb4945
2010-03-10 16:42:49 -08:00
Makoto Onuki
69ade70f2d Log recent network activities when IMAP parser crashes.
Added DiscourseLogger, which stores last N (currently 64) lines of IMAP
commands sent to the server and responses received from the server.

We dump it to logcat when the IMAP parser crashes, that is, a) getting a
RuntimeException in ImapFolder.fetch() or b) getting a Runtime/IOException
in ImapResponseParser.

Bug 2480227

Change-Id: I6b5a728a7df106627ec29bb3c7c04a97a99b444b
2010-03-10 13:08:09 -08:00
Andrew Stadler
44552da606 Reduce memory overhead in Message and MimeMessage
Message and MimeMessage were creating a lot of unnecessary sub-objects
even when not needed, so do a bunch of lazy initialization.  This should
raise the bar on the size of gigantic inboxes giving us trouble.

* Specific optimizations:
  * Replace date formatter with a shared static
  * lazy create mHeader (ArrayList)
  * lazy create mFlags (HashSet)
  * optimize MimeHeader fields class
  * lazy create local message-ID (expensive-to-make uuid String)
  * make message-id string less expensive to create
* Other cleanups:
  * add some override annotations
  * privatize some members
  * update a fragile test (not a deep fix, it's still fragile)

Side effect, should be faster too.

Bug: 2357564
Bug: 2093422
Change-Id: I8a873879d402e2662339d5398ad0b15da6e580e9
2010-03-09 20:30:47 -08:00
Andrew Stadler
fd249f61dd More work on account migration
* Split account copy loop to do POP3 accounts first, then IMAP
* After upgrading accounts, upgrade folders
* Upgrade messages in those folders
* Preserve attachments on outgoing messages (e.g. drafts)
* Enable composer and start syncing after upgrade
* Fix latent bug in LocalStore (which was not used in Eclair)
* Add tests for upgrade workers in LegacyConversions

Bug: 2065528
2010-03-08 13:53:01 -08:00
Makoto Onuki
726a9fcef3 Ignore FETCH responses that don't have UID.
We've observed that the secure.emailsrvr.com email server returns an excess
FETCH response for a UID FETCH command.  Excess responses don't have the
UID field, even though we request, which led the response parser to crash.

This patch fixes it by making the parser ignore response lines that don't
have UID.

Bug: 2441065
2010-03-01 16:00:05 -08:00
Andrew Stadler
cb95fbe135 Send CAPABILITY to all IMAP servers
* Send CAPABILITY to all servers, not just when we check TLS
* Feed capabilities into IMAP ID generation
* Unit tests updated

Bug: 2332183
2010-02-25 22:48:11 -08:00
Andrew Stadler
5afa187791 Create a PackedString utility class
* We can use this for meeting request information which will start out with
  one or two pieces of information, but might grow in the future.
* Binary compatible with Address.pack() format, so we can eventually
  combine code.
2010-02-22 22:34:45 -08:00
Makoto Onuki
961ee151fc Adding more tests for MimeMessage.
Added 2 tests to make sure the parse won't crash with invalid header.

Bug: 2258519
2010-02-10 11:32:05 -08:00
Andrew Stadler
ecb1af8041 Finish up IMAP ID implementation
* scrub all external strings to keep them compliant for IMAP protocol
* move Build.MODEL to x-android-device-model
* send x-android-mobile-net-operator
* send AGUID
* unit tests for above
* retrieve providers from VendorPolicyLoader

Bug: 2332183
2010-02-01 15:53:46 -08:00
The Android Open Source Project
d423fb996f am 08d92ebf: merge from open-source master
Merge commit '08d92ebf4722e45216ee225775a3b86f9ceecc0d'

* commit '08d92ebf4722e45216ee225775a3b86f9ceecc0d':
  Trim the mime type portion of Content-Type.
2010-01-27 16:29:17 -08:00
Andrew Stadler
468371917e IMAP ID
* Add IMAP ID command to all login sequences
* Send generic information for now
* Explicitly catch & discard parsing errors, since we really don't
  care if the command succeeds or not.
* Unit tests

Bug: 2332183
2010-01-25 18:40:45 -08:00
Patrick Horn
2517a3dfd1 Trim the mime type portion of Content-Type.
Fixes parsing of Content-Type headers like "text/plain ; charset=iso-8859-1"
Added unit tests for whitespace and the charset parameter
2010-01-21 11:46:20 -08:00
Jean-Baptiste Queru
56e48981f0 eclair snapshot 2009-11-12 18:46:09 -08:00
Andrew Stadler
b6756688b1 Handle IMAP empty bodies more safely
Some IMAP servers return NIL if you BODY.PEEK[TEXT] a messsage with
no body, instead of the more canonical {0}CRLF.  Instead of messing with the
parser to deal with that, it makes more sense not to try and fetch empty
bodies.  So there are three changes:

* Don't fetch parts when size = 0
* Don't append "null" when there is null body text
* Slight change to attachment handling so size is reported >0
* Unit tests on some of the related lower-level protocol stuff

Bug http://b/issue?id=2160387

Change-Id: Ifb8fb0ed5ce7297908e1ae8d5a02dda5975c4a3c
2009-10-07 11:42:27 -07:00
Andrew Stadler
e4a7cc440f Re-enable modernized version of "optional" SSL/TLS
* Add "Accept all certificates" modes to incoming/outgoing secure choices
* Change URI scheme slightly to make "trust" a flag, not part of the
    protocol.
* Change Stores to know about new URI scheme
* Slightly rework Transport API to make "trust" an independent flag
* Adapt HostAuth to handle new Uri scheme
* Remove the old ambiguous "optional" code, which was allowing
    some unsigned certificates, but was *also* allowing TLS to
    optionally start (though not SSL, despite the UI strings.)
* Add a few unit tests to EmailContent
* Add logging and a bunch of comments to TrustManagerFactory, and a bit
    of simple cleanup to make it more readable.
* Add missing conversion of SSLException->CertificateValidationException
    in TLS so we get the correct certificate errors from TLS too.
* Re-enable TLS for mac.com accounts (which had a certificate problem)

Fixes bug http://b/2119755, http://b/1374780, and probably a raft of
earlier and/or external bugs about certificate problems.

Change-Id: Iaf99a8da3eaadaa4cdeec224737838b5d6813e55
2009-09-29 15:28:43 -07:00
Andrew Stadler
c41c47fa07 Enable message upload
* Create logic to detect upsyncable messages in Sent
* Note:  Drafts is now local only for IMAP - no sync, either way
* Rewrite MessageController.processPendingAppend for Provider world
* Write provider message -> legacy message converter
* Fixed bug in IMAP APPEND (it was not picking the right UID for the
    uploaded message.)
* Better handling of server internaldate
* Add constants for new X-Android-Body-Quoted-Part header
* Add EmailContent routines to get each of the 5 parts of the body
* Remove "Load more" from unsynced message lists
* Add toString to MimeHeader for debug support

Bug # 2097471

TODO (next CL): Upload attachments records too

Change-Id: I209182f5adc6b6696919f559e3cbbdd58b3eed3a
2009-09-25 14:54:32 -07:00