Commit Graph

20 Commits

Author SHA1 Message Date
Christopher Tate
a100c02b8d Compress the backup output stream
Zlib compression, with a full flush between each application's
data.  Encryption will be performed on the already-compressed data
once that's implemented.

On restore, the streamed data is similarly uncompressed on the fly.

Change-Id: I19b65c88e759a66527d10913d18fffa9df0bc011
2011-07-13 15:30:41 -07:00
Christopher Tate
61334a3f83 Fix handling of directory entries
Don't emit tar blocks for directories with an invalid nonzero size.  Also, if
such an entry is encountered during restore, don't actually attempt to treat
it as valid and thus skip over the next actual tar entry.

This patch also adds tracking of the data actually consumed during restore,
and reports a total at the end of stream.

Change-Id: I625173f76df3c007e899209101ff2b587841f184
2011-06-08 20:11:49 -07:00
Christopher Tate
9f7bcda136 Fix embedded spaces in tar stream EVEN HARDER
Change-Id: I97ac586ff3541a05d73e1e53f680517c15e6c662
2011-06-07 13:17:17 -07:00
Christopher Tate
72a404d102 Implement shared-storage full backup/restore
Every available shared-storage volume is backed up, tagged with its
ordinal in the set of mounted shared volumes.  This is an approximation
of "internal + the external card".  This lets us restore things to the
same volume [or "equivalent" volume, in the case of a cross-model
restore] as they originated on.

Also fixed a bug in the handling of files/dirs with spaces in
their names.

Change-Id: I380019da8d0bb5b3699bd7c11eeff621a88e78c3
2011-06-07 12:16:27 -07:00
Christopher Tate
15e113033b Restore from a previous full backup's tarfile
Usage:  adb restore [tarfilename]

Restores app data [and installs the apps if necessary from the backup
file] captured in a previous invocation of 'adb backup'.  The user
must explicitly acknowledge the action on-device before it is allowed
to proceed; this prevents any "invisible" pushes of content from the
host to the device.

Known issues:

* The settings databases and wallpaper are saved/restored, but lots
  of other system state is not yet captured in the full backup.  This
  means that for practical purposes this is usable for 3rd party
  apps at present but not for full-system cloning/imaging.

Change-Id: I0c748b645845e7c9178e30bf142857861a64efd3
2011-06-01 15:09:55 -07:00
Christopher Tate
b41c5a5ae7 Full backup tweaks
* provide placeholder UI showing backup/restore start/stop/timeout
* don't kill the progress UI in mid stream
* tidy up the pax extended header data writing a little

Change-Id: Ife0cb78e3facb541d8327f1d5ca5fe77faa6cbca
2011-05-13 15:38:02 -07:00
Christopher Tate
ffd242ed55 Use pax extended tar format to support long filenames etc.
'tar' supports only 100-character paths; 'ustar' supports only
155+100 character prefix + paths; neither supports files larger
than about 8 gigabytes.  We now use the POSIX.1-2001 'pax'
extended tar format for those files in the backup stream that
are too large or have too-long paths for the 'ustar' format.

Change-Id: I2f256823091deaec9b1ccea685d2344753c6cb67
2011-05-12 18:07:57 -07:00
Christopher Tate
d78797f6e6 Full local backup infrastructure
This is the basic infrastructure for pulling a full(*) backup of the
device's data over an adb(**) connection to the local device.  The
basic process consists of these interacting pieces:

1. The framework's BackupManagerService, which coordinates the
   collection of app data and routing to the destination.

2. A new framework-provided BackupAgent implementation called
   FullBackupAgent, which is instantiated in the target applications'
   processes in turn, and knows how to emit a datastream that contains
   all of the app's saved data files.

3. A new shell-level program called "bu" that is used to bridge from
   adb to the framework's Backup Manager.

4. adb itself, which now knows how to use 'bu' to kick off a backup
   operation and pull the resulting data stream to the desktop host.

5. A system-provided application that verifies with the user that
   an attempted backup/restore operation is in fact expected and to
   be allowed.

The full agent implementation is not used during normal operation of
the delta-based app-customized remote backup process.  Instead it's
used during user-confirmed *full* backup of applications and all their
data to a local destination, e.g. via the adb connection.

The output format is 'tar'.  This makes it very easy for the end
user to examine the resulting dataset, e.g. for purpose of extracting
files for debug purposes; as well as making it easy to contemplate
adding things like a direct gzip stage to the data pipeline during
backup/restore.  It also makes it convenient to construct and maintain
synthetic backup datasets for testing purposes.

Within the tar format, certain artificial conventions are used.
All files are stored within top-level directories according to
their semantic origin:

apps/pkgname/a/  : Application .apk file itself
apps/pkgname/obb/: The application's associated .obb containers
apps/pkgname/f/  : The subtree rooted at the getFilesDir() location
apps/pkgname/db/ : The subtree rooted at the getDatabasePath() parent
apps/pkgname/sp/ : The subtree rooted at the getSharedPrefsFile() parent
apps/pkgname/r/  : Files stored relative to the root of the app's file tree
apps/pkgname/c/  : Reserved for the app's getCacheDir() tree; not stored.

For each package, the first entry in the tar stream is a file called
"_manifest", nominally rooted at apps/pkgname.  This file contains some
metadata about the package whose data is stored in the archive.

The contents of shared storage can optionally be included in the tar
stream. It is placed in the synthetic location:

shared/...

uid/gid are ignored; app uids are assigned at install time, and the
app's data is handled from within its own execution environment, so
will automatically have the app's correct uid.

Forward-locked .apk files are never backed up.  System-partition
.apk files are not backed up unless they have been overridden by a
post-factory upgrade, in which case the current .apk *is* backed up --
i.e. the .apk that matches the on-disk data.  The manifest preceding
each application's portion of the tar stream provides version numbers
and signature blocks for version checking, as well as an indication
of whether the restore logic should expect to install the .apk before
extracting the data.

System packages can designate their own full backup agents.  This is
to manage things like the settings provider which (a) cannot be shut
down on the fly in order to do a clean snapshot of their file trees,
and (b) manage data that is not only irrelevant but actively hostile
to non-identical devices -- CDMA telephony settings would seriously
mess up a GSM device if emplaced there blind, for example.

When a full backup or restore is initiated from adb, the system will
present a confirmation UI that the user must explicitly respond to
within a short [~ 30 seconds] timeout.  This is to avoid the
possibility of malicious desktop-side software secretly grabbing a copy
of all the user's data for nefarious purposes.

(*) The backup is not strictly a full mirror.  In particular, the
    settings database is not cloned; it is handled the same way that
    it is in cloud backup/restore.  This is because some settings
    are actively destructive if cloned onto a different (or
    especially a different-model) device: telephony settings and
    AndroidID are good examples of this.

(**) On the framework side it doesn't care that it's adb; it just
    sends the tar stream to a file descriptor.  This can easily be
    retargeted around whatever transport we might decide to use
    in the future.

KNOWN ISSUES:

* the security UI is desperately ugly; no proper designs have yet
  been done for it
* restore is not yet implemented
* shared storage backup is not yet implemented
* symlinks aren't yet handled, though some infrastructure for
  dealing with them has been put in place.

Change-Id: Ia8347611e23b398af36ea22c36dff0a276b1ce91
2011-05-10 17:52:51 -07:00
Joe Onorato
6eabaa311f Better (and less) logging from backup. 2009-06-26 17:19:11 -04:00
Christopher Tate
bd95c1d3af Only report "unknown metadata" once per restore helper
Also removes the auto-free object, replacing it with direct memory manipulation.
2009-06-24 13:57:29 -07:00
Christopher Tate
6441a76b2e Put back LOGP -> printf in the backup helper code 2009-06-24 11:20:51 -07:00
Christopher Tate
5c2882b25c Preserve file access mode when backing up / restoring files
This change adds a fixed-size metadata block at the head of each file's content
entity.  The block is versioned, and fixed-size on the theory that it might be
nice to be able to recover the content (if not the full metadata) of the files
if we're ever confronted with data backed up some hypothetical future helper
that stored expanded metadata.

The net effect is that now on restore, we assign the same access mode to the
file that it originally had when backed up.

Also, some of the code was failing to properly free transient heap-based buffers
when it encountered errors.  This has been fixed with the addition of a tiny
stack-based object whose job it is to free() its designated pointer from its
destructor.
2009-06-23 17:40:44 -07:00
Christopher Tate
ab2e9e81e1 Add file mode to the file-backup saved state blobs
This change puts the file's access mode into the saved-state blob used by the
file backup helpers.  The tests have been updated for the new blob content
format.

What this change *doesn't* do is actually backup/restore the file mode.  This
change is a prerequisite for that, but mode preservation in backup/restore will
require adding metadata to the backup data stream itself, so will be approached
a bit more carefully.

(Also fixed one outright bug in the test program: ReadEntityData() had been
changed to return a ssize_t union of either a byte-count or a negative number
indicating error, but the test program was still assuming that nonzero == error,
and was spuriously failing.)
2009-06-23 13:07:47 -07:00
Joe Onorato
6bda7fd556 backup stuff 2009-06-18 18:41:11 -07:00
Joe Onorato
da1430be26 Make RestoreHelper and friends also write out the snapshot state. 2009-06-18 18:41:11 -07:00
Joe Onorato
03aa8d7d04 checkpoint BackupDatAInput / RestoreHelper 2009-06-16 18:46:50 -07:00
Joe Onorato
1a9e19a73e Make the file backup helper not crash if a file you requested
can't be stated.  This means you don't need to know if the files
you are backing up exist or not -- we'll figure it out for you.
2009-06-11 14:51:45 -07:00
Joe Onorato
0ad6120dad Fix SharedPrefsBackupHelper so it doesn't hard code the paths to the files.
This took quite a bit of refactoring.
2009-06-11 11:29:57 -07:00
Christopher Tate
173e38d47c Fix back_up_files() error detection when opening/CRCing the file 2009-06-04 17:02:56 -07:00
Mathias Agopian
f446ba9dcb rename a few files to camel-case, add copyright notices 2009-06-04 13:53:57 -07:00