replicant-frameworks_native/libs/utils/BackupData.cpp

382 lines
9.0 KiB
C++
Raw Normal View History

/*
* Copyright (C) 2009 The Android Open Source Project
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
#define LOG_TAG "backup_data"
#include <utils/BackupHelpers.h>
#include <utils/ByteOrder.h>
#include <stdio.h>
Full local backup infrastructure This is the basic infrastructure for pulling a full(*) backup of the device's data over an adb(**) connection to the local device. The basic process consists of these interacting pieces: 1. The framework's BackupManagerService, which coordinates the collection of app data and routing to the destination. 2. A new framework-provided BackupAgent implementation called FullBackupAgent, which is instantiated in the target applications' processes in turn, and knows how to emit a datastream that contains all of the app's saved data files. 3. A new shell-level program called "bu" that is used to bridge from adb to the framework's Backup Manager. 4. adb itself, which now knows how to use 'bu' to kick off a backup operation and pull the resulting data stream to the desktop host. 5. A system-provided application that verifies with the user that an attempted backup/restore operation is in fact expected and to be allowed. The full agent implementation is not used during normal operation of the delta-based app-customized remote backup process. Instead it's used during user-confirmed *full* backup of applications and all their data to a local destination, e.g. via the adb connection. The output format is 'tar'. This makes it very easy for the end user to examine the resulting dataset, e.g. for purpose of extracting files for debug purposes; as well as making it easy to contemplate adding things like a direct gzip stage to the data pipeline during backup/restore. It also makes it convenient to construct and maintain synthetic backup datasets for testing purposes. Within the tar format, certain artificial conventions are used. All files are stored within top-level directories according to their semantic origin: apps/pkgname/a/ : Application .apk file itself apps/pkgname/obb/: The application's associated .obb containers apps/pkgname/f/ : The subtree rooted at the getFilesDir() location apps/pkgname/db/ : The subtree rooted at the getDatabasePath() parent apps/pkgname/sp/ : The subtree rooted at the getSharedPrefsFile() parent apps/pkgname/r/ : Files stored relative to the root of the app's file tree apps/pkgname/c/ : Reserved for the app's getCacheDir() tree; not stored. For each package, the first entry in the tar stream is a file called "_manifest", nominally rooted at apps/pkgname. This file contains some metadata about the package whose data is stored in the archive. The contents of shared storage can optionally be included in the tar stream. It is placed in the synthetic location: shared/... uid/gid are ignored; app uids are assigned at install time, and the app's data is handled from within its own execution environment, so will automatically have the app's correct uid. Forward-locked .apk files are never backed up. System-partition .apk files are not backed up unless they have been overridden by a post-factory upgrade, in which case the current .apk *is* backed up -- i.e. the .apk that matches the on-disk data. The manifest preceding each application's portion of the tar stream provides version numbers and signature blocks for version checking, as well as an indication of whether the restore logic should expect to install the .apk before extracting the data. System packages can designate their own full backup agents. This is to manage things like the settings provider which (a) cannot be shut down on the fly in order to do a clean snapshot of their file trees, and (b) manage data that is not only irrelevant but actively hostile to non-identical devices -- CDMA telephony settings would seriously mess up a GSM device if emplaced there blind, for example. When a full backup or restore is initiated from adb, the system will present a confirmation UI that the user must explicitly respond to within a short [~ 30 seconds] timeout. This is to avoid the possibility of malicious desktop-side software secretly grabbing a copy of all the user's data for nefarious purposes. (*) The backup is not strictly a full mirror. In particular, the settings database is not cloned; it is handled the same way that it is in cloud backup/restore. This is because some settings are actively destructive if cloned onto a different (or especially a different-model) device: telephony settings and AndroidID are good examples of this. (**) On the framework side it doesn't care that it's adb; it just sends the tar stream to a file descriptor. This can easily be retargeted around whatever transport we might decide to use in the future. KNOWN ISSUES: * the security UI is desperately ugly; no proper designs have yet been done for it * restore is not yet implemented * shared storage backup is not yet implemented * symlinks aren't yet handled, though some infrastructure for dealing with them has been put in place. Change-Id: Ia8347611e23b398af36ea22c36dff0a276b1ce91
2011-04-01 21:43:32 +00:00
#include <string.h>
#include <unistd.h>
#include <cutils/log.h>
namespace android {
Full local backup infrastructure This is the basic infrastructure for pulling a full(*) backup of the device's data over an adb(**) connection to the local device. The basic process consists of these interacting pieces: 1. The framework's BackupManagerService, which coordinates the collection of app data and routing to the destination. 2. A new framework-provided BackupAgent implementation called FullBackupAgent, which is instantiated in the target applications' processes in turn, and knows how to emit a datastream that contains all of the app's saved data files. 3. A new shell-level program called "bu" that is used to bridge from adb to the framework's Backup Manager. 4. adb itself, which now knows how to use 'bu' to kick off a backup operation and pull the resulting data stream to the desktop host. 5. A system-provided application that verifies with the user that an attempted backup/restore operation is in fact expected and to be allowed. The full agent implementation is not used during normal operation of the delta-based app-customized remote backup process. Instead it's used during user-confirmed *full* backup of applications and all their data to a local destination, e.g. via the adb connection. The output format is 'tar'. This makes it very easy for the end user to examine the resulting dataset, e.g. for purpose of extracting files for debug purposes; as well as making it easy to contemplate adding things like a direct gzip stage to the data pipeline during backup/restore. It also makes it convenient to construct and maintain synthetic backup datasets for testing purposes. Within the tar format, certain artificial conventions are used. All files are stored within top-level directories according to their semantic origin: apps/pkgname/a/ : Application .apk file itself apps/pkgname/obb/: The application's associated .obb containers apps/pkgname/f/ : The subtree rooted at the getFilesDir() location apps/pkgname/db/ : The subtree rooted at the getDatabasePath() parent apps/pkgname/sp/ : The subtree rooted at the getSharedPrefsFile() parent apps/pkgname/r/ : Files stored relative to the root of the app's file tree apps/pkgname/c/ : Reserved for the app's getCacheDir() tree; not stored. For each package, the first entry in the tar stream is a file called "_manifest", nominally rooted at apps/pkgname. This file contains some metadata about the package whose data is stored in the archive. The contents of shared storage can optionally be included in the tar stream. It is placed in the synthetic location: shared/... uid/gid are ignored; app uids are assigned at install time, and the app's data is handled from within its own execution environment, so will automatically have the app's correct uid. Forward-locked .apk files are never backed up. System-partition .apk files are not backed up unless they have been overridden by a post-factory upgrade, in which case the current .apk *is* backed up -- i.e. the .apk that matches the on-disk data. The manifest preceding each application's portion of the tar stream provides version numbers and signature blocks for version checking, as well as an indication of whether the restore logic should expect to install the .apk before extracting the data. System packages can designate their own full backup agents. This is to manage things like the settings provider which (a) cannot be shut down on the fly in order to do a clean snapshot of their file trees, and (b) manage data that is not only irrelevant but actively hostile to non-identical devices -- CDMA telephony settings would seriously mess up a GSM device if emplaced there blind, for example. When a full backup or restore is initiated from adb, the system will present a confirmation UI that the user must explicitly respond to within a short [~ 30 seconds] timeout. This is to avoid the possibility of malicious desktop-side software secretly grabbing a copy of all the user's data for nefarious purposes. (*) The backup is not strictly a full mirror. In particular, the settings database is not cloned; it is handled the same way that it is in cloud backup/restore. This is because some settings are actively destructive if cloned onto a different (or especially a different-model) device: telephony settings and AndroidID are good examples of this. (**) On the framework side it doesn't care that it's adb; it just sends the tar stream to a file descriptor. This can easily be retargeted around whatever transport we might decide to use in the future. KNOWN ISSUES: * the security UI is desperately ugly; no proper designs have yet been done for it * restore is not yet implemented * shared storage backup is not yet implemented * symlinks aren't yet handled, though some infrastructure for dealing with them has been put in place. Change-Id: Ia8347611e23b398af36ea22c36dff0a276b1ce91
2011-04-01 21:43:32 +00:00
static const bool DEBUG = false;
/*
* File Format (v1):
*
* All ints are stored little-endian.
*
* - An app_header_v1 struct.
* - The name of the package, utf-8, null terminated, padded to 4-byte boundary.
* - A sequence of zero or more key/value paires (entities), each with
* - A entity_header_v1 struct
* - The key, utf-8, null terminated, padded to 4-byte boundary.
* - The value, padded to 4 byte boundary
*/
const static int ROUND_UP[4] = { 0, 3, 2, 1 };
static inline size_t
round_up(size_t n)
{
return n + ROUND_UP[n % 4];
}
static inline size_t
padding_extra(size_t n)
{
return ROUND_UP[n % 4];
}
BackupDataWriter::BackupDataWriter(int fd)
:m_fd(fd),
m_status(NO_ERROR),
m_pos(0),
m_entityCount(0)
{
}
BackupDataWriter::~BackupDataWriter()
{
}
// Pad out anything they've previously written to the next 4 byte boundary.
status_t
BackupDataWriter::write_padding_for(int n)
{
ssize_t amt;
ssize_t paddingSize;
paddingSize = padding_extra(n);
if (paddingSize > 0) {
uint32_t padding = 0xbcbcbcbc;
Full local backup infrastructure This is the basic infrastructure for pulling a full(*) backup of the device's data over an adb(**) connection to the local device. The basic process consists of these interacting pieces: 1. The framework's BackupManagerService, which coordinates the collection of app data and routing to the destination. 2. A new framework-provided BackupAgent implementation called FullBackupAgent, which is instantiated in the target applications' processes in turn, and knows how to emit a datastream that contains all of the app's saved data files. 3. A new shell-level program called "bu" that is used to bridge from adb to the framework's Backup Manager. 4. adb itself, which now knows how to use 'bu' to kick off a backup operation and pull the resulting data stream to the desktop host. 5. A system-provided application that verifies with the user that an attempted backup/restore operation is in fact expected and to be allowed. The full agent implementation is not used during normal operation of the delta-based app-customized remote backup process. Instead it's used during user-confirmed *full* backup of applications and all their data to a local destination, e.g. via the adb connection. The output format is 'tar'. This makes it very easy for the end user to examine the resulting dataset, e.g. for purpose of extracting files for debug purposes; as well as making it easy to contemplate adding things like a direct gzip stage to the data pipeline during backup/restore. It also makes it convenient to construct and maintain synthetic backup datasets for testing purposes. Within the tar format, certain artificial conventions are used. All files are stored within top-level directories according to their semantic origin: apps/pkgname/a/ : Application .apk file itself apps/pkgname/obb/: The application's associated .obb containers apps/pkgname/f/ : The subtree rooted at the getFilesDir() location apps/pkgname/db/ : The subtree rooted at the getDatabasePath() parent apps/pkgname/sp/ : The subtree rooted at the getSharedPrefsFile() parent apps/pkgname/r/ : Files stored relative to the root of the app's file tree apps/pkgname/c/ : Reserved for the app's getCacheDir() tree; not stored. For each package, the first entry in the tar stream is a file called "_manifest", nominally rooted at apps/pkgname. This file contains some metadata about the package whose data is stored in the archive. The contents of shared storage can optionally be included in the tar stream. It is placed in the synthetic location: shared/... uid/gid are ignored; app uids are assigned at install time, and the app's data is handled from within its own execution environment, so will automatically have the app's correct uid. Forward-locked .apk files are never backed up. System-partition .apk files are not backed up unless they have been overridden by a post-factory upgrade, in which case the current .apk *is* backed up -- i.e. the .apk that matches the on-disk data. The manifest preceding each application's portion of the tar stream provides version numbers and signature blocks for version checking, as well as an indication of whether the restore logic should expect to install the .apk before extracting the data. System packages can designate their own full backup agents. This is to manage things like the settings provider which (a) cannot be shut down on the fly in order to do a clean snapshot of their file trees, and (b) manage data that is not only irrelevant but actively hostile to non-identical devices -- CDMA telephony settings would seriously mess up a GSM device if emplaced there blind, for example. When a full backup or restore is initiated from adb, the system will present a confirmation UI that the user must explicitly respond to within a short [~ 30 seconds] timeout. This is to avoid the possibility of malicious desktop-side software secretly grabbing a copy of all the user's data for nefarious purposes. (*) The backup is not strictly a full mirror. In particular, the settings database is not cloned; it is handled the same way that it is in cloud backup/restore. This is because some settings are actively destructive if cloned onto a different (or especially a different-model) device: telephony settings and AndroidID are good examples of this. (**) On the framework side it doesn't care that it's adb; it just sends the tar stream to a file descriptor. This can easily be retargeted around whatever transport we might decide to use in the future. KNOWN ISSUES: * the security UI is desperately ugly; no proper designs have yet been done for it * restore is not yet implemented * shared storage backup is not yet implemented * symlinks aren't yet handled, though some infrastructure for dealing with them has been put in place. Change-Id: Ia8347611e23b398af36ea22c36dff0a276b1ce91
2011-04-01 21:43:32 +00:00
if (DEBUG) LOGI("writing %d padding bytes for %d", paddingSize, n);
amt = write(m_fd, &padding, paddingSize);
if (amt != paddingSize) {
m_status = errno;
return m_status;
}
m_pos += amt;
}
return NO_ERROR;
}
status_t
BackupDataWriter::WriteEntityHeader(const String8& key, size_t dataSize)
{
if (m_status != NO_ERROR) {
return m_status;
}
ssize_t amt;
amt = write_padding_for(m_pos);
if (amt != 0) {
return amt;
}
String8 k;
if (m_keyPrefix.length() > 0) {
k = m_keyPrefix;
k += ":";
k += key;
} else {
k = key;
}
Full local backup infrastructure This is the basic infrastructure for pulling a full(*) backup of the device's data over an adb(**) connection to the local device. The basic process consists of these interacting pieces: 1. The framework's BackupManagerService, which coordinates the collection of app data and routing to the destination. 2. A new framework-provided BackupAgent implementation called FullBackupAgent, which is instantiated in the target applications' processes in turn, and knows how to emit a datastream that contains all of the app's saved data files. 3. A new shell-level program called "bu" that is used to bridge from adb to the framework's Backup Manager. 4. adb itself, which now knows how to use 'bu' to kick off a backup operation and pull the resulting data stream to the desktop host. 5. A system-provided application that verifies with the user that an attempted backup/restore operation is in fact expected and to be allowed. The full agent implementation is not used during normal operation of the delta-based app-customized remote backup process. Instead it's used during user-confirmed *full* backup of applications and all their data to a local destination, e.g. via the adb connection. The output format is 'tar'. This makes it very easy for the end user to examine the resulting dataset, e.g. for purpose of extracting files for debug purposes; as well as making it easy to contemplate adding things like a direct gzip stage to the data pipeline during backup/restore. It also makes it convenient to construct and maintain synthetic backup datasets for testing purposes. Within the tar format, certain artificial conventions are used. All files are stored within top-level directories according to their semantic origin: apps/pkgname/a/ : Application .apk file itself apps/pkgname/obb/: The application's associated .obb containers apps/pkgname/f/ : The subtree rooted at the getFilesDir() location apps/pkgname/db/ : The subtree rooted at the getDatabasePath() parent apps/pkgname/sp/ : The subtree rooted at the getSharedPrefsFile() parent apps/pkgname/r/ : Files stored relative to the root of the app's file tree apps/pkgname/c/ : Reserved for the app's getCacheDir() tree; not stored. For each package, the first entry in the tar stream is a file called "_manifest", nominally rooted at apps/pkgname. This file contains some metadata about the package whose data is stored in the archive. The contents of shared storage can optionally be included in the tar stream. It is placed in the synthetic location: shared/... uid/gid are ignored; app uids are assigned at install time, and the app's data is handled from within its own execution environment, so will automatically have the app's correct uid. Forward-locked .apk files are never backed up. System-partition .apk files are not backed up unless they have been overridden by a post-factory upgrade, in which case the current .apk *is* backed up -- i.e. the .apk that matches the on-disk data. The manifest preceding each application's portion of the tar stream provides version numbers and signature blocks for version checking, as well as an indication of whether the restore logic should expect to install the .apk before extracting the data. System packages can designate their own full backup agents. This is to manage things like the settings provider which (a) cannot be shut down on the fly in order to do a clean snapshot of their file trees, and (b) manage data that is not only irrelevant but actively hostile to non-identical devices -- CDMA telephony settings would seriously mess up a GSM device if emplaced there blind, for example. When a full backup or restore is initiated from adb, the system will present a confirmation UI that the user must explicitly respond to within a short [~ 30 seconds] timeout. This is to avoid the possibility of malicious desktop-side software secretly grabbing a copy of all the user's data for nefarious purposes. (*) The backup is not strictly a full mirror. In particular, the settings database is not cloned; it is handled the same way that it is in cloud backup/restore. This is because some settings are actively destructive if cloned onto a different (or especially a different-model) device: telephony settings and AndroidID are good examples of this. (**) On the framework side it doesn't care that it's adb; it just sends the tar stream to a file descriptor. This can easily be retargeted around whatever transport we might decide to use in the future. KNOWN ISSUES: * the security UI is desperately ugly; no proper designs have yet been done for it * restore is not yet implemented * shared storage backup is not yet implemented * symlinks aren't yet handled, though some infrastructure for dealing with them has been put in place. Change-Id: Ia8347611e23b398af36ea22c36dff0a276b1ce91
2011-04-01 21:43:32 +00:00
if (DEBUG) {
LOGD("Writing header: prefix='%s' key='%s' dataSize=%d", m_keyPrefix.string(), key.string(),
2009-06-26 21:19:11 +00:00
dataSize);
}
entity_header_v1 header;
ssize_t keyLen;
keyLen = k.length();
header.type = tolel(BACKUP_HEADER_ENTITY_V1);
header.keyLen = tolel(keyLen);
header.dataSize = tolel(dataSize);
Full local backup infrastructure This is the basic infrastructure for pulling a full(*) backup of the device's data over an adb(**) connection to the local device. The basic process consists of these interacting pieces: 1. The framework's BackupManagerService, which coordinates the collection of app data and routing to the destination. 2. A new framework-provided BackupAgent implementation called FullBackupAgent, which is instantiated in the target applications' processes in turn, and knows how to emit a datastream that contains all of the app's saved data files. 3. A new shell-level program called "bu" that is used to bridge from adb to the framework's Backup Manager. 4. adb itself, which now knows how to use 'bu' to kick off a backup operation and pull the resulting data stream to the desktop host. 5. A system-provided application that verifies with the user that an attempted backup/restore operation is in fact expected and to be allowed. The full agent implementation is not used during normal operation of the delta-based app-customized remote backup process. Instead it's used during user-confirmed *full* backup of applications and all their data to a local destination, e.g. via the adb connection. The output format is 'tar'. This makes it very easy for the end user to examine the resulting dataset, e.g. for purpose of extracting files for debug purposes; as well as making it easy to contemplate adding things like a direct gzip stage to the data pipeline during backup/restore. It also makes it convenient to construct and maintain synthetic backup datasets for testing purposes. Within the tar format, certain artificial conventions are used. All files are stored within top-level directories according to their semantic origin: apps/pkgname/a/ : Application .apk file itself apps/pkgname/obb/: The application's associated .obb containers apps/pkgname/f/ : The subtree rooted at the getFilesDir() location apps/pkgname/db/ : The subtree rooted at the getDatabasePath() parent apps/pkgname/sp/ : The subtree rooted at the getSharedPrefsFile() parent apps/pkgname/r/ : Files stored relative to the root of the app's file tree apps/pkgname/c/ : Reserved for the app's getCacheDir() tree; not stored. For each package, the first entry in the tar stream is a file called "_manifest", nominally rooted at apps/pkgname. This file contains some metadata about the package whose data is stored in the archive. The contents of shared storage can optionally be included in the tar stream. It is placed in the synthetic location: shared/... uid/gid are ignored; app uids are assigned at install time, and the app's data is handled from within its own execution environment, so will automatically have the app's correct uid. Forward-locked .apk files are never backed up. System-partition .apk files are not backed up unless they have been overridden by a post-factory upgrade, in which case the current .apk *is* backed up -- i.e. the .apk that matches the on-disk data. The manifest preceding each application's portion of the tar stream provides version numbers and signature blocks for version checking, as well as an indication of whether the restore logic should expect to install the .apk before extracting the data. System packages can designate their own full backup agents. This is to manage things like the settings provider which (a) cannot be shut down on the fly in order to do a clean snapshot of their file trees, and (b) manage data that is not only irrelevant but actively hostile to non-identical devices -- CDMA telephony settings would seriously mess up a GSM device if emplaced there blind, for example. When a full backup or restore is initiated from adb, the system will present a confirmation UI that the user must explicitly respond to within a short [~ 30 seconds] timeout. This is to avoid the possibility of malicious desktop-side software secretly grabbing a copy of all the user's data for nefarious purposes. (*) The backup is not strictly a full mirror. In particular, the settings database is not cloned; it is handled the same way that it is in cloud backup/restore. This is because some settings are actively destructive if cloned onto a different (or especially a different-model) device: telephony settings and AndroidID are good examples of this. (**) On the framework side it doesn't care that it's adb; it just sends the tar stream to a file descriptor. This can easily be retargeted around whatever transport we might decide to use in the future. KNOWN ISSUES: * the security UI is desperately ugly; no proper designs have yet been done for it * restore is not yet implemented * shared storage backup is not yet implemented * symlinks aren't yet handled, though some infrastructure for dealing with them has been put in place. Change-Id: Ia8347611e23b398af36ea22c36dff0a276b1ce91
2011-04-01 21:43:32 +00:00
if (DEBUG) LOGI("writing entity header, %d bytes", sizeof(entity_header_v1));
amt = write(m_fd, &header, sizeof(entity_header_v1));
if (amt != sizeof(entity_header_v1)) {
m_status = errno;
return m_status;
}
m_pos += amt;
Full local backup infrastructure This is the basic infrastructure for pulling a full(*) backup of the device's data over an adb(**) connection to the local device. The basic process consists of these interacting pieces: 1. The framework's BackupManagerService, which coordinates the collection of app data and routing to the destination. 2. A new framework-provided BackupAgent implementation called FullBackupAgent, which is instantiated in the target applications' processes in turn, and knows how to emit a datastream that contains all of the app's saved data files. 3. A new shell-level program called "bu" that is used to bridge from adb to the framework's Backup Manager. 4. adb itself, which now knows how to use 'bu' to kick off a backup operation and pull the resulting data stream to the desktop host. 5. A system-provided application that verifies with the user that an attempted backup/restore operation is in fact expected and to be allowed. The full agent implementation is not used during normal operation of the delta-based app-customized remote backup process. Instead it's used during user-confirmed *full* backup of applications and all their data to a local destination, e.g. via the adb connection. The output format is 'tar'. This makes it very easy for the end user to examine the resulting dataset, e.g. for purpose of extracting files for debug purposes; as well as making it easy to contemplate adding things like a direct gzip stage to the data pipeline during backup/restore. It also makes it convenient to construct and maintain synthetic backup datasets for testing purposes. Within the tar format, certain artificial conventions are used. All files are stored within top-level directories according to their semantic origin: apps/pkgname/a/ : Application .apk file itself apps/pkgname/obb/: The application's associated .obb containers apps/pkgname/f/ : The subtree rooted at the getFilesDir() location apps/pkgname/db/ : The subtree rooted at the getDatabasePath() parent apps/pkgname/sp/ : The subtree rooted at the getSharedPrefsFile() parent apps/pkgname/r/ : Files stored relative to the root of the app's file tree apps/pkgname/c/ : Reserved for the app's getCacheDir() tree; not stored. For each package, the first entry in the tar stream is a file called "_manifest", nominally rooted at apps/pkgname. This file contains some metadata about the package whose data is stored in the archive. The contents of shared storage can optionally be included in the tar stream. It is placed in the synthetic location: shared/... uid/gid are ignored; app uids are assigned at install time, and the app's data is handled from within its own execution environment, so will automatically have the app's correct uid. Forward-locked .apk files are never backed up. System-partition .apk files are not backed up unless they have been overridden by a post-factory upgrade, in which case the current .apk *is* backed up -- i.e. the .apk that matches the on-disk data. The manifest preceding each application's portion of the tar stream provides version numbers and signature blocks for version checking, as well as an indication of whether the restore logic should expect to install the .apk before extracting the data. System packages can designate their own full backup agents. This is to manage things like the settings provider which (a) cannot be shut down on the fly in order to do a clean snapshot of their file trees, and (b) manage data that is not only irrelevant but actively hostile to non-identical devices -- CDMA telephony settings would seriously mess up a GSM device if emplaced there blind, for example. When a full backup or restore is initiated from adb, the system will present a confirmation UI that the user must explicitly respond to within a short [~ 30 seconds] timeout. This is to avoid the possibility of malicious desktop-side software secretly grabbing a copy of all the user's data for nefarious purposes. (*) The backup is not strictly a full mirror. In particular, the settings database is not cloned; it is handled the same way that it is in cloud backup/restore. This is because some settings are actively destructive if cloned onto a different (or especially a different-model) device: telephony settings and AndroidID are good examples of this. (**) On the framework side it doesn't care that it's adb; it just sends the tar stream to a file descriptor. This can easily be retargeted around whatever transport we might decide to use in the future. KNOWN ISSUES: * the security UI is desperately ugly; no proper designs have yet been done for it * restore is not yet implemented * shared storage backup is not yet implemented * symlinks aren't yet handled, though some infrastructure for dealing with them has been put in place. Change-Id: Ia8347611e23b398af36ea22c36dff0a276b1ce91
2011-04-01 21:43:32 +00:00
if (DEBUG) LOGI("writing entity header key, %d bytes", keyLen+1);
amt = write(m_fd, k.string(), keyLen+1);
if (amt != keyLen+1) {
m_status = errno;
return m_status;
}
m_pos += amt;
amt = write_padding_for(keyLen+1);
m_entityCount++;
return amt;
}
status_t
BackupDataWriter::WriteEntityData(const void* data, size_t size)
{
Full local backup infrastructure This is the basic infrastructure for pulling a full(*) backup of the device's data over an adb(**) connection to the local device. The basic process consists of these interacting pieces: 1. The framework's BackupManagerService, which coordinates the collection of app data and routing to the destination. 2. A new framework-provided BackupAgent implementation called FullBackupAgent, which is instantiated in the target applications' processes in turn, and knows how to emit a datastream that contains all of the app's saved data files. 3. A new shell-level program called "bu" that is used to bridge from adb to the framework's Backup Manager. 4. adb itself, which now knows how to use 'bu' to kick off a backup operation and pull the resulting data stream to the desktop host. 5. A system-provided application that verifies with the user that an attempted backup/restore operation is in fact expected and to be allowed. The full agent implementation is not used during normal operation of the delta-based app-customized remote backup process. Instead it's used during user-confirmed *full* backup of applications and all their data to a local destination, e.g. via the adb connection. The output format is 'tar'. This makes it very easy for the end user to examine the resulting dataset, e.g. for purpose of extracting files for debug purposes; as well as making it easy to contemplate adding things like a direct gzip stage to the data pipeline during backup/restore. It also makes it convenient to construct and maintain synthetic backup datasets for testing purposes. Within the tar format, certain artificial conventions are used. All files are stored within top-level directories according to their semantic origin: apps/pkgname/a/ : Application .apk file itself apps/pkgname/obb/: The application's associated .obb containers apps/pkgname/f/ : The subtree rooted at the getFilesDir() location apps/pkgname/db/ : The subtree rooted at the getDatabasePath() parent apps/pkgname/sp/ : The subtree rooted at the getSharedPrefsFile() parent apps/pkgname/r/ : Files stored relative to the root of the app's file tree apps/pkgname/c/ : Reserved for the app's getCacheDir() tree; not stored. For each package, the first entry in the tar stream is a file called "_manifest", nominally rooted at apps/pkgname. This file contains some metadata about the package whose data is stored in the archive. The contents of shared storage can optionally be included in the tar stream. It is placed in the synthetic location: shared/... uid/gid are ignored; app uids are assigned at install time, and the app's data is handled from within its own execution environment, so will automatically have the app's correct uid. Forward-locked .apk files are never backed up. System-partition .apk files are not backed up unless they have been overridden by a post-factory upgrade, in which case the current .apk *is* backed up -- i.e. the .apk that matches the on-disk data. The manifest preceding each application's portion of the tar stream provides version numbers and signature blocks for version checking, as well as an indication of whether the restore logic should expect to install the .apk before extracting the data. System packages can designate their own full backup agents. This is to manage things like the settings provider which (a) cannot be shut down on the fly in order to do a clean snapshot of their file trees, and (b) manage data that is not only irrelevant but actively hostile to non-identical devices -- CDMA telephony settings would seriously mess up a GSM device if emplaced there blind, for example. When a full backup or restore is initiated from adb, the system will present a confirmation UI that the user must explicitly respond to within a short [~ 30 seconds] timeout. This is to avoid the possibility of malicious desktop-side software secretly grabbing a copy of all the user's data for nefarious purposes. (*) The backup is not strictly a full mirror. In particular, the settings database is not cloned; it is handled the same way that it is in cloud backup/restore. This is because some settings are actively destructive if cloned onto a different (or especially a different-model) device: telephony settings and AndroidID are good examples of this. (**) On the framework side it doesn't care that it's adb; it just sends the tar stream to a file descriptor. This can easily be retargeted around whatever transport we might decide to use in the future. KNOWN ISSUES: * the security UI is desperately ugly; no proper designs have yet been done for it * restore is not yet implemented * shared storage backup is not yet implemented * symlinks aren't yet handled, though some infrastructure for dealing with them has been put in place. Change-Id: Ia8347611e23b398af36ea22c36dff0a276b1ce91
2011-04-01 21:43:32 +00:00
if (DEBUG) LOGD("Writing data: size=%lu", (unsigned long) size);
if (m_status != NO_ERROR) {
Full local backup infrastructure This is the basic infrastructure for pulling a full(*) backup of the device's data over an adb(**) connection to the local device. The basic process consists of these interacting pieces: 1. The framework's BackupManagerService, which coordinates the collection of app data and routing to the destination. 2. A new framework-provided BackupAgent implementation called FullBackupAgent, which is instantiated in the target applications' processes in turn, and knows how to emit a datastream that contains all of the app's saved data files. 3. A new shell-level program called "bu" that is used to bridge from adb to the framework's Backup Manager. 4. adb itself, which now knows how to use 'bu' to kick off a backup operation and pull the resulting data stream to the desktop host. 5. A system-provided application that verifies with the user that an attempted backup/restore operation is in fact expected and to be allowed. The full agent implementation is not used during normal operation of the delta-based app-customized remote backup process. Instead it's used during user-confirmed *full* backup of applications and all their data to a local destination, e.g. via the adb connection. The output format is 'tar'. This makes it very easy for the end user to examine the resulting dataset, e.g. for purpose of extracting files for debug purposes; as well as making it easy to contemplate adding things like a direct gzip stage to the data pipeline during backup/restore. It also makes it convenient to construct and maintain synthetic backup datasets for testing purposes. Within the tar format, certain artificial conventions are used. All files are stored within top-level directories according to their semantic origin: apps/pkgname/a/ : Application .apk file itself apps/pkgname/obb/: The application's associated .obb containers apps/pkgname/f/ : The subtree rooted at the getFilesDir() location apps/pkgname/db/ : The subtree rooted at the getDatabasePath() parent apps/pkgname/sp/ : The subtree rooted at the getSharedPrefsFile() parent apps/pkgname/r/ : Files stored relative to the root of the app's file tree apps/pkgname/c/ : Reserved for the app's getCacheDir() tree; not stored. For each package, the first entry in the tar stream is a file called "_manifest", nominally rooted at apps/pkgname. This file contains some metadata about the package whose data is stored in the archive. The contents of shared storage can optionally be included in the tar stream. It is placed in the synthetic location: shared/... uid/gid are ignored; app uids are assigned at install time, and the app's data is handled from within its own execution environment, so will automatically have the app's correct uid. Forward-locked .apk files are never backed up. System-partition .apk files are not backed up unless they have been overridden by a post-factory upgrade, in which case the current .apk *is* backed up -- i.e. the .apk that matches the on-disk data. The manifest preceding each application's portion of the tar stream provides version numbers and signature blocks for version checking, as well as an indication of whether the restore logic should expect to install the .apk before extracting the data. System packages can designate their own full backup agents. This is to manage things like the settings provider which (a) cannot be shut down on the fly in order to do a clean snapshot of their file trees, and (b) manage data that is not only irrelevant but actively hostile to non-identical devices -- CDMA telephony settings would seriously mess up a GSM device if emplaced there blind, for example. When a full backup or restore is initiated from adb, the system will present a confirmation UI that the user must explicitly respond to within a short [~ 30 seconds] timeout. This is to avoid the possibility of malicious desktop-side software secretly grabbing a copy of all the user's data for nefarious purposes. (*) The backup is not strictly a full mirror. In particular, the settings database is not cloned; it is handled the same way that it is in cloud backup/restore. This is because some settings are actively destructive if cloned onto a different (or especially a different-model) device: telephony settings and AndroidID are good examples of this. (**) On the framework side it doesn't care that it's adb; it just sends the tar stream to a file descriptor. This can easily be retargeted around whatever transport we might decide to use in the future. KNOWN ISSUES: * the security UI is desperately ugly; no proper designs have yet been done for it * restore is not yet implemented * shared storage backup is not yet implemented * symlinks aren't yet handled, though some infrastructure for dealing with them has been put in place. Change-Id: Ia8347611e23b398af36ea22c36dff0a276b1ce91
2011-04-01 21:43:32 +00:00
if (DEBUG) {
LOGD("Not writing data - stream in error state %d (%s)", m_status, strerror(m_status));
}
return m_status;
}
// We don't write padding here, because they're allowed to call this several
// times with smaller buffers. We write it at the end of WriteEntityHeader
// instead.
ssize_t amt = write(m_fd, data, size);
if (amt != (ssize_t)size) {
m_status = errno;
Full local backup infrastructure This is the basic infrastructure for pulling a full(*) backup of the device's data over an adb(**) connection to the local device. The basic process consists of these interacting pieces: 1. The framework's BackupManagerService, which coordinates the collection of app data and routing to the destination. 2. A new framework-provided BackupAgent implementation called FullBackupAgent, which is instantiated in the target applications' processes in turn, and knows how to emit a datastream that contains all of the app's saved data files. 3. A new shell-level program called "bu" that is used to bridge from adb to the framework's Backup Manager. 4. adb itself, which now knows how to use 'bu' to kick off a backup operation and pull the resulting data stream to the desktop host. 5. A system-provided application that verifies with the user that an attempted backup/restore operation is in fact expected and to be allowed. The full agent implementation is not used during normal operation of the delta-based app-customized remote backup process. Instead it's used during user-confirmed *full* backup of applications and all their data to a local destination, e.g. via the adb connection. The output format is 'tar'. This makes it very easy for the end user to examine the resulting dataset, e.g. for purpose of extracting files for debug purposes; as well as making it easy to contemplate adding things like a direct gzip stage to the data pipeline during backup/restore. It also makes it convenient to construct and maintain synthetic backup datasets for testing purposes. Within the tar format, certain artificial conventions are used. All files are stored within top-level directories according to their semantic origin: apps/pkgname/a/ : Application .apk file itself apps/pkgname/obb/: The application's associated .obb containers apps/pkgname/f/ : The subtree rooted at the getFilesDir() location apps/pkgname/db/ : The subtree rooted at the getDatabasePath() parent apps/pkgname/sp/ : The subtree rooted at the getSharedPrefsFile() parent apps/pkgname/r/ : Files stored relative to the root of the app's file tree apps/pkgname/c/ : Reserved for the app's getCacheDir() tree; not stored. For each package, the first entry in the tar stream is a file called "_manifest", nominally rooted at apps/pkgname. This file contains some metadata about the package whose data is stored in the archive. The contents of shared storage can optionally be included in the tar stream. It is placed in the synthetic location: shared/... uid/gid are ignored; app uids are assigned at install time, and the app's data is handled from within its own execution environment, so will automatically have the app's correct uid. Forward-locked .apk files are never backed up. System-partition .apk files are not backed up unless they have been overridden by a post-factory upgrade, in which case the current .apk *is* backed up -- i.e. the .apk that matches the on-disk data. The manifest preceding each application's portion of the tar stream provides version numbers and signature blocks for version checking, as well as an indication of whether the restore logic should expect to install the .apk before extracting the data. System packages can designate their own full backup agents. This is to manage things like the settings provider which (a) cannot be shut down on the fly in order to do a clean snapshot of their file trees, and (b) manage data that is not only irrelevant but actively hostile to non-identical devices -- CDMA telephony settings would seriously mess up a GSM device if emplaced there blind, for example. When a full backup or restore is initiated from adb, the system will present a confirmation UI that the user must explicitly respond to within a short [~ 30 seconds] timeout. This is to avoid the possibility of malicious desktop-side software secretly grabbing a copy of all the user's data for nefarious purposes. (*) The backup is not strictly a full mirror. In particular, the settings database is not cloned; it is handled the same way that it is in cloud backup/restore. This is because some settings are actively destructive if cloned onto a different (or especially a different-model) device: telephony settings and AndroidID are good examples of this. (**) On the framework side it doesn't care that it's adb; it just sends the tar stream to a file descriptor. This can easily be retargeted around whatever transport we might decide to use in the future. KNOWN ISSUES: * the security UI is desperately ugly; no proper designs have yet been done for it * restore is not yet implemented * shared storage backup is not yet implemented * symlinks aren't yet handled, though some infrastructure for dealing with them has been put in place. Change-Id: Ia8347611e23b398af36ea22c36dff0a276b1ce91
2011-04-01 21:43:32 +00:00
if (DEBUG) LOGD("write returned error %d (%s)", m_status, strerror(m_status));
return m_status;
}
m_pos += amt;
return NO_ERROR;
}
void
BackupDataWriter::SetKeyPrefix(const String8& keyPrefix)
{
m_keyPrefix = keyPrefix;
}
BackupDataReader::BackupDataReader(int fd)
:m_fd(fd),
m_done(false),
m_status(NO_ERROR),
m_pos(0),
m_entityCount(0)
{
memset(&m_header, 0, sizeof(m_header));
}
BackupDataReader::~BackupDataReader()
{
}
status_t
BackupDataReader::Status()
{
return m_status;
}
#define CHECK_SIZE(actual, expected) \
do { \
if ((actual) != (expected)) { \
if ((actual) == 0) { \
m_status = EIO; \
m_done = true; \
} else { \
m_status = errno; \
LOGD("CHECK_SIZE(a=%ld e=%ld) failed at line %d m_status='%s'", \
long(actual), long(expected), __LINE__, strerror(m_status)); \
} \
return m_status; \
} \
} while(0)
#define SKIP_PADDING() \
do { \
status_t err = skip_padding(); \
if (err != NO_ERROR) { \
LOGD("SKIP_PADDING FAILED at line %d", __LINE__); \
m_status = err; \
return err; \
} \
} while(0)
status_t
BackupDataReader::ReadNextHeader(bool* done, int* type)
{
*done = m_done;
if (m_status != NO_ERROR) {
return m_status;
}
int amt;
amt = skip_padding();
if (amt == EIO) {
*done = m_done = true;
return NO_ERROR;
}
else if (amt != NO_ERROR) {
return amt;
}
amt = read(m_fd, &m_header, sizeof(m_header));
*done = m_done = (amt == 0);
if (*done) {
return NO_ERROR;
}
CHECK_SIZE(amt, sizeof(m_header));
2009-06-19 01:23:43 +00:00
m_pos += sizeof(m_header);
if (type) {
*type = m_header.type;
}
// validate and fix up the fields.
m_header.type = fromlel(m_header.type);
switch (m_header.type)
{
case BACKUP_HEADER_ENTITY_V1:
2009-06-19 01:23:43 +00:00
{
m_header.entity.keyLen = fromlel(m_header.entity.keyLen);
if (m_header.entity.keyLen <= 0) {
LOGD("Entity header at %d has keyLen<=0: 0x%08x\n", (int)m_pos,
(int)m_header.entity.keyLen);
m_status = EINVAL;
}
m_header.entity.dataSize = fromlel(m_header.entity.dataSize);
m_entityCount++;
2009-06-19 01:23:43 +00:00
// read the rest of the header (filename)
size_t size = m_header.entity.keyLen;
char* buf = m_key.lockBuffer(size);
if (buf == NULL) {
m_status = ENOMEM;
return m_status;
}
int amt = read(m_fd, buf, size+1);
CHECK_SIZE(amt, (int)size+1);
m_key.unlockBuffer(size);
m_pos += size+1;
SKIP_PADDING();
m_dataEndPos = m_pos + m_header.entity.dataSize;
break;
2009-06-19 01:23:43 +00:00
}
default:
LOGD("Chunk header at %d has invalid type: 0x%08x",
(int)(m_pos - sizeof(m_header)), (int)m_header.type);
m_status = EINVAL;
}
return m_status;
}
bool
BackupDataReader::HasEntities()
{
return m_status == NO_ERROR && m_header.type == BACKUP_HEADER_ENTITY_V1;
}
status_t
BackupDataReader::ReadEntityHeader(String8* key, size_t* dataSize)
{
if (m_status != NO_ERROR) {
return m_status;
}
if (m_header.type != BACKUP_HEADER_ENTITY_V1) {
return EINVAL;
}
2009-06-19 01:23:43 +00:00
*key = m_key;
*dataSize = m_header.entity.dataSize;
return NO_ERROR;
}
status_t
BackupDataReader::SkipEntityData()
{
if (m_status != NO_ERROR) {
return m_status;
}
if (m_header.type != BACKUP_HEADER_ENTITY_V1) {
return EINVAL;
}
if (m_header.entity.dataSize > 0) {
int pos = lseek(m_fd, m_dataEndPos, SEEK_SET);
if (pos == -1) {
return errno;
}
}
SKIP_PADDING();
return NO_ERROR;
}
ssize_t
BackupDataReader::ReadEntityData(void* data, size_t size)
{
if (m_status != NO_ERROR) {
2009-06-19 01:23:43 +00:00
return -1;
}
int remaining = m_dataEndPos - m_pos;
//LOGD("ReadEntityData size=%d m_pos=0x%x m_dataEndPos=0x%x remaining=%d\n",
// size, m_pos, m_dataEndPos, remaining);
if (remaining <= 0) {
return 0;
}
if (((int)size) > remaining) {
2009-06-19 01:23:43 +00:00
size = remaining;
}
//LOGD(" reading %d bytes", size);
int amt = read(m_fd, data, size);
2009-06-19 01:23:43 +00:00
if (amt < 0) {
m_status = errno;
return -1;
}
if (amt == 0) {
m_status = EIO;
m_done = true;
}
2009-06-19 01:23:43 +00:00
m_pos += amt;
return amt;
}
status_t
BackupDataReader::skip_padding()
{
ssize_t amt;
ssize_t paddingSize;
paddingSize = padding_extra(m_pos);
if (paddingSize > 0) {
uint32_t padding;
amt = read(m_fd, &padding, paddingSize);
CHECK_SIZE(amt, paddingSize);
m_pos += amt;
}
return NO_ERROR;
}
} // namespace android