Introduction

Research on mobile computing has made significant progress in adapting applications for viewing multimedia content on mobile devices [5,8,21]. Multimedia authoring and collaborative work on these platforms remain, however, open problems.

We identify three factors that hinder multimedia authoring and collaborative work over bandwidth-limited links:

1.

Read adaptations. The adaptation techniques used to lower resource usage (e.g., energy, bandwidth) may result in situations where content present at the mobile device differs significantly from the versions stored at the server. Typical adaptation techniques adapt by downloading just a fraction of a multimedia document, or by trancoding content into lower-fidelity representations. Naively storing user modifications made to an adapted document may delete elements that were not present at the mobile device, or it may replace high-fidelity data with the transcoded versions sent to the mobile device (even in cases where the user did not modify the transcoded elements).

2.

Large updates. Mobile users can generate large multimedia content (e.g., photographs, drawings, audio notes) whose propagation may result in large resource expenditures or long upload latencies over a bandwidth-limited link .

3.

Conflicts. The use of optimistic replication models [12,23] allows concurrent modifications that may conflict with each other. Conflicts can occur in other circumstances as well, but low bandwidth and the possibility of frequent disconnection make their occurrence more likely.

This paper introduces adaptation-aware editing and progressive update propagation, two novel mechanisms that enable document authoring and collaborative work over bandwidth-limited links. These mechanisms extend traditional replication models to account for the fidelity level of replicated content. Both mechanisms decompose multimedia documents into their component structure (e.g., pages, images, sounds, video), and keep track of consistency and fidelity at a component granularity. Adaptation-aware editing enables editing adapted documents by differentiating between modifications made by the user and those that result from adaptation. Progressive update propagation reduces the time and the resources required to propagate components created or modified at the bandwidth-limited device by transmitting subsets of the modified components or transcoded versions of those modifications. Adaptation-aware editing and progressive update propagation also reduce the likelihood of update conflicts in two ways. First, by working at the component level rather than the whole-document level, they reduce the sharing granularity. Second, because both mechanisms lower the cost to download and upload component data, they encourage more frequent communication, hence increasing the awareness that users have of their collaborators' activities [3].

By reducing the cost of propagating multimedia content, adaptation-aware editing and progressive update propagation enable new types of applications and extend the reach of existing applications into the mobile realm. The following two examples illustrate the use of both mechanisms:

4.

Maintenance. A work crew inspects damage to a plant caused by an explosion. They use a digital camera to take pictures of the problem area, and send the pictures over a wireless connection to the head office. Since bandwidth is low, and they want an urgent assessment of the seriousness of the situation, they use progressive update propagation to initially send low-resolution versions of the pictures. These initial images allow the head office to determine quickly that there is no need to declare an emergency, but that repair work nonetheless needs to be started immediately. The crew continues to use progressive update propagation to send higher-resolution versions of the pictures, sufficiently detailed to initiate repairs. The head office forwards these pictures to a trusted contractor and to the insurance company. The contractor uses adaptation-aware editing to indicate the suggested repairs on the pictures, and sends the marked-up pictures back to the head office and the insurance company. Both approve the repairs, and the contractor heads out to the site. When the work crew arrives back at the office, full-resolution pictures are saved for later investigation.

5.

Collaborative presentation design. A team member on a mobile device takes advantage of adaptation-aware editing to reduce download time by downloading and editing an adapted version of a presentation. The adapted document consists of just a few slides of the original presentation and has low-fidelity images, sounds, and videos. The team member then uses progressive update propagation to share her modifications to the presentation, which include a photograph taken with a digital camera. Progressive update propagation reduces the time for uploading the photograph by sending a low-fidelity version of the image. When the team member reconnects over a high-bandwidth link, the system automatically upgrades the version of the photograph.

The previous scenarios cannot be handled by current adaptation systems that only handle adaptation of read-only content. They also cannot be supported by current replication systems. Propagating transcoded versions of components as described in the above examples, requires the replication model to account for the fidelity level of replicated content. Upgrading the fidelity of an image in a particular version of a document is different from creating a new version with (user) modifications to the document.

This paper shows that fidelity can be added to a replication protocol independently of the mechanisms used for concurrency control and consistency maintenance. Replication models are typically represented by state diagrams, and we follow this general paradigm. We present state diagrams that incorporate the presence of transcoded versions of components, for use with both optimistic and pessimistic replication. The introduction of transcoded component versions is orthogonal to the maintenance of consistency between replicas. More specifically, new states are added to represent transcoded versions, but the semantics of the existing states and the transitions between them remain unchanged. Therefore, fidelity can be added easily to any replication protocol, whether optimistic or pessimistic.

There are several possible implementations of adaptation-aware editing and progressive update propagation. We present a prototype implementation of these mechanisms that takes advantage of existing run-time APIs and structured document formats [5]. This implementation allows us to adapt applications for multimedia authoring and collaboration without changing their source code.

We demonstrate our implementation by experimenting with the Outlook email browser and the PowerPoint presentation software. Both applications see large reductions in user-perceived latencies. For Outlook, progressive update propagation reduces the time a wireless author has to stay connected to propagate emails with multimedia attachments. For PowerPoint, adaptation-aware editing and progressive update propagation reduce the time that wireless collaborators need to wait to view changes made to the presentation by their colleagues.

The rest of this paper is organized as follows. Section 2 introduces adaptation-aware editing and progressive update propagation and explores the implications of extending pessimistic and optimistic replication models to support these mechanisms. Sections 3 and  4 present the design and evaluation of our prototype implementation of adaptation-aware editing and progressive update propagation. Finally, Sections 5 and 6 discuss related work and conclude the paper.