Storage

Goals

• Provided a unified architecture for storage on the web.
• Enable persistent storage with the same "unlimited" quota as native.
• Enable applications to more effectively manage their storage.

Use cases

Music

User visits a music application and decides they want to sign up and store 200 music files so they are available offline as the user will travel abroad the next day. It is imperative that these songs cannot be removed without explicit consent from the user.

Games

User plays an RPG for which you need to be online. To improve the user experience the RPG wants to cache as much as possible of the user's current location, nearby locations, and further away locations in game, in decreasing order of priority. So that when the user agent requires space the locations the user is furthest away from will be wiped first. The engine is expected to be stored in persistent storage, though preferable without asking the user.

Photos & video

User wants to store photos taken on a trip abroad without connectivity without losing them. User wants to store photos and always have them available offline for sharing with others.

Social

A social network wants to temporarily cache the latest activities locally.

Infrastructure

A user agent has zero or more site storage units. A site storage unit's scope is eTLD+1. A site storage unit consists of a 'default box' and zero or more origin storage units.

An origin storage unit's cope is origin. An origin unit consists of a default box and zero or more boxes.

HTTP cookies are stored in a site storage unit's default box.

localStorage, sessionStorage, IDB, Service Workers, Cache API, History API, and Notifications API are stored in an origin storage unit's default box.

Boxes

A box is where the bytes end up. A box has an id (default box's id is the empty string) and a mode ("atomic", "cache", or "persistent"). Storage APIs that support boxes can associate themselves with a box by passing it in.

Note: going forward we might add the ability to associate UI labels with boxes so end-users can make decisions on a per-box basis as well.

// If the persistent permission is granted, you can add boxes that are persistent
navigator.storage.addBox("the-good-wife-s01", { mode: "persistent" }).then(
  box => self.caches.open("the-good-wife-s01", { box: box }) // it might be worth changing .open() to be simpler
)

// Not all APIs will support "cache" boxes, but it's a natural fit for the Cache API
// even though by default it will be "atomic"
navigator.storage.addBox("localcache", { mode: "cache" }).then(
  box => self.caches.open("localcache", { box: box })
)

// Games might want to store levels in distinct boxes so that when the user needs to
// free up space the game engine remains intact while levels not played for a while
// get cleared.
navigator.storage.addBox("game-level-1", { mode: "atomic" }).then(
  box => indexedDB.open("game-level-1", 1, { box: box }) // again, it might be worth changing .open() to be simpler
)

Atomic boxes

An "atomic" box is cleared in its entirety. User agents are encouraged to reason globally about all "atomic" boxes in such a way that boxes the user least likely needs are cleared before others without paying much heed to site or origin storage unit boundaries.

User agents could potentially reason globally about "atomic" boxes and individual items of "cache" boxes together.

Cache boxes

A "cache" box consists of a sequence of items. What exactly constitutes an item is up to the API that supports a "cache" box. User agents are encouraged to reason globally about all items in all "cache" boxes in such a way that items the user least likely needs are cleared before others.

Note: A "cache" box is distinct from an "atomic" (and "persistent") box in that it provides no all-or-nothing guarantees. It's ideal for storing resources that help improve performance but are not essential.

Persistent boxes

Through a user-mediated action an origin can acquire persistent storage. This ensures that origin storage unit's default box is persistently stored and will not be wiped without user consent. Persistence can be granted through navigator.storage.requestPersistent(), through bookmarking, or a combination of sorts. The expectation is that user agents will converge on a suitable pattern over time.

A site storage unit's default box cannot obtain persistence and will always be atomic.

Quota

The global quota is the available hard drive space minus any space the user agent wants to reserve. The non-persistent global quota is the limit of storage available within the global quota to "atomic" and "cache".

A site storage unit quota is a segment of the non-persistent global quota. User agents are free to let factors such as navigation frequency, bookmarking, and other indications of popularity impact the site storage unit quota.

An origin storage unit quota is that of its site storage unit, with the exception of its boxes that use "persistent" rather than other types of storage. "Persistent" boxes may use the global quota.

API

[NoInterfaceObject, 
 Exposed=(Window,Worker)]
interface NavigatorStorage {
  readonly attribute StorageManager storage;
};
Navigator implements NavigatorStorage;
WorkerNavigator implements NavigatorStorage;

This ensures the API is available everywhere under navigator.storage. (At some point we should probably clean up this pattern.)

[Exposed=(Window,Worker)]
interface StorageManager {
  Promise<PersistentStoragePermission> requestPersistent();
  Promise<PersistentStoragePermission> persistentPermission();

  Promise<StorageInfo]> getEstimate();
  Promise<StorageInfo> getPersistentEstimate();
  
  Promise<StorageBox> addBox(DOMString id, StorageBoxInit init);
  Promise<boolean> deleteBox(optional DOMString id = "");
  Promise<sequence<StorageBox>> boxes();
};

StorageManager provides access to getting permission for "persistent" boxes (and changing the origin storage unit's default box to use it).

getEstimate() provides access to the estimated usage and quota for the origin storage unit's segment of the non-persistent global quota.

getPersistentEstimate() provides access to the estimated usage and quota for the global quota, provided the persistent permission is "granted".

The remainder provides access to boxes. An origin storage unit's default box can be cleared by invoking deleteBox(). Invoking deleteBox() with a non-empty-string argument simply removes that box. addBox()'s first argument cannot be the empty string as it is reserved for the default box.

Note: No access is given to a site storage unit for now as it crosses the ordinary security boundary.

interface StorageBox {
   readonly attribute DOMString id;
   readonly  attribute StorageMode mode;
   readonly attribute unsigned long long usageEstimate;
};

dictionary StorageBoxInit {
  StorageMode mode = "atomic";
};

interface StorageInfo {
  readonly attribute unsigned long long usage;
  readonly attribute unsigned long long quota;
};

enum StorageMode { "atomic", "cache", "persistent" };
enum PersistentStoragePermission { "default", "denied", "granted" }; // merge with NotificationPermission?

Rollout

If this is too much at once, we can partition it as such:

• v1: Persistent permission and usage/quota estimates.
• v2: Boxes.

Existing APIs

These APIs need to be (re)written in terms of the new low-level storage primitives so that they have a common grounding.

• Cookies
• localStorage
• sessionStorage
• Indexed DB
• Cache API (from service workers, to be clear)
• History API
• Notifications API
• Appcache (legacy)
• Filesystem (legacy, might make a comeback)
• WebSQL (legacy)

HTTP cache is probably distinct.

FAQ

Why no eviction event?

There are two main reasons to avoid this:

• Does not allow for global reasoning by the user agent
• Tragedy of the commons

Also, this has been tried on other systems and they always end up with /tmp or some such. When you need space you don't want to ask a dozen applications to clear some before you can start allocating or ask the user to remove something.

What is the relationship to the Quota API?

The main difference is that requesting a specific quota from the user is not possible as making that work UX-wise is not possible. This API also puts emphasis on usage and quota being estimates.

Elsewhere

• https://github.com/w3ctag/spec-reviews/blob/master/2014/02/quota-management-api.md
• https://w3c.github.io/quota-api/
• https://github.com/w3c/quota-api/issues
• https://github.com/slightlyoff/StorageDurability/issues
• https://bugzilla.mozilla.org/show_bug.cgi?id=1147820