[css-view-transitions-2] keep ::view-transition-old(root) screenshot

[tinchox5]

I'm currently developing a Zoomable User Interface (ZUI), and one of its key points is stacking views to maintain context. You can find a basic example here.

I have always wondered about taking snapshots of previous view states, and I am confident that the View Transitions API will be perfect for performing zoomable navigation. However, I am unsure if this API offers an option to retain old screenshots or the capacity to reuse them as a background for the new view state.

Therefore, I would like to request information or guidance on how to retain old screenshots in the View Transitions API or if there is an option to reuse them as a background for the new view state. This would be essential to keep the screenshot of the old view "alive" to serve as context and to smooth the zoomable transition since previous elements are "flattened."

Thank you for your time and assistance.

[khushalsagar]

I think this is a good idea in terms of an API which reuses a subset of the concepts in ViewTransitions.

The model with ViewTransitions is the following:

1. Capture previous view state as an image.
2. Create live images from new view state.
3. Build a new stacking layer (which sits on top of the Document) to animate between old and new images (built from the view states).

Because its meant to be a transient effect, we're ok with an inert stacking layer on top of the entire Document. For this use-case, you want the new DOM to be interactive and sit on top of a snapshot of the old view state. So you don't need 2 and 3 above. Just 1 with the ability to use that image.

We talked about a primitive like this. In fact, I think its API shape can be very similar to the existing CSS element() function. Except that feature is about live images of existing DOM vs you want snapshots of previous DOM state. Since this snapshot mechanism has to be async, we need a bit of script. For example:

async function SetUpBackground() {
   let image = await element.takeSnapshot();
   document.body.background = "snapshot(image.id)";
}

So you call an async script function to capture an element. Then hook that up to the background of any DOM element. We'll need to sort out ownership of the captured image, once you set it in CSS the element which is using it as a background should keep it alive. But the handoff from script to CSS is tricky.

We can't give authors access to the image contents since it could have cross-origin data but this is the same restriction that VT works with. FWIW native platforms (like Android) also have APIs to snapshot parts of the UI so it seems like a good addition for the web.

[tinchox5]

Thank you for providing a detailed explanation of the API model. Based on what you've mentioned, it seems that I only need the ability to use a captured image of the previous view state, and the CSS element() logic is a perfect fit for this purpose (thank you for introducing me to this function!).

It would be great using the View Transition API in conjunction with something similar to CSS element() to create a smoother transition between views in a Single Page Application. However, it would be even better if this feature were to be integrated into the View Transition API itself with the capacity of retain an image of previus DOM.

Please let me know if there is anything else I can do to contribute to this API.

[nickcoury]

The snapshot API for non-live pixel snapshots would be useful for some advanced use cases. While it's possible to use VT API for most of them with some careful configuration, VT is still largely created around the idea of a playable transition with a discrete start and end.

Some use cases this could simplify:

• Gesture-linked animations where touch coordinates are used to position elements. Many native apps utilize this when dismissing layers or performing other actions. This is possible but difficult with VT API.
• Caching snapshots of SPA views to provide previews while re-rendering previous content. This is commonly used with native app switchers or mobile browsers when an app or page has been destroyed in the background, but the user is shown a snapshot of the previous state to be able to accurately select the desired app/page. Native APIs equivalent to this are heavily used for that purpose.
• Precomputing transition elements to enhance responsiveness (especially when talking about draggable gestures which can easily feel sluggish on touch move) would not be possible with VT API, but could be possible with this API. In general, keeping the page interactive can be difficult (though again not impossible) with the VT API especially if the transition only changes part of the page and the user can reasonably try to interact or scroll the rest of the page.

The cross-origin restriction make sense, and should not affect most use cases since this would be used for visual animations to the user. Inspecting the pixel data might be desired for certain use cases, but I don't see those overlapping closely with the animation case.

[khushalsagar]

Thanks for the feedback Nick! The use-cases you outlined make sense. There is a tradeoff about exposing a primitive to cache a DOM subtree as an image vs a higher level primitive for that use-case. Some thoughts below around that.

• Gesture-linked animations where touch coordinates are used to position elements.

  That's a key use-case we've seen with transition UX. Breaking this down into the primitives needed:

  1. A way to map the user's touch events to a gesture that animates a previous view/layer.
  2. A way to cache an image of the view which can be instantly shown for this animation. This also means the site needs to do budgeting for all these images assuming there is a stack of layers as the user navigates through the site.
  3. A way to persist the image while the view's DOM is re-generated.

  Exposing an API for 2 will require authors to do 1 and 3 manually. Ideal would be if the browser could provide a combined solution. For example, the edge swipe UX to navigate is one use-case where that's trivially possible, since the browser is aware of both the gesture and the navigation (so knows when to snapshot). We can provide a timeline that lets authors trivially set up an animation using that timeline.

  Does that sound like a reasonable path to consider? Or are there cases where letting the browser manage all of this is not the right call.

• Caching snapshots of SPA views to provide previews while re-rendering previous content.

  Is this closely tied to the use-case above? User comes to a previous view via a gesture and now a cached snapshot has to be replaced with a live DOM.

• Precomputing transition elements to enhance responsiveness

  Sorry I didn't completely follow this.

  Re: "keeping the page interactive can be difficult (though again not impossible) with the VT API especially if the transition only changes part of the page", I think scoped transitions is the primitive you're looking for.

[nickcoury]

Those sound like the right primitives for our use cases, and the second bullet point comment sounds like the same use case.

If I'm thinking about use cases that this wouldn't cover, an in-page non-layered and non-navigational RecyclerView comes to mind for infinite scrolling or carousels. I can think of some native app experiences that have infinite content consumption where this might use more flexibility, though I think there are better currently buildable solutions without this complexity, including content-visibility and just regular scrollable containers plus intersection observers so not a great example.

For the last use case, we sometimes use intersection observers to do some pre-work for animations to respect RAIL responsiveness guidelines. If we wait for a click to do certain operations needed for animation, it can sometimes take over 100ms on slower devices. We might choose to opportunistically run preprocessing for viewport elements to optimize this which can't be done for multiple possible transitions at a time with VT API.

[tinchox5]

Hi, I noticed that in a similar issue (#10568), @khushalsagar mentioned some security concerns about accessing old snapshot views. I don't feel qualified to express my opinion on that matter, but I think it might be less critical in an SPA.

Nevertheless, if old view snapshots can't be accessed due to security concerns, perhaps another solution would be to add the ability to "keep alive" the old view snapshot. This way, even you can't directly access the old snapshot, you can manipulate its CSS properties.

[khushalsagar]

The security concern is with preventing access to pixels, allowing the page to render it is ok.