Localizing With Marker Sync

This document describes how to localize multiple users in a shared environment using the ARDK by having users scan a visual marker, such as a QR-code.

Other forms of AR localization provided by ARDK involve scanning an object in the environment, which can be difficult for users in some scenarios. Scanning markers instead of objects in the environment is often easier, and works in situations where users can’t find or capture good objects to scan. Additionally markers can contain session information so clients can scan a marker to both join a specific shared AR session and synchronize with that session.

Synchronizing using markers is generally less accurate than synchronizing using the environment and can incur more drift between peer positions and orientations. Use marker synchronization in spaces with few scannable objects or features, such as outdoor spaces.

Synchronize Using Markers

Synchronizing using markers works as follows:

• Host generates marker data

• Host calls ARNetworking.InitializeForMarkerScanning and displays the marker on screen

• Non-host clients use ARNetworking.ScanForMarker and scan the marker displayed on the host device. Clients use results to join session and/or localize in the shared session. A client PeerState of Stabilizing indicates the client has successfully localized in the shared environment using markersync.

The following flowchart shows the basic flow and the API calls needed. Dotted line boxes indicate steps taken by the developer. Solid line boxes indicate steps taken by the user/player. Steps in gray should be happening simultaneously.

Generating Markers

The IARNetworking.InitializeForMarkerScanning method only requires the marker’s points’ positions, which means that markers can be generated, displayed, parsed, and/or scanned in any way, as long as the following are adhered to:

• Marker consists of at least 4 identifiable points (ex. The 3 position points and 1 alignment point in a QR code)

• Marker points are stationary on screen while displayed

• An implementation of IMarkerScanner can be used to scan the world for a marker

  • Optionally passed in with the call to IARNetworking.ScanForMarker.

  • If none passed in, the default implementation (ARFrameMarkerScanner), which scans for QR codes, will be used.

• An implementation of IMarkerParser can be used to extract MarkerMetadata (serialized as a byte[]) from an array of pixels provided by an IMarkerScanner.

  • Optionally passed in when constructing an ARFrameMarkerScanner.

  • Required when using custom implementations of IMarkerScanner.

  • If none are passed in, or if the default IMarkerScanner is used, the default implementation (ZXingBarcodeParser) will be used. This only works when the marker is a QR code.

• The same implementation of IMarkerParser can be used to identify the 4 or more marker points registered by the host’s call to IARNetworking.InitializeForMarkerScanning

• An implementation of IMetadataSerializer can be used to deserialize the bytes from the IMarkerParser into a MarkerMetadata.

  • Optionally passed in with the call to IARNetworking.ScanForMarker

  • If none passed in, the default implementation (BasicMetadataSerializer) will be used.

Example

An example scene of utilizing a QR code to quickly join+sync multiple players into a Multiplayer AR session can be found in the ARDK Examples project under Assets/ARDKExamples/MarkerSync.

Helpers

• ZXingMarkerGenerator is an example of how a barcode is generated with a MarkerMetadata, and of how to calculate the positions needed for the call to IARNetworking.InitializeForMarkerScanning

• BarcodeDisplay is a helper component that can be placed in a scene to display a barcode generated using the ZXingMarkerGenerator.

Best Practices

• Call IARNetworking.ScanForMarker after the device’s AR tracking state is optimal.

  • Tracking quality is accessible by through your ARSession’s CurrentFrame.Camera.TrackingState. Best quality is TrackingState.Normal.

• Even after achieving a sync with a marker, keep the players looking in the same area (so it’s more likely a map is found and synced against) until PeerState is Stable for all players.

  • While Marker Sync allows for a near instantaneous sync between players, there’s no drift correction active like there is in ARDK’s traditional syncing methods.

  • The PeerState.Stabilizing state achieved through marker sync should be acceptable for play, but for the most reliable sync, using maps and getting to PeerState.Stable is best.

Using MarkerSync with Shared Environment Localization

Note that the ARDK can collect synchronization information for both environment synchronization and marker synchronization at the same time. Currently, the ARDK collects environment mapping data in the background even while marker synchronization is in progress. This allows the possibility of starting with marker synchronization to quickly synchronize all clients, and then attempt environment synchronization for better accuracy.

To do this, first synchronize using marker synchronization so that all clients are in Stablizing PeerState. Then, instruct players to scan the environment and try to localize for a reasonable amount of time. If the host moves to Stable it means the environment was scanned. If all clients move to Stable it means they were able to localize using the environment. If the host or clients don’t move to Stable within a reasonable amount of time, you can choose to proceed with the experience anyway, as the devices are already synchronized using marker sync.