GEM GUI Introduction

Throughout production, you can expect terminology and functionality to evolve as we make improvements.

The managing of a GEM-enabled location involves three primary activities as far as the GEM software is concerned:

• Site configuration
• Daily live operations
• Maintenance

Site Configuration is an infrequent operation. It is initially necessary since every location and situation is unique, but once a site has been configured and has started operations, it can be relatively infrequent.

Daily live operations are where the vast bulk of operations time is spent.

Maintenance may involve some IT-level maintenance of the infrastructure at the location, but in the context of this document, it refers to regular maintenance of the software installation, including upgrading the GEM platform, upgrading Experience software, installing new Experiences, etc.

The initial release of GEM will focus on an efficient GUI interface for the daily live operations of a GEM-enabled location. The operators will have a GUI for tasks they are expected to do regularly. While it may provide interfaces for the site's initial configuration or ongoing maintenance, in the initial release of GEM, these less frequent operations may be data file-driven or command-line interface-driven.

Operator devices

The primary GUI used by operators is a web-based GUI that runs in a browser on either a PC or a tablet device. The site operator determines what kind of device to equip each individual operator with based on factors such as the layout of the location, the number of operators, and the expectation that that specific operator may need to be mobile in order to do their job.

A number of operator functions involve identifying specific devices and/or specific users. To facilitate that, GEM will use a QR code scanning system. Tickets will have QR codes printed on them, which can be used to identify a user and thus access all information relevant to that user. Devices will have QR codes attached to them for the same reason. Many operator functions can be triggered by simply scanning a ticket or a device. GEM can tell the possible operations from the context and prompt the operator.

The exact nature of the device used to scan the codes has not been determined yet. Needless to say, each operator will need access to one. If the operator is using a tablet, it may be possible to use the tablet's camera, but we will need to investigate the usability of such an approach further. If the operator uses a PC, some additional scanning devices will need to be attached.

One of the interface "screens" contemplated is one that gives a map view of the game area and shows where all the customers are and makes it easier to locate those that are having problem. We expect that that screen (and other comprehensive type dashboard screens) will probably need to run on a larger screen devices and may not need to support small screen devices like the phone.

Daily live operations

There are three primary areas of operating an experience that the operators need to be involved in:

• Suit-up and onboarding
• Experience monitoring
• Offboarding

There are some daily functions - for example, the sanitizing of headsets or the replacing of batteries - that fall in between these main roles. Depending on the nature of the experience, the physical layout of the space, and the customer throughput, the operator may be able to combine these roles into just one or two people.

All GUI functions are available to all operators at all times. However, the GUI provides specific screens for the three areas mentioned above. Each of these screens has the same primary functional areas:

• Status information
• One button access to common actions (Quick actions)
• Notification area

The contents of the status area and the list of actions are uniquely designed for the role that the operator is performing at the time.

Because the operators have to spend a significant amount time interacting with and observing customers, their attention is not focused on the screen constantly and the notification area in the document below is used as a catch-all term for ways to notify the operator of current outstanding issues that need to be addressed.

Suit-up and onboarding

We differentiate between suit-up (getting the equipment, putting it on, adjusting the fit) and onboarding (synchronizing the group and starting the experience). Suit-up is the most operator-intense portion of running a Show. This is true for a number of reasons:

• Common friction points for people entering the experience

  • Ticket issues: invalid (already used) tickets or the wrong experience
  • Time issues: Entering too soon (or too late) for an assigned time window
  • Carrying things they should have put in a locker first

• Friction points during suit-up

  • Not understanding how to put on equipment
  • Not understanding how to tighten the strap for a reliable fit
  • Not knowing what to do with glasses
  • Obstacles: hats, ponytails, scarves problems, etc.

• Hardware friction points

  • The assigned device does not work correctly
  • Unexpected error message displaying on screen
  • Battery Warning

• Obstacles to onboarding a group

  • A member of the group is missing
  • A member of the group has not completed suit-up
  • Incorrect avatars displaying
  • A member's avatar is not showing up

For this reason, it is almost always a requirement that an attendant be available to help customers in the suit-up area. The software aspires to automate the process when possible, but often, customers will look to an attendant for help because they are lost as to what to do next. Usually, the attendant can spot that there is a problem, both by looking at their console and by looking at what is going on in the onboarding area.

In this area, GEM automatically identifies groups and individuals who are not having problems and ensures that they can smoothly proceed through the suit-up and onboarding process with no operator interaction at all and transition to the experience. This should be the most common case if instructional material in the suit-up area is appropriate.

Status information

The onboarding area has the notion of "slots". These are areas for groups that will be going through the experience together to assemble and suit up together. Because smaller groups of people that signed up together (e.g., families) are put together with others to make up a larger "gem group" of 8 people that will travel through the experience together, they need to be told which onboarding slot to go to since they don't know who those other people are. The onboarding slots are numbered and each person/signup group is assigned their onboarding slot on entry into the onboard area.

The status information in the onboarding view of the GEM GUI allows the operator to quickly assess the capacity of the onboarding area and identify customers/groups that are encountering friction.

• Identify which group is in which onboarding slot
• Identify free onboarding slots
• Identify onboarding slots with available capacity
• Only displays information on groups that are active in the area
• For each group, shows the overall status of the group
• For each group, shows the status of each group member
• If a group member is taking too long to progress, their status is yellow
• If a group member has encountered a detectable problem, their status is red

The status line for each member of the group displays:

• Does the user have an assigned device
• Which of the following states is the customer in:
  • Suiting-up
  • Prep in progress
  • Prep completed

Under normal circumstances, it is expected for the group to take some amount of time to suit-up and get ready. Only if that amount of time is exceeded do the status displays for that group start turning yellow.

With one glance at their screen, an operator should see all the groups currently onboarding. All of the information should generally be green If an individual is taking too long or has encountered an error they will show up color coded and the operator can click on them in the status screen and see more detail on the problem.

If everyone in the group has successfully suited up but there was an issue onboarding the group because of a game-server issue, for example, the entire group may turn red in the status display.

Notifications

While the status screen displays a constantly updated picture of what is going on in the onboarding area, there are situations where the operators assigned to the area are helping customers and may not notice issues in a timely manner. Therefore, there is a notification area where any status changes from green to another color will trigger a notification and a sound to alert the operator. Upon hearing the notification, the operator can glance at the notification area to see what the last change was and at the status screen to get a visual representation.

Quick actions

The following are common actions that operators tending to the suit-up / onboarding area will need to perform.

Entry control

This is an optional process and it can be used at locations with Shows that are busy, and/or have space constrained onboarding areas and there is a desire to regulate the entry of people into the onboarding area.

Operator flow:

• Operator selects "Entry" action
• Operator scans user ticket
• GEM determines:
  • Is the ticket valid
  • Is this the correct onboarding area
  • Is this the correct time window
  • Is there space in the onboarding area
• GUI displays a green light and a friendly beep if all is correct
• GUI displays a red light and unfriendly beep if there is a problem
• Operator directs customer appropriately
  • Wait if the onboarding area is full
  • Go away if you are in the wrong place or wrong time window

Assign devices

Sites can choose "self-assigned" devices, where the user picks up a device and assigns it to themselves by scanning their ticket. However, sites can also choose operator-assigned devices, where the operator performs the assignment.

If the user is having problems with self-assignment, the operator can always perform this action for them:

Operator flow:

• Operator selects "Assign" action
• Operator scans ticket
  • GEM determines:
    • Is the ticket valid
    • Is this the correct onboarding area
  • GUI displays a red light and unfriendly beep if there is a problem with the ticket
• Operator scans device
  • GEM determines:
    • Is this a valid device for this show
  • GUI displays a red light and unfriendly beep if there is a problem
• GUI displays a green light and a friendly beep if all is correct
• Assignment is done

Once an assignment is done, subsequent operator actions involving the user are done using a scan of the device, rather than a scan of the ticket.

Reassign device

If the user already has an assigned device but there is a problem (e.g., adjustment strap to loose, battery giving warning, etc.), the operator has a quick action to reassign the device.

Operator flow:

• Operator selects "Reassign" action
• Operator scans old device
• Operator scans replacement device
  • GEM determines:
    • Is this a valid device for this show
  • GUI displays a red light and unfriendly beep if there is a problem
• GUI displays a green light and a friendly beep if all is correct
• Reassignment is done

Restart user onboard

If a member of a group has had problems joining the experience server, but other members joined OK, the operator will see that status on the status display and initiate a retry/restart for that user.

Operator flow:

• Operator sees red status for user indicating connection issue
• Operator clicks on that user and selects "Restart"

Restart group onboard

If multiple members of a group show problems joining the experience server, or a restart for an individual user has not had the desired effect, a restart of the experience server is appropriate.

Operator flow:

• Operator sees multiple red statuses in a group, indicating connection issues
• Operator clicks on that group and selects "Restart"

Deboard user

If a user has already started the onboarding process and been assigned a headset, but if they change their mind and decide that they do not wish to do the experience after all, they need to be deboarded so that GEM knows that they are not expected to continue.

Operator flow:

• Operator selects "Deboard user" action
• Operator scans headset
• The user has now been removed from the experience
• Return headset to "need to be processed" pile

Deboard signup group

If a signup group (e.g., family) has already started the onboarding process and been assigned headsets, but if they change their mind and decide that they do not wish to do the experience after all, they need to be deboarded so that GEM knows that they are not expected to continue.

Operator flow:

• Operator selects "Deboard signup group" action
• Operator scans one headset of a group member
• All members of the signup group have now been removed from the experience
• Return headsets to the "need to be processed" pile

When a signup group has been aggregated into a larger "gem group" and that signup group is deboarded, the rest of the gem group continues on as before and are not impacted. If they have not finished suiting up yet, GEM may still add new incoming users to that gem group to max out capacity.

Deboard gem group

Suppose a gem group (e.g.) has already started the onboarding process and been assigned a headset, but there is a problem or they change their mind and decide that they do not wish to do the experience after all. In that case, they need to be deboarded so that GEM knows that they are not expected to continue.

Operator flow:

• Operator selects "Deboard gem group" action
• Operator scans one headset of a group member
• All members of the gem group have now been removed from the experience
• Return headsets to the "need to be processed" pile

Remove user from group

If a user has already been assigned to a group, GEM, by default, will wait for that user to get ready before the group as a whole is allowed to proceed. If that user is late or needs to go to the lockers, or in some such situation, the operator has the option to remove that user from the group and allow the rest of the group to proceed without them. In a situation such as this, the user removed from the group is automatically placed back in the queue of people who have not started onboarding yet, and when they are ready and start the suit-up process, they will be added to some other group.

This quick action is for customers who have not been assigned a device yet and thus the operator needs to scan their ticket instead of the device.

Operator flow:

• Select "remove from group"
• Scan ticket

Add an existing user to group

When user start the suit-up process, they have been automatically been assigned to a group that consist of any people they signed-up together with, and other people to make up a full group. If, however, a user did not sign-up with a specific group (or was removed from one earlier) and now, instead of being assigned to the next available group has identified a specific group they wish to travel with, the operator can add a user to that group.

Once a user has been assigned a device, they cannot be reassigned to a different group with this action.

Operator flow:

• Select "add to group"
• Scan the ticket of the user
• Scan the ticket of someone from the target group

Create user

On occasion, a user will need to go with a group without having signed up for the experience. This can happen when docents, for example, want to go with a group, developers, or VIPs. In this case, an operator has the ability to create a new GEMUser record for that individual, and they can subsequently onboard with others. If the newly created user wishes to go through the experience with a specific group, the operator can use the "add to group" action as described above to add them.

Operator flow:

• Select "Create user"
• Scan a device
• Hand the device to the user

Note that in this action a ticket with a code is never generated. The operator simply assigns a headset. Once this has been done, the new user is like any other user.

Create a new group

The operator has the ability to create a new group. It will be created with no people and GEM will recognize it as an operator created group and will not auto-assign any people to it. The operator can then assign specific individuals to that group.

This is used if there is some combination of individuals (either legitimately signed up users and/or operator created users) who wish to go through the experience themselves, with no other people in their group. E.g. press, VIPs, etc.

Optionally, the operator can select a number of new users and GEM will not only create the group but also create that many new users in a single operation. It is equivalent to the operator having created an empty group and then having created a number of users and assigned them to the group.

Merge groups

If the operator notices on their status display that there are two groups onboarding and those two group could be combined into one group (based on the group size for the experience), the operator has the ability to combine the groups and send them through the experience together. This is a throughput optimization.

Operator flow: - Select "Merge groups" - Select 1st group on display - Select 2nd group on display

Experience monitoring

The vast bulk of the time that users spend in a Show is actually in the experience. However, having successfully suited up and onboarded, this is also the area where fewest friction points and operator interventions are expected.

Nevertheless, it is necessary to monitor the Stage, both visually and using the GEM operator GUI. Visually, the operator needs to watch for situations that may not be automatically tracked and thus may not show up on the GUI status screen:

• Users raising their hand (they have been instructed to do so if they have a problem)
• Users who have taken their device off (and thus may no longer be tracked correctly
• Untracked people (may not be customers) waking onto the stage

There are other situations that the operator may notice visually, which will also appear on the GUI status screen:

• Users not keeping up with their group
• Users walking into inappropriate parts of the trackable space

There are situations that are hard to identify visually and can only be spotted by looking at the GUI status screen:

• Network connectivity problems for a particular user
• User stuck in chaperone for a long time
• Frequent tracking problems for a particular user
• Low battery alert
• Device overheating alert
• TBD

The status information and quick actions for the experience monitoring area are focused on helping the operator quickly identify anyone who may need attention, whether because of problems or misbehavior, and a set of common quick actions to help resolve those situations:

Status information

The status screen for this area is designed to quickly identify users with problems and locate them.

The status screen provides a top-down map of the stage area and dots representing the users on the stage. As the users move around the stage, the dots are updated in real time. The dots are color-coded with green, meaning everything is OK. Yellow indicates that the user has been in chaperone for some period of time and may need attention soon. Red means that a definite issue that requires operator intervention has been spotted. Each dot representing the user has a number in it to identify the zone number that that user should be in.

If the experience has built-in a "call operator" interaction and the user has triggered it, the user will turn red on the status screen, and the operator will be notified.

If the operator sees a non-green user, they can click on the user and get details on the issue. From there the operator can determine next steps.

If the operator sees someone raise their hand or take their headset off, they can walk to the user to determine the problem without needing to determine on the status screen which user it is. The operator can initiate quick actions by scanning the code on the problem user's headset to identify the user.

Notifications

As in the onboarding area, because operators assigned to this area spend a significant portion of their time looking at the stage to see if users need help, not all their attention is on the GUI, and they may miss status changes. For this reason, all status changes also show up as notifications with audio.

Quick actions

Reassign device

If the user is having a problem with their device, the operator can quickly replace it and the experience will resume at the appropriate place. When the user rejoins the experience on the new device, they will be with the rest of their group. It is up to the experienced developer to decide whether to pause the experience when users drop out and wait for them to rejoin or whether to continue without them. If the experience did not pause when the user was switching out their headset, the user will miss that part of the experience and rejoin the group at their current location.

Operator flow:

• Operator selects "Reassign" action
• Operator scans old device
• Operator scans replacement device
  • GEM determines:
    • Is this a valid device for this show
  • GUI displays a red light and unfriendly beep if there is a problem
• GUI displays a green light and a friendly beep if all is correct
• Reassignment is done

Deboard (bounce) user

If a user is going to leave the experience (whether it is their own choice or they are being removed from the experience by the operator for misbehavior), GEM needs to be notified that they are being removed and not coming back. This ensures that GEM will not generate any warnings or errors related to the situation and that the experience logic will not be confused.

Since players are generally deboarded in an unpredictable part of the stage, one cannot assume that the device can be immediately processed and marked as returned as it would be under normal circumstances in the offboarding area. For this reason, the return of the device is done as a separate action, probably at a slightly different time.

Operator flow:

• Operator selects "Deboard user" action
• If on stage with the user
  • Operator scans device
• If at operator console watching status screen
  • Operator clicks on user in status display
• The user has now been removed from the experience
• The users device is in passthrough with instructions to wait for operator
• Operator collects headset and return device to "need to be processed" pile

When the operator returns the device to the offboarding area - Operator select "Return device" action - Operator scans device

Deboard (bounce) group

If a group is going to leave the experience, GEM needs to be notified that they are being removed and not coming back. This ensures that GEM will not generate any warnings or errors related to the situation and that the experience logic will not be confused.

Since players are generally deboarded in an unpredictable part of the stage, one cannot assume that the device can be immediately processed and marked as returned as it would be under normal circumstances in the offboarding area. For this reason, the returning of the device is done as a separate action, probably at a slightly different time.

Operator flow:

• Operator selects "Deboard group" action
• If on stage with user
  • Operator scans device of one user from group
• If at operator console watching status screen
  • Operator clicks on group in status display
• The group of users has now been removed from the experience
• The users device are in passthrough with instructions to wait for operator
• Operator collects headsets and return device to "need to be processed" pile

When the operator returns the devices to the offboarding area - Operator select "Return device" action - Operator scans devices

Emergency Pause

The operator can declare an emergency (when, for example, an alarm in the building goes off, or someone on stage is injured). A pause notification will be sent to all groups on the stage and their pass-through will be activated. At this point it is up to the operator to give further instructions to the user based on the nature of the emergency.

If the operator has declared an emergency pause, they may resume the experience and everyone on stage will resume from where they paused.

Operator flow: - Select "Emergency pause" - Select "Emergency pause" again to unpause.

Emergency announcement

The operator can make an emergency announcement to everyone all users on the stage.

Operator flow: - Select "Emergency announcement" - Speak the message.

Emergency announcement to user

The operator can make an emergency announcement to a specific user.

Operator flow: - Click on user in status screen and select "Emergency announcement" - Speak the message.

Offboarding

Status information

Notifications

Quick actions

Device management

Most infrequent "maintenance" operations will be performed by command line as described in the operations manual. However, the maintenance of the device pool is an area where frequent operator interactions is expected and thus is included in the GUI.

Status information

The status screen will display a list of all devices. These can be filtered by the show/onboarding area that they have been assigned to. By default the devices are grouped by their device state, as described elsewhere in this document.

Devices are allocated to specific OnboardAreas and by default the status shown the operator only includes the devices managed by the GEM server that the operator is currently connected to. The ability is provided to show all devices at the location.

Generally the devices that operators will be interested in are the ones that are in less common states:

• Quarantined devices
• Devices not contacted in a long time
• Missing devices (devices that finished an experience but were not scanned after)
• etc.

The status screen will be organized to help the operator resolve inventory issues such as these.

Quick actions

Add a device

When a new device is acquired, it is stickered with a code, and its data entered as a new entry into the database of devices. The precise operator flow, in order to facilitate minimal errors and greatest ease of entry, is still being investigated.

Delete a device

If a device is being taken out of circulation permanently because it is broken or has been lost, the DB entry for it can be deleted from the database.

Allocate a device

The central database maintains a set of devices. Each device can be assigned to (and only one) OnboardArea at a time. "Allocating" a device refers to a the process of assigning a device for use in a specific OnboardArea. If the device is already allocated, it must be deallocated from the current OnboardArea before it can be allocated to another.

Deallocate a device

Unassigns a device from its currently assigned OnboardArea. GEM will not allow the device to be used for a show until it is assigned to an OnboardArea.

Quarantine a device

The device is flagged as quarantined with a simple one step process. This prevents it from being assigned at an onboard area until someone has unquarantined it. This quick action is also available to operators monitoring the experience as well as the offboarding area. If customers complain about a device, it should be quarantined for further investigation.

Unquarantine a device

If a device has been checked out and appears to be functioning properly, it can be unquarantined. It will be placed in an unassigned device state, ready for use at the allocated onboarding area.