Troubleshooting Monitor Authorization Permissions

If a configured monitor that was working previously begins to raise an alert with the following message, there is a groups permission error to troubleshoot:

Error: chaossumo.util.akka.http.ChaosDirectives$Exceptions$AuthorizationException$: Authorization failed.

The problem could be that the groups for the user that created or last updated the monitor did not have the
kibana-opendistro:alerting:alerts:read permission. The monitor does not have permission to run. There could also be an issue where the RBAC groups associated with the user who last saved the monitor changed and no longer have the proper permissions to use the views, object groups, or query associated with the monitor. Or, the groups associated with the monitor definition changed when a user updated the monitor.

When troubleshooting this error, it can be very helpful to note the groups that are configured for the view(s) used by the monitor that raised the alert, so that one or more groups could be investigated for the alerts permission. One way to obtain the group IDs for a monitor is to use the Browser DevTools window to display more information about the monitor.

Navigate to the Monitors page and open the DevTools window.
Select the monitor that triggered the alert. You should see a Name with the same Monitor ID value in the DevTools left frame of the Network tab.
Select the monitor ID name element, then click Preview. The right pane updates with information about the resource. Click to expand the groupIds property.

In the example above, groupIds is set to default, which is common for monitors created by the root user, especially during the ChaosSearch trial phase. During the production transition, the default group is usually updated to have a smaller set of basic permissions for new users who are not otherwise assigned to groups. Each site administrator typically creates new groups for the production environment to specify the various levels of user access that are needed. The resolution for this problem is to update and save the monitor while logged in as a user who has proper group assignments with the full complement of permissions so that the monitor can run successfully. As an alternative, if the current user is the person who must manage the monitors, the solution is to ensure that the monitor administrator has the correct group assignments to fully manage and run monitors.

Sometimes the groupIds assigned to a monitor are a sequence of one or more internal group IDs for the user who last saved the monitor. In this case, if a monitor sends Authorization alerts, a group is missing the alerts permission, or the user might not be assigned to the correct groups needed for the monitor to run. The group IDs list can provide the ChaosSearch Customer Success team member with information needed to diagnose the root cause of the authorization alert.