Fine tuning of alert thresholds

When you are working with SAP Focused Run alert management, you might detect that some alerts are triggered too often. This will lead to work checking the alert and finding out it is more or less false alarm.

So a new threshold is needed, but what is a good threshold?

Questions that will be answered in this blog are:

  • Which tools can I use to perform fine tuning of alert thresholds?
  • Can I perform a forecast based on the data?
  • Can I perform a sensitivity analysis?
  • Which installation activities are required to enable the forecast and sensitivity functions?

Fine tuning alert threshold

In our example we will look at the Dialog Response Time metric. The current threshold for red alert is set to 5000 ms (5 seconds). The alert is triggered too often. But the question to answer now: what is a good threshold to set based on the historical data?

First click the Open metric in new window icon to enlarge the screen:

The enlarged screen now opens:

As you can see 2 times the red threshold was hit. We want to fine tune now. First select the calendar icon and select last 7 days to get full week overview:

You can use the forecast button to let the system create a forecast:

The forecast will now show mean, mean low and mean high forecast:

In this specific use case the prediction is that the maximum is 3300 ms (3.3 seconds).

Now open the statistics button to see the statistics and the recommended threshold tool button:

By changing the Sensitivity slider, the system will calculate different proposal for the alert threshold. In our case when we move sensitivity to 4 the new recommended threshold value is recalculated:

In this case it is 7669 ms.

So we now have collected following facts:

  • Current threshold of 5 seconds is reached too often
  • Average forecast based on history has a mean value of 3.3 seconds
  • Performing the sensitivity analysis the threshold recommendation is about 7.7 seconds

Based on this data the red threshold is best to increase from 5 to 8 seconds to get a good alert function. It will not reach too soon, hence limiting false alerts, but it will still alert in time in case poor performance happens.

Enabling forecast and sensitivity analysis

The forecast and sensitivity analysis function use the Application Function Library (AFL) and SAP HANA Automated Predictive Library (APL). These must be installed separately. The installation details and post steps for granting permissions are described in the Focused Run master guide in the section “Predictive Analytics Setup – Metric Forecasting”.

After the installation you must activate and assign PFCG role SAP_FRN_APP_PAS_DISP to be able to see the buttons.

Work mode management

There will be planned maintenance on SAP systems and the below databases, operating systems and the below infrastructure. During this planned maintenance, you don’t want any alerts and mails from SAP Focused Run, since the application is not available for a planned time.

Using the work mode management function you plan these maintenance events.

Questions that will be answered in this blog are:

  • How can I set up planned maintenance for systems monitored by SAP Focused Run to temporarily suppress the alerts during maintenance?
  • How can I see the current maintenance and planned upcoming maintenance?
  • Can I make a public page for the IT calendar?

Planning maintenance

Goto the IT calendar and workmode management tile:

Select the system (or select multiple systems in one go) and create the work mode for maintenance, and immediately switch to Expert Mode:

With expert mode you can simply set start and stop date and time.

You also should check the scope (the system will make a proposal) and extend if needed:

After saving, you can send notification via mail on the planned maintenance:

You can now see the planned maintenance in the calendar mode.

Overview of upcoming events

If you are in the calendar mode there can be a lot of systems. The overview screen for upcoming events will filter on the systems which have maintenance mode planned:

Actual maintenance

Actual maintenance can be seen in the System Down monitor:

You will only see maintenance that is currently running in the system down monitor, not the upcoming events.

Tips and tricks

Can I select multiple systems for planned maintenance in one go? Yes you can, this will save you a lot of time to input.

Can I make a recurring maintenance? Yes you can, in the scheduling screen mark the maintenance as recurring.

Can I make a public page for the IT calendar? Yes, you can, follow the instructions in OSS note 2926433 – IT Calendar access to Non-SAP Focused Run Users.

Alert management outbound integration to ServiceNow

SAP Focused Run alert management function can send out mails to alert to mail addresses (see this blog).

SAP Focused Run can also call an outbound integration to a ITIL tool like ServiceNow. This can help to speed up incident creation.

Questions that will be answered in this blog are:

  • How does the high level integration between SAP Focused Run and ServiceNow look like?
  • Where can I find information on the to-be-implemented ABAP BADI?
  • How can I send an alert directly to ServiceNow from the Alert management detailed page?
  • How can I automate in template settings to send an alert via outbound integration towards ServiceNow?

Setting up the integration

For setting up the integration to ServiceNow the AEM third party consumer connection BADI must be implemented. The full manual for the BADI itself can be found on the SAP Focused Run Expert portal.

The documents describes the BADI in generic way.

To call ServiceNow you have to use one of the following 2 integration methods:

  • Call webservice: in this case you import the WSDL from ServiceNow and generate the proxy and execute the SOAMANAGER settings to logon to ServiceNow. You need ABAP code in the BADI to call the proxy. See this blog for generic use of setting up webservice consumption in ABAP stack. Available webservices for ServiceNow can be found on the ServiceNow page.
  • Call the ServiceNow midserver: in this case you call a REST interface. In this case you need to setup a HTTP RFC connection to the midserver. ABAP code in the BADI is needed to make the REST call. See this blog for generic use of REST call in ABAP stack. REST API references from ServiceNow can be found on the ServiceNow page.

In a future blog we will explain the ABAP code to put inside the BAPI.

From alert trigger integration

If you are inside an alert, you can trigger the alert reaction:

Then select the reaction to forward to ServiceNow:

Within few seconds the alert in ServiceNow is created:

Alert reaction automation in template settings

The alert reaction to ServiceNow can also be automated as Outbound Integration. If you are in template maintenance mode, switch to Expert mode.

In the alerts tab now configure the alert type for Forward to and Outbound Connector:

Assign the correct variant.

If you click on the variant you go to the variant configuration screen:

Then select the outbound integration name to see the details:

Important here is the where used list, which shows you from which templates and template elements the connector is called.

Whenever the alert is raised, also the outbound integration connector to ServiceNow is called.