Follow

Manage Orchestrations

Manage Orchestrations

The Orchestration Analyzer provides the features you need to analyze and manage Kahuna Orchestration.

  • Compare goal achievement of messaged users to control groups.
  • Analyze touchpoint and Orchestration success based on metrics.
  • Manage path steps and touchpoints to optimize Orchestration performance.

To analyze an Orchestration, navigate to Manage > Orchestration in the left navigation pane.

Orchestration List Page

The Orchestration List provides information about each Orchestration that has been launched, saved, disabled, locked, or cloned. You can perform the following basic actions.

The Orchestration list displays Kahuna Orchestrations in chronological order, newest to oldest.

On the top row of each list entry, the Orchestration list displays the following:

Orchestrations Name

The name of the Orchestration. Click the Orchestration name to edit it (see Edit an Orchestration).

Segment Name

The filter set you defined as the segment for the Orchestration (see Kahuna Orchestrations).

The collapsible rows of each list entry display the following:

State

One of the following Orchestration states:

Draft

A saved Orchestration that has not yet been launched.

Active

A saved Orchestration that has been launched and can be monitored.

You can edit a launched Orchestration. See Edit an Orchestration.

Disabled

A saved Orchestration that has been launched but disabled from sending messages. You can monitor the results that continue to arrive after the Orchestration has been disabled (see Disable an Orchestration).

Locked

A saved Orchestration you have locked. Kahuna continues to count user responses and distribute messages; however, no new users may enter the Orchestration.

Version

The number of the Orchestration version. Latest is the most recent version of the Orchestration.

Created Date

The date the Orchestration was created.

Created By

The username of the person who created the Orchestration.

Actions

Actions available to perform:

To search the Orchestration list, enter a search string in the search box.

Orchestration Overview

The Orchestration Overview provides a summary of your Orchestration.

Entered

The number of times a user entered the Orchestration by meeting the segment criteria and triggering the start event.

In Progress

The number of times a user entered the Orchestration and has not been removed or finished.

Removed

TThe number of times a user was removed from the orchestration because of a user merge. When a user is in an orchestration and their account gets merged with an existing user, the user may be removed from the orchestration because they no longer qualify for it or the merged user was already in progress.

Finished

The number of times a user reached the end of a path.

Next to each delay point, you will see the number of users who are currently waiting to proceed. The number of users who are not in the control group are followed by the number of users in the control group in parentheses.

Similarly, next to each Path End, you will see the number of times that users have completed that path.

In each touchpoint, you will see metrics about message delivery.

Delivered

The number of messages delivered to a user

Undelivered

The number of messages that were unable to be delivered.

Disqualified

The number of messages not sent because a user no longer qualified to receive that message. For example, a user may be disqualified for a touchpoint because they do not have an email address and the only messages in the touchpoint are emails.

Clicking on a touchpoint shows more detailed metrics about the touchpoint campaign. When the touchpoint campaign metrics are visible, the canvas shows on the side of the window, allowing you to click on other touchpoint campaigns. To return to the full-screen canvas, click on the "Back to canvas" button above the touchpoint metrics.

Touchpoint Campaign Details

Campaign Details Summary

The campaign details summary section contains an overview of the campaign definition. It includes the following sections.

Campaign Group ID & Channels

The Campaign Group ID is the identifier of this campaign, which can be used in API calls or support tickets. Channels are the delivery mechanisms used in this campaign.

Segment & Filters

Displays the trigger event that starts this campaign for a user and the filter. Users who do not match the filter conditions do not qualify for the campaign.

Campaign Goal

Displays the goal event and secondary tracking events being monitored for this campaign.

Time & Duration

Describes when the campaign message is sent and over what time period. For scheduled campaigns, the last and next run are also displayed.

Optimization

Describes the Kahuna optimizations that are activated for this campaign, including Cross-channel Optimization, Send Time Optimization, and Message Optimization.

Campaign Details Tabs

Campaign Details pages display the following tabs. They provide detailed information about your campaign so that you can make informed decisions about current and future campaigns.

 

Overview Tab

The Overview tab provides high-level details about the campaign delivery and engagement.

Engagement Rate

The percentage of users who engaged with the product after receiving the campaign message.

Engagement Uplift

The percent increase in the user whoengaged with the product after receiving the message versus the users in the Control Group. If a control group was not used, the uplift will not be available.

Goal Completion Rate

The percentage of users who completed the primary goal after receiving the campaign message.

Goal Completion Uplift

The percent increase in the user who completed the primary goal after receiving the campaign message versus the users in the Control Group. If a control group was not used, the uplift will not be available.

Delivery metrics diagram

The diagram displays the counts of users at various stages of campaign delivery and interaction. The KPI boxes show the number of users at each stage of the delivery and interaction process.

The following KPI tiles are shown for users in the messaged and control groups:

  • Requested For Adaptive campaigns, the total number of rows of data processed via CSV upload or API.
  • Found For Adaptive campaigns, the total number of users identified in Kahuna from a credential.
  • Not Found For Adaptive campaigns, the total number of records unable to be matched to a Kahuna profile.
  • Triggered For Conversion and Trigger campaigns, the number of users who have triggered the start event.
  • Selected: The number of users who met the filter criteria. The Selected KPI tile in the Control Group section shows the number of users who were selected to be in the control group.
  • Disqualified: The number of users who do not have a valid channel andor do not meet the filter criteria.
  • In Progress: The number of users currently in the delivery window. The messages for these users are queued for delivery.
  • Delivered: The total number of messages delivered to your users. This KPI tile includes the percentage of these users against the total number of users selected.
  • Undeliverable: The total number of messages that were unable to be delivered. This KPI tile includes the percentage of these users against the total number of users selected.
  • Engagement: The number of users who return to your application after receiving the message. This KPI tile includes the percentage of these users against the total number of messages delivered.
  • Displayed: The number of marketer-driven in-app messages displayed to a user.
  • Not Displayed The number of marketer-driven in-app messages that were available, but were not retrieved before the message expired.
  • Pending: The number of users who have not opened the app to view the marketer-driven in-app message.
  • Goal Achieved: The number of users that completed the primary Goal Event after receiving a message. This KPI tile includes the percentage of these users against the total number of messages delivered. The Goal Achieved KPI tile in the Control Group section shows the number of users in the control group who completed the primary Goal Event without being messaged by this campaign.

 

If a Kahuna Optimization such as Cross-channel optimization, Send Optimally, or Message Optimization is enabled, a Kahuna badge will appear at the point int the diagram where the optimization occurs.

 

 

Details Tab

The Details tab shows a breakdown of the campaign by channel and message. If only one channel was in chosen for the campaign, a badge will indicate the type. If more than one channel was used, the tab will default to showing metrics that apply to all campaign types. To view the message delivery details for a specific channel, press the appropriate button.

Within each channel, you can view the message variants by clicking on the variant header.

The metrics available are described below, along with the channels they apply to:

 

Channels Metric Definition
Requested

For Adaptive campaigns, the total number of rows of data processed via CSV upload or API.

Found

For Adaptive campaigns, the total number of users identified in Kahuna from a credential.

Not Found

For Adaptive campaigns, the total number of records unable to be matched to a Kahuna profile.

Triggered

For Conversion and Trigger campaigns, the number of users who have triggered the start event.

Selected

The number of users who met the filter criteria. The Selected KPI tile in the Control Group section shows the number of users who were selected to be in the control group.

In Progress

The number of users currently in the delivery window. The messages for these users are queued for delivery.

Delivered

The total number of messages delivered to your users. This KPI tile includes the percentage of these users against the total number of users selected.

Displayed

The total number of market-driven in-app messages displayed to your users.

Not Displayed

The number of marketer-driven in-app messages that were available, but were not retrieved before the message expired.

Pending

The number of users who have not opened the app to view the marketer-driven in-app message.

Disqualified The number of users who do not have a valid channel and/or do not meet the filter criteria.
Undeliverable

The total number of messages that were unable to be delivered. This KPI tile includes the percentage of these users against the total number of users selected.

• Blacklisted The number of SMS messages not delivered because a user replied to a previous message with STOP, CANCEL, or UNSUBSCRIBE (the user unsubscribed).
• Blocked The number of emails blocked because of a previous bounce, reject, or spam report.
• Bounced The number of emails that could not be delivered because the email account no longer exists or is invalid.
• Invalid Number The number of SMS messages not delivered because a recipient phone number is invalid; for example, the phone number is inactive or not recognized by the carrier, the number is incorrectly formatted or incomplete.
• Local Send Missed The number of messages not delivered because the target time for delivery had passed when local send time was chosen.
Missing Params The number of messages not sent because a filter or personalization parameter was not supplied.
• No International Permissions The number of messages not sent because permission is not enabled to send to international phone numbers.
• No Longer Valid The number of people who no longer qualified for the campaign because they did not meet the filter criteria at the time of the delivery attempt.
• No Routing Number of messages not sent because of telecom routing error.
• Unavailable The number of SMS messages not delivered because the phone was switched off and the message expired.
• Not Reachable The number of messages that did not reach their intended recipients due to issues such as system failures, network security, or non-existing accounts.
• Previous Uninstalled The number of users who uninstalled the application after qualifying but before message delivery.
Provider Error The number of users who were not sent a message because of an unexpected error in Apple's or Google's push system.
Quiet Hours The number of emails not sent because quiet hours were in effect at send time.
Rate Limited The number of users who were not sent a message because they had received a message on the same channel within the timeframe defined by the cooldown setting for that channel.
• Rejected The number of emails not sent because the target email addresses are on a blacklist or invalid email domain.
User Missing The number of messages not sent because the user was not found at the time of delivery.
Open The number of times an email sent through this campaign has been opened. A single email can be opened multiple times.
Unique Open The number of users who opened the email sent through this campaign.
Clicks The number of times a user clicked on an action button in an in-app or push message, or clicked on a link in an email sent through this campaign. A single email can be clicked multiple times.
Unique Clicks The number of users who clicked on an action button in a push message or at least once on an email sent through this campaign.
Dismissed The number of users who dismissed the in-app message without clicking on an action button.
Engaged The number of users who returned to the application before the end of the next day after receiving the message.
Goal Completed The number of users that completed the primary Goal Event after receiving a message.
Tracking Events The number of users that completed the non-primary goal events after receiving a message. Each additional tracking event is listed individually, and summarized
Attrition The number of users who became unreachable through push notifications after receiving your message. Attritition is tracked throuh the following metrics:
• Opt-Out The number of users who opted out of push notifications after receiving a message.
• Report Spam The number of users who reported an email message as Spam.
Uninstall The number of users who uninstalled the application after receiving a message
Unsubscribe The number of users who unsubscribed for your email list after receiving a message or responded to an SMS message with STOP, CANCEL, or UNSUBSCRIBE.

 

 

Orchestration Details

Kahuna displays the following KPIs in the Details panels.

Touchpoints

The number of touchpoint steps in the orchestration

Logic Splits

The number of logic split step in the orchestration

Percent Splits

The number percent splits in the orchestration

Entered

The number of times a user has entered the orchestration

In Progress

The number of users who have started the orchestration but have not finished or been removed from it.

Removed

The number of times a user was removed from the orchestration because of a user merge. When a user is in an orchestration and their account gets merged with an existing user, the user may be removed from the orchestration because they no longer qualify for it or the merged user was already in progress.

Finished

The number of times a user reached the end of a path.

Below the KPIs, you can view statistics for individual touchpoints in the orchestration. The columns of the table are:

  • Touchpoints: The name of the touchpoint
  • Considered: The number of times a user reached the touchpoint
  • Disqualified: The number of times a user was not meesaged because they do not have a valid channel
  • Delivered: The number of messages delivered to your users
  • Undeliverable: The number of messages that were unable to be delivered
  • Engaged: The number of times a user returned to your application after receiving your message
  • Eng. Rate: The percentage of times a user returned to your application compared to the number of delivered messages
  • Clicks: The number of times a user clicked on your message
  • CTR: Click-through rate; The number of times a user clicked on your message compared to the number of delivered messages
  • Attrition: The number of times a user became unreachable after receiving your message
  • Attrition Rate: The number of times a user became unreachable after receiving your message compared to the number of delivered messages
  • Goal Completed: The number of times a user reached the goal defined within a touchpoint
  • Goal Completed Rate: The number of times a user reached the goal defined within a touchpoint compared to the number of delivered messages
Edit an Orchestration

You can edit both a launched and a draft Orchestration.

Edit a Draft Orchestration

If you have saved a draft of an Orchestration but have not launched the Orchestration, you can edit Orchestration components, such as segments, splits, and touchpoints. When your edits are complete, you can save and launch your orchestration.

To edit a draft Orchestration
  1. From the Kahuna Orchestration list page, click the draft Orchestration you want to edit. The Kahuna Orchestration Creator opens.

  2. Add or delete segments, goals, or touchpoints. See Create an orchestration.

  3. Either click Save to save the draft or click Review and Launch to launch the Orchestration.

Edit a Launched Orchestration

If you have already launched your Orchestration, you can edit just the messages of a touchpoint. In this case, the Orchestration continues to run and any new users that enter the Orchestration will receive the new messages.

To edit the touchpoint messages of a launched Orchestration
  1. From the Kahuna Orchestration list page, click the launched orchestration you want to edit. The Kahuna Orchestration Creator opens.
  2. Click the touchpoint whose messages you want to edit. The Orchestration touchpoint page opens.
  3. Click "Edit" under the message variant you want to edit.

    The message content dialog appears.

  4. Click the Edit Content button. The message content editor opens.

  5. Make any necessary changes to your message content, then click Update to return to the touchpoint page. For details about editing touchpoint messages, see Push, Email, SMS, or In-App Messaging.

    Note: You cannot enable or disable channels; you can only change a message or add variants.

Disable an Orchestration

After you have launched a Kahuna Orchestration, you can disable it by hovering over the Action column of a launched Orchestration and selecting Disable from the pop-up. A disabled orchestration continues to count user responses; however, no new users may enter the segment and no messages are distributed. You cannot enable a disabled Orchestration.

Lock an Orchestration

After you have launched a Kahuna Orchestration, you can lock it by hovering over the Action column of a launched Orchestration and selecting Lock from the pop-up. A locked Orchestration continues to count user responses and distribute messages; however, no new users may enter the segment. You cannot unlock a locked Orchestration.

Clone an Orchestration

After you have created a Kahuna Orchestration, you can clone it by hovering over the Action column of a launched Orchestration and selecting Clone from the pop-up.

Delete an Orchestration

After you have saved a Kahuna Orchestration, you can delete it by hovering over the Action column of a launched Orchestration and selecting Delete from the pop-up. You cannot delete a launched Orchestration.

Was this article helpful?
0 out of 0 found this helpful
Have more questions? Submit a request

Comments