SKAN Advanced Analytics FAQ

SKAN Advanced Analytics is a series of features offered by Singular to improve SKAdNetwork reporting capabilities. Singular uses proprietary data-science models to supplement partial SKAdNetwork data with modeled (extrapolated) metrics.

SKAN Advanced Analytics metrics are available in the SKAdNetwork Report (see SKAdNetwork Report FAQ) and the SKAdNetwork Raw Data Report (see SKAdNetwork Raw Data FAQ).

Update (July 2023):

  • Modeled cohort metrics are now available for hybrid SDK+S2S Singular integrations as well as pure Singular SDK integrations.
  • Modeled cohort metrics are now based on MMP data for improved accuracy. This means they are available even when there is no revenue model defined.

Having trouble using Advanced Analytics? Seeing unexpected data in your reports?
See the troubleshooting section below.

Screenshot 2023-09-12 at 18.55.46.png

FAQ

What are modeled metrics?
Modeled SKAN metrics are offered by Singular to supplement partial SKAdNetwork data. Modeled metrics are extrapolated using proprietary data science models from existing data, including SKAN postbacks and IDFV-based tracker data.
Why are modeled metrics needed?

For most iOS 14+ app installs, any information about user behavior - including post-install interactions with the app and how much revenue they brought in - is encoded in the conversion value that is included in Apple's SKAdNetwork postback.

However, not all postbacks contain a conversion value. Apple censors the conversion value for privacy reasons when the number of installs for a campaign doesn't reach a certain threshold. This results in installs for which we don't have any data apart from the app and the attributed network.

To bridge this gap and offer you a fuller picture of the results of your marketing efforts, Singular offers modeled metrics, which are extrapolated from the existing data.

What can I do if the data looks inaccurate or the confidence intervals are high?

To improve the accuracy of modeled SKAN cohort metrics, you can try the following:

  1. If Singular is offering an optimized model, use it. Optimized models are offered if they are believed to improve the accuracy of your results significantly (learn more in the Optimized Models FAQ).  To check, go to the SKAdNetwork > Model Configuration page and select an app. If an optimized model is available for the app, you will see a message in green.

    skan5.png

  2. Use models with better predictive power. Typically, this means mixed models, which take into account both revenue and other user behavior.
  3. Increase the size of the campaigns (run fewer campaigns). Campaigns with more installs are easier to model for.

Modeled Metrics in the SKAdNetwork Raw Data Report

How does Modeled Conversion Value Count work?

The Modeled Conversion Value Count estimates how many installs in a campaign had a specific conversion value

This is calculated based on the prevalence of these conversion values in the installs we do have conversion values for.

In the example below, in a campaign with 105 installs, only 35 had conversion values included in the postback. The Modeled Conversion Value Count shows how many installs had each conversion value if we assume that the partial data is representative of all the installs.

skan3.png

skan4.png

Modeled Metrics in the SKAdNetwork Report

What are modeled cohort metrics?

Modeled cohort metrics are modeled metrics that are cohort-based. Currently, Singular offers cohort-based estimates of revenue and ROI for SKAN campaigns. 

Modeled cohort metrics are calculated based on the known revenue data gathered through the MMP (tracker). The models also take advantage of the revenue buckets defined in the conversion model, if they exist. Optimized revenue buckets will improve the accuracy of the modeled metrics.

Advantages of modeled revenue and ROI over the SKAN Revenue metric:

  • Modeled revenue and ROI are not limited by SKAN measuring periods, privacy thresholds, etc.
  • Modeled revenue and ROI are available even without a revenue-type SKAN model.

Limitations of modeled revenue and ROI:

  • To use modeled cohort metrics, you must send revenue events to Singular from the Singular SDK embedded in your app (SDK integration or hybrid SDK+S2S integration). These metrics are not available for S2S integrations.
  • Model accuracy may change from advertiser to advertiser and from app to app.
  • The modeling is currently limited to 7-day cohorts.
How accurate are modeled cohort metrics?

The accuracy of modeled revenue and ROI varies from advertiser to advertiser and from app to app, and it is improved when there is an optimized conversion model in place. For the majority of apps, we can reach an accuracy of 88% if the apps use an optimized conversion model.

You can check the accuracy of each 7-day revenue metric by looking at the confidence interval. If the 7-day revenue is $100 and the confidence interval is +/-10, it means that we can say with a high level of confidence that the 7-day revenue is between $90 and $110.
How do Modeled Events work?

One of the modeled metrics Singular offers for SKAN is Modeled Events (modeled event counts). Modeled Events are available only for campaigns that use an Events conversion model. You can select the events you want to see counts for in the drop-down list.

Modeled events are extrapolated for the entire segment of users based on the data of those users for which Singular did receive a conversion value.

What are Events Postback Periods (P1/P2/P3 Reporting)?

postback_periods1.png

Choose one or more of the event postback periods in the SKAdNetwork Report to see modeled unique event counts based on the first, second, or third SKAN 4.0 postback (a.k.a., P1, P2, and P3).

As explained in the SKAN 4.0 FAQ (see What are the main changes in SKAN 4.0?), the three postbacks sent in the SKAN 4.0 framework contain information about separate periods in the user journey post-install. 

postback_periods2.png

While Singular has no way of knowing which P2 or P3 that we receive represents the same user as a P1, we can sum up the information from each type of postback separately, and show you the results.

The event counts will be displayed in separate columns in the report. For example, the following report shows stats for the event In-App Purchase based on P1, P2, and P3, for the chosen date range. Assuming you run the report based on estimated install date, this gives you the most complete information possible about user events triggered by a cohort of users.

postback_periods3.png

Important Usage Notes

Note: Read the below carefully to understand what the numbers in the report mean - and what they don't!

  1. The numbers represent unique events per user per postback period.

    The limitations inherent in the SKAdNetwork framework make it unfeasible to count how many times an event was triggered by a user. Therefore, our event-based conversion models, including mixed and funnel models, only let you track whether the event occurred or not during the period.

    As a result, the modeled event count metric in the SKAdNetwork report actually counts the number of users who triggered the event during that postback's measurement period.

    Example: If a user makes 1 in-app purchase during the P1 measurement period, 2 purchases during the P2 measurement period, and 2 during the P3 period, each of the postbacks sent for that user will report that an in-app purchase event has occurred. In the report, this user will be counted as 1 in the P1 column, 1 in the P2 column, and 1 in the P3 column.

    Screenshot 2023-09-18 at 19.21.16.png

  2. Don't add the numbers for P1 + P2 + P3.

    If you try to add up the numbers in the three columns, you will not get a good representation of either the total event count or the number of users who triggered the event.

    See the example above: one user made a total of 5 in-app purchases. But the report counts them as 1 in the P1 measurement period, 1 in the P2 period, and 1 in P3.

  3. To use this metric, you need a SKAN 4.0 model that includes event measurement in the postback period you want to measure. 

    For example, the following screenshot from the Model Configuration page shows a SKAN 4.0 Funnel model that includes an event called Watched_Tutorial in the second postback (P2).

    postback_periods5.png

  4. Be sure to run the report by estimated install date.

    If you run the report based on estimated install date, the P1, P2, and P3 data refers to the same cohort of users (based on our install date calculation). For example, if the chosen date range for the report is January 1-10, you will see data based on P1s sent for users who installed the app on January 1-10, P2s sent for the same users, and P3s sent for the same users.

    If you run the report based on postback date, the P1, P2, and P3 counts are not related to each other at all. The postbacks were sent on the same date, but they represent different cohorts of users.

What are the differences between SKAN Revenue and SKAN Modeled Cohort (7d) Revenue?
Metric Description Period Source of Data
SKAN Revenue Singular decodes the conversion value in your SKadNetwork postbacks based on your defined conversion model. Assuming it is a revenue model, Singular adds up the sums to show the campaign revenue. Measurement period defined in conversion model SKAdNetwork postbacks, configured revenue model
Modeled SKAN Revenue Singular estimates the revenue for the selected cohort period based on IDFV-based MMP data. 

You can select:

  • The measurement period defined in the conversion model 
  • Or 7 days
IDFV-based MMP data, SKAdNetwork postbacks, configured SKAN model
Are modeled cohort metrics available through the API?

Modeled cohort metrics are available through the Singular SKAdNetwork API in the following metrics:

  • modeled_skan_revenue 
  • modeled_skan_roi
Are modeled cohort metrics available through data destinations (ETL)?

Modeled cohort metrics are not currently included in default data destinations schemas. If you're interested in having them included in your data, contact Singular support.

Troubleshooting

Why do some apps have no modeled revenue?
If we determine that we cannot provide an accurate enough modeling, we don't show the metrics.
Why do some apps have modeled cohort metrics but no SKAN revenue?

Modeled revenue is based on MMP data. Therefore, Singular can provide the modeled revenue even if you don't use a revenue conversion model for a particular campaign or app.

Why don't I see modeled metrics for a campaign?
  • The campaign may have no conversion values at all (all of the postbacks were sent with censored conversion values), which leaves the Singular algorithm with no data to base a model on.
  • For Google campaigns: Google currently reports SKAN installs with missing conversion values as if they have the conversion value 0. Since Singular doesn't know how many conversion values are missing for any campaign, we can't show modeled metrics.
Why can't I see modeled metrics with a Country or Publisher Site ID breakdown?

Country data is unavailable in SKAdNetwork, and Publisher Site ID is unavailable whenever the conversion value is censored. To see modeled metrics/events in your report, be sure to unselect the Country and Publisher Site ID dimensions.

Why do the modeled metrics change when I change the date range for the report?

Modeled metrics are extrapolated based on the entire date range you picked for the report. This means, for example, that a weekly report with a daily breakdown may show slightly different modeled metrics than a daily report.

Why don't I see modeled cohort metrics?

SKAN cohort metrics are currently only supported for customers who have the Singular SDK integrated in their apps or who use a hybrid SDK+S2S integration.

If you're using an S2S integration, contact Singular support for implementation guidelines. 

Why is my SKAN Revenue significantly different from my Modeled SKAN Revenue?

SKAN revenue is calculated based on the buckets defined in the revenue model, using the middle of the bucket in each case. Modeled revenue is more accurate.

Large differences may be due to:

  • Defining buckets that are too large
  • Not using all the buckets available in the Revenue model
Why am I seeing different revenues in Facebook vs. Singular's reports?

Both Facebook and Singular provide model revenue based on the obfuscated conversion values. However, Singular's model is more accurate, as we have access to more information based on all your SKAN campaigns.