SKAdNetwork Advanced Analytics: FAQ and Troubleshooting

SKAN Advanced Analytics is a series of features being developed by Singular to improve SKAdNetwork reporting capabilities using propriety data science models.

SKAN Advanced Analytics features are available in SKAN Reports and SKAN Raw Data.

 

Troubleshooting

Why don't I see modeled metrics for a campaign?
  • The campaign may have no conversion values at all (all of the postbacks were sent with censored conversion values), which leaves the Singular algorithm with no data to base a model on.
  • For Google campaigns: Google currently reports SKAN installs with missing conversion values as if they have the conversion value 0. Since Singular doesn't know how many conversion values are missing for any campaign, we can't show modeled metrics.
Why doesn't the Modeled Revenue always equal Revenue / CV Ratio?

Modeled metrics are calculated per campaign (or sub campaign, where available). If you run a higher-level report, e.g., only App and Source, the modeled metrics are not recalculated at the App + Source level but rather added up based on the modeled metrics for the relevant campaigns. This is why the Modeled Revenue metric (or other modeled metrics) may be slightly different from the SKAN Metric divided by Conversion Value Ratio.

Why can't I see modeled metrics with a Country or Publisher Site ID breakdown?

Country data is unavailable in SKAdNetwork, and Publisher Site ID is unavailable whenever the conversion value is censored. To see modeled metrics/events in your report, be sure to unselect the Country and Publisher Site ID dimensions.

Why do the modeled metrics change when I change the date range for the report?

Modeled metrics are extrapolated based on the entire date range you picked for the report. This means, for example, that a weekly report with a daily breakdown may show slightly different modeled metrics than a daily report.

FAQ

Modeled Metrics

What are modeled metrics?

For SKAdNetwork campaigns with partial data (due to censored conversion values), Singular can show modeled (extrapolated) metrics for the entire campaign.

Modeled metrics are derived from the installs for which we do have data.

For example:

  • A SKAN campaign has 50 installs, out of which 25 came with a conversion value (the Conversion Value Ratio is 50%).
  • Based on the existing conversion values, the SKAN revenue is $30. 
  • The modeled revenue for the campaign, based on the assumption that the installs with censored conversion values behave like the installs with available conversion values, is $60.

image3.png

If you run a SKAN report at the highest granularity, you can see that the Modeled Revenue is equal to SKAN Revenue divided by the Conversion Value Ratio. The same goes for other modeled metrics and events.

Note that if you run a higher-level report, e.g., only App and Source, the modeled metrics are not recalculated at the App + Source level but rather added up based on the modeled metrics for the relevant campaigns.

This is why the modeled metric may be slightly different from the result of dividing the SKAN metric by the Conversion Value Ratio.

What metrics can Singular model?

Revenue (SKAdNetwork Reports)

Singular offers Modeled SKAN Revenue (relevant only for campaigns with a Revenue model).

For example: 

  • A SKAN campaign has 50 installs, out of which 25 came with a conversion value (the Conversion Value Ratio is 50%).
  • Based on the existing conversion values, the SKAN revenue is $30. 
  • The modeled revenue for the campaign, based on the assumption that the installs with censored conversion values behave like the installs with available conversion values, is $60.

image3.png

Events (SKadNetwork Reports)

Singular offers Modeled SKAN Events (relevant only for campaigns with an Events model). You can select the events you want to see modeled counts for out of a drop-down list.

Conversion Value Count (SKAdNetwork Raw Data Report)

The Modeled Conversion Value Count estimates how many installs in a campaign had a specific conversion value. This is calculated based on the prevalence of these conversion values in the installs we do have conversion values for.

image2.png

image1.png

In the example above, in a campaign with 105 installs, only 35 had conversion values included in the postback. The Modeled Conversion Value Count shows how many installs had each conversion value if we assume that the partial data is representative of all the installs.

Why are modeled metrics needed?

For most iOS 14+ app installs, any information about user behavior - including post-install interactions with the app and how much revenue they brought in - is encoded in the conversion value that is included in Apple's SKAdNetwork postback.

However, not all postbacks contain a conversion value. Apple censors the conversion value for privacy reasons when the number of installs for a campaign doesn't reach a certain threshold. This results in installs for which we don't have any data apart from the app and the attributed network.

To bridge this gap, Singular offers modeled metrics, which are extrapolated for all installs in a certain campaign or sub campaign based on the installs that do have a conversion value.

What is the confidence interval?

To get more information about the reliability of modeled metrics and events, you can choose to see the confidence interval. Singular's algorithm indicates with a high level of confidence that the real value is within this range. 

The value displayed as the modeled metric is the most highly probable value within the confidence interval.

Modeled Cohort Metrics

What are modeled cohort metrics?

In SKAdNetwork campaigns, depending on how you define your conversion model, you typically get information about user behavior and revenue in the 1-3 days after the install.

Modeled cohort metrics offer you an estimate of the revenue and ROI garnered from these campaigns in the 7 days after the install.

Notes:

  • To use modeled cohort metrics, you must be sending revenue events to Singular from the Singular SDK embedded in your app (not through an S2S or hybrid implementation).
  • Model accuracy may change from advertiser to advertiser and from app to app.
  • The modeling is currently limited to 7 days (no 30-days cohorts, etc.)
  • The modeling is currently limited to revenue.
Are modeled cohorts based on the estimated install date or the postback date?

Currently, cohorts are based on the estimated install date.

How are modeled cohort metrics calculated?

To model 7-day revenue for SKAdNetwork campaigns, we compare known SKAdNetwork conversion values for the campaign with any information we get from IDFV-based tracker attribution for the same campaign. We can then model, for a given conversion value, what revenue can be gained for that user after 7 days.

How accurate is modeled 7-day revenue?

The accuracy of modeled 7-day revenue varies from advertiser to advertiser and from app to app, and it relies strongly on having an optimized conversion model. For the majority of apps, we can reach an accuracy of 88% if the apps use an optimized conversion model.

You can check the accuracy of each 7-day revenue metric by looking at the confidence interval. If the 7-day revenue is $100 and the confidence interval is +/-10, it means that we can say with a high level of confidence that the 7-day revenue is betwee $90 and $110.

What can I do if the data looks inaccurate or the confidence intervals are high?

To improve the accuracy of modeled SKAN cohort metrics, you can try the following:

  1. If Singular is offering an optimized model, use it. Optimized models are offered if they are believed to improve the accuracy of your results significantly (learn more in the Optimized Models FAQ).

    To check, go to the SKAdNetwork > Model Configuration page and select an app. If an optimized model is available for the app, you will see a message in green.

    Screen_Shot_2022-01-27_at_16.52.03.png

  2. Use models with better predictive power. Typically, this means mixed models, which take into account both revenue and other user behaviour.
  3. Increase the size of the campaigns (run fewer campaigns). Campaigns with more installs are easier to model for.
Why do some apps have no modeled 7-day revenue?

If we determine that we cannot provide an accurate enough modeling, we don't show the metrics.

Why do some apps have modeled 7-day revenue but no SKAN revenue?

We can model revenue even if you don't use a revenue conversion model.

What is the difference between SKAN Revenue, Modeled SKAN Revenue, and Modeled Cohort (7d) Revenue?
Metric Description Period Source of Data
SKAN Revenue We decode the conversion value in your SKadNetwork postbacks based on your defined conversion model, and assuming it is a revenue model, we add up the sums to show the campaign revenue. Measurement period defined in conversion model SKAdNetwork postbacks
Modeled SKAN Revenue We estimate the total campaign revenue by extrapolating from the known SKAN Revenue to installs that did not have a conversion value in the postback. Measurement period defined in conversion model SKAdNetwork postbacks
Modeled SKAN Revenue 7d We model the 7-day revenue based on SKAdNetwork postbacks and IDFV-based attribution data 7 days SKAdNetwork postbacks, IDFV-based attribution data
Are modeled cohort metrics available through the API?

The modeled cohort metrics are available through the Singular SKAdNetwork API as:

  • modeled_cohorted_skan_revenue 
  • modeled_cohorted_skan_roi
Are modeled cohort metrics available through data destinations (ETL)?

Modeled cohort metrics are not currently included in default data destinations schemas. If you're interested in having them included in your data, contact Singular support.