Content drop announcements might be lowering your KPIs

Most games will announce new content when it is available. Users are greeted with splashy messages explaining the exciting new content. Developers feel a sense of accomplishment of pushing through a new release. But just how do these announcements affect your KPIs?

From our many customer implementations, we will examine the most common method of content announcements, a wide message sent to all active players, usually limited to once or twice per user.

These active players include both noobs and veterans. Many game developers are probably doing some flavor of this type of announcement.

Comparing post message revenue

Figure 1 below shows the post-announcement revenue for equally sized control and variant group of an Android content drop announcement. This is for a relatively mature mobile game which features regular weekly content drops.  The orange colored variant seems to be performing approximately 5% better than the control group. Usually any revenue improvement in the 5% range is cause for celebration; any such improvement from a simple pop-up message justifies breaking out the champagne.

     

 Figure 1 - Overall revenue comparison between equally sized Android control and announcement groups. y-axis baseline is 0

However, "if it's too good to be true" applies here. By segmenting the users into noobs and veterans, the post-announcement revenue shows a much different picture. Figure 2 below shows the revenue comparison between noobs in the control and variant group. Clearly the content announcement is having a negative impact on the noobs.  The content drop in this case can be immediately consumed by noobs as well as veterans.  There are no prerequisities, like level attainment, to enjoy this new content.  So it was not immediately intuitive that the annoucement should be limited to any user segment.  But the effect is clearly negative for noobs.

      

 

Figure 2 - Revenue comparison between equally sized Android control and announcement groups, but limited to noobs. y-axis baseline is 0.


To make things more interesting, take a look at Figure 3, which is the IOS version of Figure 2. Using the same segmentation of noobs and veterans, the content announcement appears to improve revenues for IOS.

     

 

Figure 3 - Overall revenue comparison between equally sized IOS control and announcement groups, but limited to noobs. y-axis baseline is 0

 

Targeted announcements

A common objection to targeting any promotions and messages is that players will find out and complain on the community boards. This is understandable if you are offering different discounts and prices to different users. But targeting content drop notifications to different user segments, you might actually be streamlining and improving the players' overall experience.

Takeaways

1) Do not assume the impact of any "wide" announcements will be positive.
2) Use a holdout control group of users who do not receive the content announcement.
3) Compare overall impact as well as sub-segments of users.

 

Sonamine uses machine learning scores to execute live ops and lifecycle campaigns, email This email address is being protected from spambots. You need JavaScript enabled to view it.to learn more.

© 2009-2022 Sonamine LLC