Downloading Expansion Files in Xamarin.Android

Google Play currently requires that our APK file be no more than 50MB. For most apps, this is plenty of space for all the app’s code and assets. However, some apps need more space for high-fidelity graphics, media files, or other large assets. Google Play allows us to attach two large expansion files that supplement our APK.

Google Play hosts the expansion files for our app and serves them to the device at no cost to us. The expansion files are saved to the device’s shared storage location (the SD card or USB-mountable partition, also known as the “external” storage) where our app can access them. On most devices, Google Play downloads the expansion file(s) at the same time it downloads the APK, so our app has everything it needs when the user opens it for the first time. In some cases, however, our app must download the files from Google Play when our app starts.

We can obtain the library from NuGet or we can compile the code directly from GitHub.

The Expansion Files

Expansion files are treated as opaque binary blobs (obb) and each may be up to 2GB in size. Android does not perform any special processing on these files after they are downloaded – the files can be in any format that is appropriate for the app. Conceptually, each expansion file plays a different role:

  • The main expansion file is the primary expansion file for additional resources
  • The patch expansion file is optional and intended for small updates to the main expansion file

When Google Play downloads our expansion files to a device, it saves them to the system’s shared storage location. To ensure proper behavior, we must not delete, move, or rename the expansion files. In the event that our app must perform the download from Google Play itself, we must save the files to the exact same location. Any updates to the expansion files overwrite the existing files.

The specific location for our expansion files is:
[shared-storage]/Android/obb/[package-name]/

  • [shared-storage] is the path to the shared storage space, available from Environment.ExternalStorageDirectory
  • [package-name] is our app’s Java-style package name, available from PackageName

Each expansion file we upload will be renamed to match the pattern:
[main|patch].[expansion-version].[package-name].obb

  • [main|patch] specifies whether the file is the main or patch expansion file. There can be only one main file and one patch file for each APK
  • [expansion-version] is an integer that matches the version code of the APK with which the expansion is first associated
    “First” is emphasized because although the Developer Console allows we to reuse an uploaded expansion file with a new APK, the expansion file’s name does not change – it retains the version applied to it when we first uploaded the file*
  • [package-name] the Java-style app package name

For example, suppose our APK version is 23 and our package name is com.example.app. If we upload a main expansion file, the file is renamed to main.23.com.example.app.obb.

Getting Things Ready

We will need to obtain the base64-encoded RSA public key. More information on this in the Getting Things Ready section of this post.

Implementing the Downloader

Once we have installed the library and have our key, we need to ensure that the app has the appropriate permissions to access Play, the licensing service, the Internet, the network state and the external storage:

    [assembly: UsesPermission(Manifest.Permission.Internet)]
    [assembly: UsesPermission(Manifest.Permission.WriteExternalStorage)]
    // required to poll the state of the network connection
    // and respond to changes
    [assembly: UsesPermission(Manifest.Permission.AccessNetworkState)]
    // required to keep CPU alive while downloading files
    [assembly: UsesPermission(Manifest.Permission.WakeLock)]
    [assembly: UsesPermission(Manifest.Permission.AccessWifiState)]
    [assembly: UsesPermission("com.android.vending.CHECK_LICENSE")]

Once we have permission, we can then create a DownloaderService service that will manage the downloads of the expansion files. In addition to the download management, the service will create an alarm to resume downloads and build and update a notification that displays download progress. We can create the service by simply overriding three properties:

    [Service]
    public class SampleDownloaderService : DownloaderService
    {
        // the API key used to access Play
        protected override string PublicKey
        {
            get { return "Base64 API Public Key"; }
        }

        // the salt used to encrypt the cached server response
        protected override byte[] Salt
        {
            get { return new byte[] { ... }; }
        }

        // the name of the broadcast reciever that will resume 
        // the downloads if the service is stopped
        protected override string AlarmReceiverClassName
        {
            get { return "namespace.ClassName"; }
        }
    }

Finally, we will need to implement a BroadcastReciever that will be used to resume any downloads if the service is stopped:

    [BroadcastReceiver(Exported = false)]
    public class SampleAlarmReceiver : BroadcastReceiver
    {
        public override void OnReceive(Context context, Intent intent)
        {
            // start the service if necessary
            DownloaderService.StartDownloadServiceIfRequired(
                context, intent, typeof(SampleDownloaderService));
        }
    }

Starting the Check

Once we have the download service and the broadcast receiver, we can then start the downloading. Before we start any downloads, we should make sure that we have not already downloaded all the files:

    // get a list of all the downloaded expansion files
    var downloads = DownloadsDatabase.GetDownloads();
    if (!downloads.Any())
    {
        // start the download as nothing is here
    }
    foreach (var file in downloads)
    {
        if (!Helpers.DoesFileExist(this, file.FileName, file.TotalBytes, false))
        {
            // start the download as this file is incomplete
            break;
        }
    }

If the expansion files are not there or not complete, we need to start the download from the OnCreate method. In order to start the service, we need to provide an Intent that will be used to launch the app when the user taps the notification:

    // build the intent that launches this activity.
    Intent launchIntent = this.Intent;
    var intent = new Intent(this, typeof(SuperSimpleActivity));
    intent.SetFlags(ActivityFlags.NewTask | ActivityFlags.ClearTop);
    intent.SetAction(launchIntent.Action);
    if (launchIntent.Categories != null)
    {
        foreach (string category in launchIntent.Categories)
        {
            intent.AddCategory(category);
        }
    }

    // build PendingIntent used to open this activity when user 
    // taps the notification.
    PendingIntent pendingIntent = PendingIntent.GetActivity(
        this, 0, intent, PendingIntentFlags.UpdateCurrent);

Now that we have the Intent, we can start the service. Starting the service will return a result that indicates whether the service was started:

    // request to start the download
    var startResult = DownloaderService.StartDownloadServiceIfRequired(
        this, pendingIntent, typeof(SampleDownloaderService));

If the service was started, we obtain an IDownloaderServiceConnection to the service which will be used to communicate with the service from the activity:

    // the DownloaderService has started downloading the files
    if (startResult != DownloadServiceRequirement.NoDownloadRequired)
    {
        // create the connection to the service so that we can show progress.
        // when creating the marshaller, we pass in the IDownloaderClient
        // that will be used to handle the updates from the service
        connection = ClientMarshaller.CreateStub(
          this, typeof(SampleDownloaderService));
    }
    else
    {
        // all files have finished downloading already
    }

Once we have started the downloader and created a connection, we need to connect and disconnect in line with the activity’s lifecycle. In the OnResume method of the activity, we connect to the service:

    if (connection != null)
    {
        connection.Connect(this);
    }

And, in the OnStop method, we disconnect from the service:

    if (connection != null)
    {
        connection.Disconnect(this);
    }

Receiving Download Progress Updates

As the download progresses, we will receive updates such as the number of bytes downloaded, the download speed and various network states. We can use all of this to display information on the user interface. This is all in addition to the notification that is automatically created and managed by the service.

In order to receive updates, we need to get hold of an IDownloaderService from the IDownloaderServiceConnection. To do this, we have to implement the IDownloaderClient somewhere, such as on the activity:

    public class SuperSimpleActivity : Activity, IDownloaderClient
    {
        public void OnServiceConnected(Messenger m)
        {
            // create the proxy that is used to communicate with the service
            service = ServiceMarshaller.CreateProxy(m);
            // let the service know about us and request an update
            service.OnClientUpdated(connection.GetMessenger());
        }
        public void OnDownloadProgress(DownloadProgressInfo progress)
        {
            // handle download progress updates
        }
        public void OnDownloadStateChanged(DownloaderState newState)
        {
            // handle download states, such as completion or pause
        }
    }

Managing the Service

Once we have the service started and the download going, we may want to be able to pause the download. We can request the download be paused using the IDownloaderService:

    service.RequestPauseDownload();

Similarly, we can resume a download:

    service.RequestContinueDownload();

We may also want to change various properties, such as whether to download over mobile or not when the download is paused due to Wi-Fi being unavailable. To do this we use the SetDownloadFlags method on the IDownloaderService:

    public void OnDownloadStateChanged(DownloaderState newState)
    {
        if (newState == DownloaderState.PausedNeedCellularPermission ||
            newState == DownloaderState.PausedWifiDisabledNeedCellularPermission)
        {
            // let the service know that it can download over mobile
            service.SetDownloadFlags(ServiceFlags.FlagsDownloadOverCellular);
            // resume the download
            service.RequestContinueDownload();
        }
    }

Reading the Expansion Files

Because the expansion files are saved to a specific location, namely [shared-storage]/Android/obb/[package-name]/, we could read the files using any file means available.

If we must unpack the contents of our expansion files, do not delete the .obb expansion files afterwards and do not save the unpacked data in the same directory. We should save our unpacked files in the directory specified by GetExternalFilesDir(). However, if possible, it’s best if we use an expansion file format that allows we to read directly from the file instead of requiring we to unpack the data. The reason for this is that the expansion files will now exist twice on the device.

Using a ContentProvider

One way in which we can read the expansion files is to make use of a content provider that can read them. In the library, there is a special content provider, the ApezProvider, that can read uncompressed zip files. If all the expansion resources are bundled in an uncompressed, storage zip archive, this provider allows access to the individual resources without having to first extract them.

Using this provider is simple and easy to implement:

    [ContentProvider(new[] { ContentProviderAuthority }, Exported = false)]
    [MetaData(ApezProvider.MetaData.MainVersion, Value = "14")]
    [MetaData(ApezProvider.MetaData.PatchVersion, Value = "14")]
    public class ZipFileContentProvider : ApezProvider
    {
        public const string ContentProviderAuthority = "expansiondownloader.sample.ZipFileContentProvider";

        protected override string Authority
        {
            get { return ContentProviderAuthority; }
        }
    }

The [MetaData] attributes let the provider know what version of the expansion files to load. For example, if we have uploaded the app with a version number of 5 and new expansion files, the version we place in the attributes will be 5. When we update the app, we may choose to use the same expansion files. So, our new app version will be 6, but because we selected the old expansion files, they are still version 5. Thus, the provider will still use the version numbers of 5 in the attributes.

If we want to access a file in this provider, we can use a Uri to the resource in the provider:

    var uri = Uri.Parse(string.Format(
        "content://{0}/relative/path/to/movie.mp4",
        ZipFileContentProvider.ContentProviderAuthority));
    videoView.SetVideoURI(uri);

It is important to note that the provider uses file descriptors and cannot support reading compressed expansion files.

Using the Zip Archive

Another way to access the files is to access them directly by filename. We can use the ApkExpansionSupport type to obtain the files stored in each expansion file:

    var files = ApkExpansionSupport.GetApkExpansionZipFile(context, 14, 14);
    var entry = files.GetEntry("relative/path/to/file.extension");
    using (var zip = new ZipFile(entry.ZipFileName))
    using (var stream = zip.ReadFile(entry))
    {
        // process the stream of the contained file
    }

Files accessed this way can be stored in a compressed format as a decompression stream is provided. This stream can be processed using typical .NET means, such as providing this stream to a deserializer or some other reader.

The ApkExpansionSupport.GetApkExpansionZipFile method returns a combined collection of all the items in either of the expansion files. We can then query this for a specific item using the GetEntry method on the collection. We pass a relative path to the item in expansion file. Once the entry is returned, we can then create a ZipFile using the path of the entry’s containing expansion file. And, after we have the zip file, we read the stream of the item inside using the ReadFile method.

Testing the Downloader

Testing the download manager is very similar to testing the licensing. More information on this in the Testing the Licensing section of this post. The downloader performs the license check internally, and in the response, it receives the expansion files URIs from Play. It then uses these to initiate the download.

In order to be able to download the expansion files, we have to have uploaded them when updating the app. The first upload does not allow the expansion files to be added, but we can just re-upload the same package twice. On the second time, we can upload the expansion files with the actual app package, remembering to increase the version number beforehand.

We can start testing our app by checking its ability to read the expansion files. We can do this by placing and naming the files just as the downloader would. Because the downloader always places the files at the [shared-storage]/Android/obb/[package-name]/ location, with the name [main|patch].[expansion-version].[package-name].obb, we just have to place our files there. By skipping the downloading, we don’t have to upload the app first.

Once we are sure that the app can access and use the expansion files, we can upload the files with the app to Play and publish to any channel to test the download. Any channel can be used, including Alpha and Beta.

Important Things to Remember

Testing the expansion files, from downloading to reading, requires that we have the various bits in place:

  1. The app version on the device must be the same as the app that is on the store
  2. Make sure that the app is indeed published, and not in draft
  3. Ensure that the Expansion files have been associated with the app
  4. Make sure that the response in “Settings” is set to “RESPOND_NORMALLY” as this is the only response that returns the expansion files
  5. Make sure the app has all the required permissions, there are 6 at least

Play Licensing in Xamarin.Android

Google Play offers a licensing service that lets us enforce licensing policies for applications that us publish on Google Play. With Google Play Licensing, our application can query Google Play at runtime to obtain the licensing status for the current user, then allow or disallow further use as appropriate.

The Google Play Licensing service is primarily intended for paid applications that wish to verify that the current user did in fact pay for the application on Google Play. However, any app (including free apps) may use the licensing service to initiate the download of an APK expansion file.

Getting Things Ready

In order to add licensing to our app, all we will need is the licensing verification library. We can get this from NuGet.org or the Component Store (coming soon), or, we can build the source using the GitHub repository.

After adding the library, we need to get hold of our API key for this app:

  1. Browse to the Google Play Developer Console
  2. Select “All Applications” from the sidebar
  3. Select the app we want to implement licensing for
    • If we are going to create a new app, select “Add new application”
    • Enter a name for the app
    • Select “Prepare store listing”
    • Enter the required details for publishing
  4. Select “Services & APIs” from the left menu
  5. Scroll to the “Licensing & In-app billing” section
  6. Under the “Your license key for this application” heading, copy the base64-encoded RSA public key
  7. Paste it into a string constant in the app code:
    const string ApiKey = "XXX";

All Applications in Play
Public Key

Adding the License Handlers

Once we have installed the library and have our key, we need to ensure that the app has the appropriate permissions to access Play and the licensing service:

    [assembly: UsesPermission("com.android.vending.CHECK_LICENSE")] 

Once we have permission, We can then implement the ILicenseCheckerCallback interface. This can be implemented on the activity, but does not have to:

    public class MainActivity : Activity, ILicenseCheckerCallback
    {
      public void Allow(PolicyServerResponse response)
      {
        // Play has determined that the app is owned,
        // either purchased or free
      }
      public void DontAllow(PolicyServerResponse response)
      {
        // Play has determined that the app should not be available to the user,
        // either because they haven't paid for it or it is not a valid app

        // However, there may have been a problem when Play tried to connect,
        // so if this is the case, allow the user to try again
        if (response == PolicyServerResponse.Retry)
        {
          // try the check again
        }
      }
      public void ApplicationError(CallbackErrorCode errorCode)
      {
        // There was an error accessing the license
      }
    }

Starting the Check

Once we have implemented the interface, all we need to do now is start the check. There are two basic methods provided in order do this, one with caching and one without.

Using StrictPolicy

To make things easier to start off with, I will first demonstrate the one without caching, the StrictPolicy:

    // create the policy we want to use
    var policy = new StrictPolicy();
    // instantiate a checker, passing a Context, an IPolicy and the Public Key
    var checker = new LicenseChecker(this, policy, "Base64 Public Key");
    // start the actual check, passing the callback
    checker.CheckAccess(this);

As soon as the check has completed, either with an error or successfully, one of the methods on the callback will be called, either Allow, DontAllow or ApplicationError.

Allow will receive a Licensed response
DontAllow will receive a NotLicensed response
ApplicationError will have the reason for the error, such as NotMarketManaged, InvalidPublicKey or some other reason.

Using ServerManagedPolicy

Although checking with Play each time the app launches is not a problem, doing so requires additional time and resources before the app can start. Usually, we can use the one with caching, the ServerManagedPolicy policy. This is very much the same as the StrictPolicy, but with an additional step to provide an IObfuscator to store the response:

    // create a device-unique identifier to prevent other devices from decrypting the responses
    string deviceId = Settings.Secure.GetString(ContentResolver, Settings.Secure.AndroidId);
    // create a app-unique identifer to prevent other apps from decrypting the responses
    var appId = this.PackageName;
    // create a random salt to be used by the AES encryption process
    byte[] salt = new byte[] { 46, 65, 30, 128, 103, 57, 74, 64, 51, 88, 95, 45, 77, 117, 36, 113, 11, 32, 64, 89 };

    // create the obfuscator that will read and write the saved responses, 
    // passing the salt, the package name and the device identifier
    var obfuscator = new AesObfuscator(salt, appId, deviceId);
    // create the policy, passing a Context and the obfuscator
    var policy = new ServerManagedPolicy(this, obfuscator);
    // create the checker
    var checker = new LicenseChecker(this, policy, Base64PublicKey);
    // start the actual check, passing the callback
    checker.CheckAccess(this); 

As soon as the checker has returned and we know that we can start the application, we should destroy the cecker in order to free up resources and close connections:

    // free resources and close connections
    checker.OnDestroy();

Testing the Licensing

The last thing that is needed is testing. To do this, we have to be sure that we have uploaded and published the app to Play. Publishing to any of the channels, including Alpha and Beta, will work.
In order for the Alpha or Beta channels to be used on devices other than the publisher’s device, those people have to be added to the Alph or Beta testers group.
If we want to test different responses that our app may receive from Play, we can select the desired response from the settings:

  1. Select “Settings” from the sidebar
  2. Select “Account details” from the left menu
  3. Scroll dow to the “License Testing” section
  4. Select the desired response from the drop down titled “License Test Response”
    • For other testers, make sure we enter their Google account email address in the text area above titled “Gmail accounts with testing access”

Custom License Responses

Important Things to Remember

Testing licensing is easy to do, provided we have all the required bits in place. Here are some common things that we may have to check:

The Play Store

  1. The app version on the device must be the same as the app that is on the store
  2. Provide enough time for the app to appear on the store, this can be determined by a little exclamation sign next to the app title when viewing the app details
  3. Make sure that the app is indeed published, and not in draft

For Other Testers

  1. Ensure that the testers have been added to the Alpha or Beta testers group
  2. If we are testing custom responses, make sure that their emails are added to the “Gmail accounts with testing access” text area in “Settings”

For The App

  1. Make sure we have the Android permission
    com.android.vending.CHECK_LICENSE
  2. Ensure that the app has the same version code/number and package name as that which is on the store