Lecture 10 Files and Permissions

This lecture discusses how to working with files in Android. Using the file system allows us to have persistent data storage in a more expansive and flexible manner than using the SharedPreferences discussed in the previous lecture (and as a supplement to ContentProvider databases).

This lecture references code found at https://github.com/info448/lecture10-files. In order to demonstrate all of the features discussed in this lecture, your device or emulator will need to be running API 23 (6.0 Marshmallow) or later.

10.1 File Storage Locations

Android devices split file storage into two types: Internal storage and External storage. These names come from when devices had built-in memory as well as external SD cards, each of which may have had different interactions. However, with modern systems the “external storage” can refer to a section of a phone’s built-in memory as well; the distinctions are instead used for specifying access rather than physical data location.

  • Internal storage is always accessible, and by default files saved internally are only accessible to your app. Similarly, when the user uninstalls your app, the internal files are deleted. This is usually the best place for “private” file data, or files that will only be used by your application.

  • External storage is not always accessible (e.g., if the physical storage is removed), and is usually (but not always) world-readable. Normally files stored in External storage persist even if an app is uninstalled, unless certain options are used. This is usually used for “public” files that may be shared between applications.

When do we use each?34 Basically, you should use Internal storage for “private” files that you don’t want to be available outside of the app, and use External storage otherwise.

  • Note however that there are publicly-hidden External files—the big distinction between the storage locations is less visibility and more about access.

In addition, both of these storage systems also have a “cache” location (i.e., an Internal Cache and an External Cache). A cache is “(secret) storage for the future”, but in computing tends to refer to “temporary storage”. The Caches are different from other file storage, in that Android has the ability to automatically delete cached files if storage space is getting low. However, you can’t rely on the operating system to do that on its own in an efficient way, so you should still delete your own Cache files when you’re done with them! In short, use the Caches for temporary files, and try to keep them small (less than 1MB recommended).

  • The user can easily clear an application’s cache as well through the operating system’s UI.

In code, using all of these storage locations involve working with the File35 class. This class represents a “file” (or a “directory”) object, and is the same class you may be familiar with from Java SE.

  • We can instantiate a File by passing it a directory (which is another File) and a filename (a String). Instantiating the file will create the file on disk (but empty, size 0) if it doesn’t already exist.

  • We can test if a File is a folder with the .isDirectory() method, and create new directories by taking a File and calling .mkdir() on it. We can get a list of Files inside the directory with the listFiles() method. See the API documentation for more details and options.

The difference between saving files to Internal and External storage, in practice, simply involves which directory you put the file in! This lecture will focus on working with External storage, since that code ends up being a kind of “super-set” of implementation details needed for the file system in general. It will indicate what changes need to be made for interacting with Internal storage.

  • In particular, this lecture will walk through implementing an application that will save whatever the user types into an text field to a file.

Because a device’s External storage may be on removable media, in order to interact with it in any way we first need to check whether it is available (e.g., that the SD card is mounted). This can be done with the following check (written as a helper method so it can be reused):

//java
public static boolean isExternalStorageWritable() {
  String state = Environment.getExternalStorageState();
  if (Environment.MEDIA_MOUNTED.equals(state)) {
    return true;
  }
  return false;
}
//kotlin
fun isExternalStorageWritable(): Boolean {
    return Environment.getExternalStorageState() == Environment.MEDIA_MOUNTED
}

10.2 Permissions

Directly accessing the file system of any computer can be a significant security risk, so there are substantial protections in place to make sure that a malicious app doesn’t run roughshod over a user’s data. In order to work with the file system, we first need to discuss how Android handles permissions in more detail.

One of the most import aspect of the Android operating system’s design is the idea of sandboxing: each application gets its own “sandbox” to play in (where all its toys are kept), but isn’t able to go outside the box and play with someone else’s toys. In Android, the “toys” (components) that are outside of the sandbox are things that would be impactful to the user, such as network or file access. Apps are not 100% locked into their sandbox, but we need to do extra work to step outside.

  • Sandboxing also occurs at a package level, where packages (applications) are isolated from packages from other developers; you can use certificate signing (which occurs as part of the build process automatically) to mark two packages as from the same developer if you want them to interact.

  • Additionally, Android’s underlying OS is Linux-based, so it actually uses Linux’s permission system under the hood (with user and group ids that grant access to particular files or processes).

In order for an app to go outside of its sandbox (and use different components), it needs to request permission to leave. We ask for this permission (“Mother may I?”) by declaring out-of-sandbox usages explicitly in the Manifest, as we’ve done before with getting permission to access the Internet, send SMS messages, or access the User Dictionary.

However, the Android permissions we can ask for are divided into two categories: normal and dangerous:

  • Normal permissions are those that may impact the user (so require permission), but don’t pose any serious risk. They are granted by the user at install time; if the user chooses to install the app, permission is granted to that app. See this list36 for examples of normal permissions. INTERNET is a normal permission.

  • Dangerous permissions, on the other hand, have the risk of violating a user’s privacy, or otherwise messing with the user’s device or other apps. These permissions also need to be granted at install time. But IN ADDITION, since API 23 (Android 6.0 Marshmallow), users additionally need to grant dangerous permission access at runtime, when the app tries to actually invoke the “permitted” dangerous action.

    • The user grants permission via a system-generated pop-up dialog. Note that permissions are granted in “groups”, so if the user agrees to give you RECEIVE_SMS permission, you get SEND_SMS permission as well. See the list of permission groups.

    • When the user grants permission at runtime, that permission stays granted as long as the app is installed. But the big caveat is that the user can choose to revoke or deny privileges at any time (they do this though System settings)! Thus you have to check each time you want to access the feature if the user has granted the privileges or not—you don’t know if the user has currently given you permission, even if they had in the past.

Writing to external storage is a dangerous permission, and thus we will need to do extra work to support the Marshmallow runtime permission system.

  • In order to support runtime permissions, we need to specify our app’s target SDK to be 23 or higher AND execute the app on a device running Android 6.0 (Marshmallow) or higher. Runtime permissions are only considered if the OS supports and the app is targeted that high. For lower-API devices or apps, permission is only granted at install time.

First we still need to request permission in the Manifest; if we haven’t announced that we might ask for permission, we won’t be allowed to ask for it in the future. In particular, saving files to External storage requires android.permission.WRITE_EXTERNAL_STORAGE permission (which will also grant us READ_EXTERNAL_STORAGE access).

Before we perform a dangerous action, we can check that we currently have permission:

//java
int permissionCheck = ContextCompat.checkSelfPermission(activity, Manifest.permission.PERMISSION_NAME);
//kotlin
val permissionCheck = ContextCompat.checkSelfPermission(this, Manifest.permission.PERMISSION_NAME)
  • This function basically “looks up” whether the app has currently been granted a particular permission or not. It will return either PackageManager.PERMISSION_GRANTED or PackageManager.PERMISSION_DENIED.

If permission has been granted, great! We can go about our business (e.g., saving a file to external storage). But if permission has NOT been explicitly granted (at runtime), then we have to ask for it. We do this by calling:

//java
ActivityCompat.requestPermissions(activity, new String[]{Manifest.permission.PERMISSION_NAME}, REQUEST_CODE);
//kotlin
ActivityCompat.requestPermissions(activity, arrayOf(Manifest.permission.PERMISSION_NAME), REQUEST_CODE)
  • This method takes a Context and an array of permissions that we need access to (in case we need more than one). We also provide a request ID code (an int), which we can use to identify that particular request for permission in a callback that will be executed when the user chooses whether to give us access or not. This is the same pattern as when we sent an Intent for a result; asking for permission is conceptually like sending an Intent to the permission system!

We can then provide the callback that will be executed when the user decides whether to grant us permission or not:

//java
public void onRequestPermissionsResult(int requestCode, String permissions[], int[] grantResults) {
  switch (requestCode) {
    case REQUEST_CODE:
      if (grantResults.length > 0 && grantResults[0] == PackageManager.PERMISSION_GRANTED) {
        //have permission! Do stuff!
      }
    default:
        super.onRequestPermissionsResult(requestCode, permissions, grantResults);
  }
}
//kotlin
override fun onRequestPermissionsResult(requestCode: Int, permissions: Array<String>, grantResults: IntArray) {
    when (requestCode) {
        REQUEST_CODE -> {
            if (grantResults.size > 0 && grantResults[0] == PackageManager.PERMISSION_GRANTED) {
                //have permission! Do stuff!
            }
        }
        else -> super.onRequestPermissionsResult(requestCode, permissions, grantResults)
    }
}

We check which request we’re hearing the results for, what permissions were granted (if any—the user can piece-wise grant permissions!), and then we can react if everything is good… like by finally saving our file!

  • Note that if the user deny us permission once, we might want to try and explain why we’re asking permission (see best practices) and ask again. Google offers a utility method (ActivityCompat#shouldShowRequestPermissionRationale()) which we can use to show a rationale dialog if they’ve denied us once. And if that’s true, we might show a Dialog or something to explain ourselves—and if they OK that dialog, then we can ask to perform the dangerous action again.

10.3 External Storage

Once we have permission to write to an external file, we can actually do so! Since we’ve verified that the External storage is available, we now need to pick what directory in that storage to save the file in. With External storage, we have two options:

  • We can save the file publicly. We use the getExternalStoragePublicDirectory() method to access a public directory, passing in what type of directory we want (e.g., DIRECTORY_MUSIC, DIRECTORY_PICTURES, DIRECTORY_DOWNLOADS etc). This basically drops files into the same folders that every other app is using, and is great for shared data and common formats like pictures, music, etc.. Files in the public directories can be easily accessed by other apps, assuming the app has permission to read/write from External storage!

    • DIRECTORY_DOCUMENTS was added in API 19, and is a nice place to use.

  • Alternatively starting from API 18, we can save the file privately, but still on External storage (these files are world-readable, but are hidden from the user as media, so they don’t “look” like public files). We access this directory with the getExternalFilesDir() method, again passing it a type (since we’re basically making our own version of the public folders). We can also use null for the type, giving us the root directory.

    Since API 19 (4.4 KitKat), you don’t need permission to write to private External storage. If that’s all you’re doing, you can only specify in the Manifest that you need External storage access for versions lower than that:

    <uses-permission android:name="android.permission.WRITE_EXTERNAL_STORAGE" android:maxSdkVersion="18" />

We can actually look at the emulator’s file-system and see our files be created using adb. Connect to the emulator from the terminal using adb -s emulator-5554 shell (note: adb needs to be on your PATH). Public external files can usually be found in /storage/emulated/0/Folder (or /storage/sdcard/Folder), while private external files can be found in /storage/emulated/0/Android/data/package.name/files (these paths may vary on different devices).

Once we’ve opened up the file, we can write content to it by using the same IO classes we’ve used in Java:

  • The “low-level” way to do this is to create a a FileOutputStream object (or a FileInputStream for reading). We just pass this constructor the File to write to. We write bytes to this stream… but can get the bytes from a String by calling myString.getBytes(). For reading, we’ll need to read in all the lines/characters, and probably build a String out of them to show.

  • However, we can also use the same decorators as in Java (e.g., BufferedReader, PrintWriter, etc.) if we want those capabilities; it makes reading and writing to file a little easier.

  • In either case, remember to .close() the stream when done (to avoid memory leaks)!

//java
//writing
try {
    //saving in public Documents directory
    File dir = getExternalStoragePublicDirectory(Environment.DIRECTORY_DOCUMENTS);
    if (!dir.exists()) { dir.mkdirs(); } //make dir if doesn't otherwise exist (pre-19)
    File file = new File(dir, FILE_NAME);
    Log.v(TAG, "Saving to  " + file.getAbsolutePath());

    PrintWriter out = new PrintWriter(new FileWriter(file, true));
    out.println(textEntry.getText().toString());
    out.close();
} catch (IOException ioe) {
    Log.d(TAG, Log.getStackTraceString(ioe));
}

//reading
try {
    File dir = getExternalStoragePublicDirectory(Environment.DIRECTORY_DOCUMENTS);
    File file = new File(dir, FILE_NAME);
    if(!file.exists()) return; //e.g., if file doesn't exist yet
    BufferedReader reader = new BufferedReader(new FileReader(file));
    StringBuilder text = new StringBuilder();

    //read the file
    String line = reader.readLine();
    while (line != null) {
        text.append(line + "\n");
        line = reader.readLine();
    }

    textDisplay.setText(text.toString());
    reader.close();
} catch (IOException ioe) {
    Log.d(TAG, Log.getStackTraceString(ioe));
}
//kotlin
//writing
try {
    val dir = getExternalStoragePublicDirectory(Environment.DIRECTORY_DOCUMENTS)
    if (!dir.exists()) { dir.mkdirs() } //make dir if doesn't otherwise exist (pre-19)
    val file = File(dir, FILE_NAME)
    Log.v(TAG, "Saving to  " + file.absolutePath)

    val out = PrintWriter(FileWriter(file, true))
    out.println(textEntry.text.toString())
    out.close()
} catch (ioe: IOException) {
    Log.d(TAG, Log.getStackTraceString(ioe))
}

//reading
try {
    val dir = getExternalStoragePublicDirectory(Environment.DIRECTORY_DOCUMENTS)
    val file = File(dir, FILE_NAME)
    if (!file.exists()) return //e.g., if file doesn't exist yet
    val reader = BufferedReader(FileReader(file))
    val text = StringBuilder()

    //read the file
    var line: String? = reader.readLine()
    while (line != null) {
        text.append(line + "\n")
        line = reader.readLine()
    }

    textDisplay.text = text.toString()
    reader.close()

} catch (ioe: IOException) {
    Log.d(TAG, Log.getStackTraceString(ioe))
}

This will allow us to have our “save” button write the message to the file, and have our “read” button load the message from the file (and display it on the screen)!

10.4 Internal Storage & Cache

Internal storage works pretty much the same way as External storage. Remember that Internal storage is always private to the app. So we also don’t need permission to access Internal storage!

For Internal storage, we can use the getFilesDir() method to access to the files directory (just like we did with External storage). This method normally returns the folder at /data/data/package.name/files.

Alternatively, we can use Context#openFileOutput() (or Context#openFileInput()) and pass it the name of the file to open. This gives us back the Stream object for that file in the Internal storage file directory, without us needing to do any extra work (cutting out the middle-man!)

  • These methods take a second parameter: MODE_PRIVATE will create the file (or replace a file of the same name). Other modes available are: MODE_APPEND (which adds to the end of the file if it exists instead of erasing). MODE_WORLD_READABLE, and MODE_WORLD_WRITEABLE are deprecated.
  • Note that you can wrap a FileInputStream in a InputStreamReader in a BufferedReader.

We can access the Internal Cache directory with getCacheDir(), or the External Cache directory with getExternalCacheDir(). We almost always use the Internal Cache, because why would you want temporary files to be world-readable (other than maybe temporary images…)

  • Recommended practice is to make temporary Cache files by using the createTempFile() utility method.

And again, once you have the file, you use the same process for reading and writing as External storage.

For practice make the provided toggle support reading and writing to an Internal file as well. This will of course be different file than that used with the External switch. Ideally this code could be refactored to avoid duplication, but it gets tricky with the need for checked exception handling.

To sum up:

  • Private Internal storage: getFilesDir() or openFileOutput()
  • Public External storage: getExternalStoragePublicDirectory()
  • Private External storage: getExternalFilesDir()
  • Internal Cache: getCacheDir() and createTempFile()
  • External Cache: getExternalCacheDir()

10.5 Example: Saving Pictures

As another example of how we might use the storage system, consider the “take a selfie” system from Lecture 7. The code for taking a picture can be found in a separate PhotoActivity (which you can navigate to from the options menu).

To review: we sent an Intent with the MediaStore.ACTION_IMAGE_CAPTURE action, and the result of that Intent included an Extra that was a BitMap of a low-quality thumbnail for the image. But if we want to save a higher resolution version of that picture, we’ll need to store that image in the file system!

To do this, we’re actually going to modify the Intent we send so it includes an additional Extra: a file in which the picture data can be saved. Effectively, we’ll have our Activity allocate some memory for the picture, and then tell the Camera where it can put the picture data that it captures. (Intent envelops are too small to carry entire photos around!)

Before we send the Intent, we’re going to go ahead and create an (empty) file:

//java
File file = null;
try {
    String timestamp = new SimpleDateFormat("yyyyMMdd_HHmmss").format(new Date()); //include timestamp

    //ideally should check for permission here, skipping for time
    File dir = Environment.getExternalStoragePublicDirectory(Environment.DIRECTORY_PICTURES);
    file = new File(dir, "PIC_"+timestamp+".jpg");
    boolean created = file.createNewFile(); //actually make the file!
    Log.v(TAG, "File created: "+created);

} catch (IOException ioe) {
    Log.d(TAG, Log.getStackTraceString(ioe));
}
//kotlin
var file: File? = try {
    val timestamp = SimpleDateFormat("yyyyMMdd_HHmmss").format(Date()) //include timestamp
    val dir = Environment.getExternalStoragePublicDirectory(Environment.DIRECTORY_PICTURES)

    val file = File(dir, "PIC_$timestamp.jpg")
    file.createNewFile() //actually make the file!
    file //return
} catch (ioe: IOException) {
    Log.d(TAG, Log.getStackTraceString(ioe))
    null //return
}
  • Since we’re creating a file in external storage, make sure you ask for permission first!!

10.5.1 FileProviders

We will then specify an additional Extra to give that file’s location to the camera: if we use MediaStore.EXTRA_OUTPUT as our Extra’s key, the camera will know what to do with that! However, the extra won’t actually be the File but a Uri (recall: the “url” or location of a file). We’re not sending the file itself, but the location of that file (because it’s smaller data to fit in the Intent envelope).

Here’s the tricky part: if we try and share the absolute path to the file as a Uri, we’ll get an error because we’re “leaking” information—that is, we’re trying to let another app access a file that they may not have permission to read! We need to make sure that the Camera actually has access to where the file happens to live (it does because the file is is in public storage, but Android doesn’t know that—it just sees that you’re sharing an absolute path).

Rather than putting the file:// Uri in the Intent’s extra, we’ll need to create a content:// Uri for a ContentProvider who is able to provide files to anyone who requests them regardless of permissions (the provider grants permission to access its content). A ContentProvider explicitly is about making content available outside of a package. Specifically, a specialized ContentProvider called a FileProvider can convert a set of Files into a set of data contents (e.g., accessible with the content:// protocol) that can be used and returned and understood by other apps!

  • It’s kind of like a “File Server” in that respect!

Setting up a FileProvider is luckily not too complex, though it has a couple of steps. You will need to declare the <provider> inside the Manifest (see the guide link for an example).

<provider
    android:name="android.support.v4.content.FileProvider"
    android:authorities="edu.uw.myapp.fileprovider"
    android:exported="false"
    android:grantUriPermissions="true">
    <meta-data
        android:name="android.support.FILE_PROVIDER_PATHS"
        android:resource="@xml/filepaths" />
</provider>

The attributes you will need to specify are:

  • android:authority should be your package name followed by .fileprovider (e.g., edu.uw.myapp.fileprovider). This says what source/domain is granting permission for others to use the file.

  • The child <meta-data> tag includes an android:resource attribute that should point to an XML resource, of type xml (the same as used for your SharedPreferences). You will need to create this file! The contents of this file will be a list of what subdirectories you want the FileProvider to be able to provide. It will look something like:

<?xml version="1.0" encoding="utf-8"?>
<paths xmlns:android="http://schemas.android.com/apk/res/android">
    <!--shared from internal storage -->
    <files-path path="images/" name="myfiles" />

    <!--shared external storage -->
    <external-path path="Documents/" name="documents" />
    <external-path path="Pictures/" name="pictures" />

</paths>

The <files-path> entry refers to a subdirectory for Internal Storage files (the same place that .getFilesDir() points to), and <external-path> to refer to the subdirectory for External Storage files. The path attributes specifies the name of the subdirectory inside that storage location, and name is the name that the FileProvider will specify the path as (think: a public facing path).

Once you have the provider specified, you can use it to get a Uri to the “shared” version of the file using:

Uri fileUri = FileProvider.getUriForFile(context, "edu.uw.myapp.fileprovider", fileToShare);

(note that the second parameter is the “authority” you specified in your <provider> in the Manifest). You can then use this Uri as the MediaStore.EXTRA_OUTPUT extra to tell the Camera where to save the picture it takes!

  • You can also use it as an EXTRA_STREAM extra in the Intent for more general file sharing.

  • Then when we get the picture result back from the Camera (in our onActivityResult callback), we can access that file at the saved Uri and use it to display the image! The ImageView.setImageUri() is a fast way of showing an image file.

    You can also use an Intent to make sure a photo is added to the phone’s Gallery.

Note that when working with images, we can very quickly run out of memory (because images can be huge). So we’ll often want to “scale down” the images as we load them into memory. Additionally, image processing can take a while so we’d like to do it off the main thread (e.g., in an AsyncTask). This can become complicated; the recommended solution is to use a third-party library such as Glide, Picasso, or Fresco.

10.6 Sharing Files

Once we have a file storing the image, we can also share that image with other apps!

As always, in order to interact with other apps, we use an Intent. We can craft an implicit intent for ACTION_SEND, sending a message to any apps that are able to send (share) pictures. We’ll set the data type as image/* to mark this as an image. We will also attach the file as an extra (specifically an EXTRA_STREAM). Again note that we don’t actually put the file in the extra, but rather the Uri for the file—the one provided by the FileProvider!

//java
Intent intent = new Intent(Intent.ACTION_SEND);
intent.setType("image/*");
intent.putExtra(Intent.EXTRA_STREAM, this.pictureFileUri);

if (intent.resolveActivity(getPackageManager()) != null) {
    startActivity(intent);
}

If you’re sharing a file that is normally private (e.g., from Internal Storage or private External Storage), you’ll also need to make sure to that the “target” Activity has permission to read the file. You can grant this by adding a “flag” to the intent:

intent.addFlags(Intent.FLAG_GRANT_READ_URI_PERMISSION);

And with that, you should be able to take a picture, save it publicly (or privately), but still share it with the world!

  1. https://developer.android.com/training/basics/data-storage/files.html#InternalVsExternalStorage↩

  2. https://developer.android.com/reference/java/io/File.html↩

  3. https://developer.android.com/guide/topics/permissions/normal-permissions.html↩