Integrate Amazon S3 to Android Tutorial

Udemy Generic 728x90

Objective

In this tutorial we will learn how to integrate Amazon S3 with an Android app and Upload/Download files to Amazon S3 Bucket through Android Application using a simple example.

 

 

Amazon S3, Amazon Simple Storage Service is a secure and highly-scalable cloud storage offered by Amazon Web Services. It is an easy to use service with a simple interface to store and retrieve data anywhere from web. Amazon provides AWS Mobile SDK which provides JAVA APIs for various Amazon Services. Developers can use the SDK to integrate Amazon S3 in their mobile app and build interface to directly upload or download data through their application.

In this tutorial, we will integrate Amazon S3 in a simple Android Application. Also, we will learn how to upload and download file from Amazon S3 by using AWS Mobile SDK. SDK has a utility class, named S3 Transfer Utility with methods to transfer data in background (asynchronous process). To setup S3 Transfer Utility in your app, you need to create an account on the Amazon S3 and setup Amazon Cognito Identity to authenticate your app with AWS. We will explain how to set up Amazon Cognito Identity in the steps below.

Step 2

Create a New Project

Create a new Project in Android Studio, goto File ⇒ New ⇒ New Projects.

 

Step 3

Add AWS Mobile SDK as Dependencies

To access Amazon S3 services, we need to add few modules of AWS Mobile SDK as dependencies to app/build.gradle file.

   ....
    compile 'com.amazonaws:aws-android-sdk-core:2.2.13'
    compile 'com.amazonaws:aws-android-sdk-cognito:2.2.13'
    compile 'com.amazonaws:aws-android-sdk-s3:2.2.13'
    compile 'com.amazonaws:aws-android-sdk-ddb:2.2.13'
   ....

 

Step 4

Set Permissions in Your Manifest

We will upload and download file to Android Device via Internet. So, we have to request Internet, Read and Write permissions from the users. Use the following code to set up permissions in AndroidManifest.xml file:

<uses-permission android:name="android.permission.INTERNET"/>
<uses-permission android:name="android.permission.WRITE_EXTERNAL_STORAGE" />
<uses-permission android:name="android.permission.READ_EXTERNAL_STORAGE" />

 

Step 5

Configure AWS Credentials

We need to get AWS Credentials using Amazon Cognito Identity as credential provider. Use of credential provider allows your app to access Amazon services without having to provide your private credential in app.It also allows you to set the permissions for the Amazon services which can be accessed by users.

To set up Amazon Cognito, we must create an identity pool. An identity pool is used to store user identity data to your account. A developer will use one identity pool per application. Every identity pool is a configurable IAM role that monitors Amazon Mobile Analytics and allows developer to access AWS services.

 

Step 5.1

Create an Identity Pool

To create an identity pool:

    • Log in to the Amazon Cognito Console and click on Create new identity pool button.
      create identity pool
    • Enter name of identity pool in the Identity pool name field.
    • Select the check box to enable access to unauthenticated identities.
    • Click on the Create Pool button to create identity pool.

      identity pool
    • Click on the Allow button to create two default roles associated with your identity pool: one for unauthenticated users and one for authenticated users.

 

IAM role
 

Step 5.2

Configure IAM role

Also, we need to specify the bucket to store the data and provide access permission of that bucket to developer by configuring its IAM role.

  • Create a new bucket. Each bucket must have a globally unique name.
  • Go to Identity and Access Management Console and click on Roles in the left-hand pane.

    welcome identity access management
  • Click your Identity pool name for unauthenticated users (it will have “unauth” appended to your Identity Pool name).
    unautherized user
  • Create Inline Policy, click on the click here link in the Inline Policies panel.

    permission policy

  • To set the permission, click on Select button.

    set identity permissions

  • On the Edit Permissions page, select Amazon S3 as AWS Service, select All Actions(*) for Actions and enter the arn:aws:s3:::your_bucket_name/* as Amazon Resource Name (ARN).
    set permission of IAM role
  • Click on the Add Statement button to add the permission then click on Next Step button.

    amzonS3
  • Click on Apply Policy button to apply policy that generated by you.

 

apply policy

Step 6

Declare Service in Manifest file

We will transfer files to Amazon S3 by using transfer service of SDK. So, we need to add the TransferService in the AndroidManifest.xml in the application.

<service
    android:name="com.amazonaws.mobileconnectors.s3.transferutility.TransferService"
    android:enabled="true" />

 

Step 7

Create Layout

Next, we need to implement a process to upload and download on click of button. So, add Button widgets to trigger upload and download functionalities in the acitvity_main.xml file.

<Button
    android:id="@+id/upload"
    android:layout_width="match_parent"
    android:layout_height="wrap_content"
    android:text="Upload File to S3"
    android:onClick="setFileToUpload"/>

<Button
    android:id="@+id/download"
    android:layout_width="match_parent"
    android:layout_height="wrap_content"
    android:text="Download File from S3"
    android:layout_below="@+id/upload"
    android:layout_alignParentRight="true"
    android:layout_alignParentEnd="true"
    android:onClick="setFileToDownload"/>

 

s3 layout

Step 8

Initialize Credential Provider

To use Amazon S3 service, we need to create a credentials provider to integrate Cognito Identity with Android application. Pass the COGNITO_IDENTITY_POOL Id to the Credentials provider object. Credentials provider object should look like this:

public void credentialsProvider(){

    // Initialize the Amazon Cognito credentials provider
    CognitoCachingCredentialsProvider credentialsProvider = new CognitoCachingCredentialsProvider(
            getApplicationContext(),
            "us-east-1:dbacd6aa-9393-475e-b687-xxxxxxxxx", // Identity Pool ID
            Regions.US_EAST_1 // Region
    );

    setAmazonS3Client(credentialsProvider);
}

 

Step 9

Set an Amazon S3 Client

To start Amazon S3 service, pass credentials provider to the S3 client constructor. Then, set the region of the client. So, create a setAmazonS3Client() method and within this method, create a AmazonS3Client constructor and pass the credentials provider to constructor.

/**
 *  Create a AmazonS3Client constructor and pass the credentialsProvider.
 * @param credentialsProvider
 */
 public void setAmazonS3Client(CognitoCachingCredentialsProvider credentialsProvider){

    // Create an S3 client
    s3 = new AmazonS3Client(credentialsProvider);

    // Set the region of your S3 bucket
    s3.setRegion(Region.getRegion(Regions.US_EAST_1));

}

 

Step 10

Call TransferUtility Class

We will use the TransferUtility class to upload and download files from Amazon S3. Pass the Amazon S3 client and the application context to the Transfer Utility.

public void setTransferUtility(){

   transferUtility = new TransferUtility(s3, getApplicationContext());
}

 

Step 11

Upload file to S3

To upload file to S3, we will pass the MY_BUCKET, OBJECT_KEY and MY_FILE to upload() method of TransferUtility class and return it to the TransferObserver object.

  • MY_BUCKET: MY_BUCKET is the name of the S3 bucket where the file is stored
  • OBJECT_KEY: OBJECT_KEY is the name of the S3 object (a file in this case) to download
  • MY_FILE – The java.io.File object where the downloaded file will be written.
public class MainActivity extends AppCompatActivity {

    File fileToUpload = new File("/storage/sdcard0/Pictures/Screenshots/photos.png");
    AmazonS3 s3;
    TransferUtility transferUtility;

    @Override
    protected void onCreate(Bundle savedInstanceState) {
        super.onCreate(savedInstanceState);
        setContentView(R.layout.activity_main);

       // callback method to call credentialsProvider method.
        credentialsProvider();

        // callback method to call the setTransferUtility method
        setTransferUtility();
    }

    public void credentialsProvider(){

        // Initialize the Amazon Cognito credentials provider
        CognitoCachingCredentialsProvider credentialsProvider = new CognitoCachingCredentialsProvider(
                getApplicationContext(),
                "us-east-1:dbacd6aa-9393-475e-b687-xxxxxxxx", // Identity Pool ID
                Regions.US_EAST_1 // Region
        );

        setAmazonS3Client(credentialsProvider);
    }

    /**
     *  Create a AmazonS3Client constructor and pass the credentialsProvider.
     * @param credentialsProvider
     */
    public void setAmazonS3Client(CognitoCachingCredentialsProvider credentialsProvider){

        // Create an S3 client
        s3 = new AmazonS3Client(credentialsProvider);

        // Set the region of your S3 bucket
        s3.setRegion(Region.getRegion(Regions.US_EAST_1));

    }

    public void setTransferUtility(){

        transferUtility = new TransferUtility(s3, getApplicationContext());
    }

    /**
     * This method is used to upload the file to S3 by using TransferUtility class
     * @param view
     */
    public void setFileToUpload(View view){

                TransferObserver transferObserver = transferUtility.upload(
                "test.numetric",     /* The bucket to upload to */
                "photos.png",       /* The key for the uploaded object */
                fileToUpload       /* The file where the data to upload exists */
        );
    }
}

 

Step 12

Call the TransferObserver Listener

To know the status of uploading and downloading the file, we need to set transfer listeners. Amazon SDK provides TransferObserver class which have various listener methods to track the status of the upload or download. So, implement and override onStateChanged(), onProgressChanged(), onError() methods of TransferObserver class.

/**
 * This is listener method of the TransferObserver
 * Within this listener method, we get the status of uploading and downloading the file,
 * it displays percentage part of the  file to be uploaded or downloaded to S3
 * It also displays an error, when there is problem to upload and download file to S3.
 * @param transferObserver
 */

 public void transferObserverListener(TransferObserver transferObserver){

    transferObserver.setTransferListener(new TransferListener(){

        @Override
        public void onStateChanged(int id, TransferState state) {
                Log.e("statechange", state+"");
        }

        @Override
        public void onProgressChanged(int id, long bytesCurrent, long bytesTotal) {
            int percentage = (int) (bytesCurrent/bytesTotal * 100);
            Log.e("percentage",percentage +"");
        }

        @Override
        public void onError(int id, Exception ex) {
             Log.e("error","error");
        }

    });
}

 

upload file to S3

Step 13

Download File from S3

To download the file from S3, we need to call the download method of the TransferUtility class and return TransferObserver object which will track the status of the download. Pass the MY_BUCKET, OBJECT_KEY and MY_FILE to the download() method.

/**
 *  This method is used to Download the file to S3 by using transferUtility class
 * @param view
 **/
 public void setFileToDownload(View view){

     TransferObserver transferObserver = transferUtility.download(
             "test.numetric",     /* The bucket to download from */
             "MY",    /* The key for the object to download */
             fileToDownload        /* The file to download the object to */
     );

     transferObserverListener(transferObserver);

}

 

Step 14

Final Code

Here is final code of the MainActivity.java class:

public class MainActivity extends AppCompatActivity {

    File fileToUpload = new File("/storage/sdcard0/Pictures/Screenshots/photos.png");
    File fileToDownload = new File("/storage/sdcard0/Pictures/MY");
    AmazonS3 s3;
    TransferUtility transferUtility;

    @Override
    protected void onCreate(Bundle savedInstanceState) {
        super.onCreate(savedInstanceState);
        setContentView(R.layout.activity_main);

        // callback method to call credentialsProvider method.
        credentialsProvider();

        // callback method to call the setTransferUtility method
        setTransferUtility();
    }

    public void credentialsProvider(){

        // Initialize the Amazon Cognito credentials provider
        CognitoCachingCredentialsProvider credentialsProvider = new CognitoCachingCredentialsProvider(
                getApplicationContext(),
                "us-east-1:dbacd6aa-9393-475e-b687-059de565eab2", // Identity Pool ID
                Regions.US_EAST_1 // Region
        );

        setAmazonS3Client(credentialsProvider);
    }

    /**
     *  Create a AmazonS3Client constructor and pass the credentialsProvider.
     * @param credentialsProvider
     */
    public void setAmazonS3Client(CognitoCachingCredentialsProvider credentialsProvider){

        // Create an S3 client
        s3 = new AmazonS3Client(credentialsProvider);

        // Set the region of your S3 bucket
        s3.setRegion(Region.getRegion(Regions.US_EAST_1));

    }

    public void setTransferUtility(){

        transferUtility = new TransferUtility(s3, getApplicationContext());
    }

    /**
     * This method is used to upload the file to S3 by using TransferUtility class
     * @param view
     */
    public void setFileToUpload(View view){

                TransferObserver transferObserver = transferUtility.upload(
                "test.numetric",     /* The bucket to upload to */
                "photos.png",    /* The key for the uploaded object */
                fileToUpload       /* The file where the data to upload exists */
        );

        transferObserverListener(transferObserver);
    }

    /**
     *  This method is used to Download the file to S3 by using transferUtility class
     * @param view
     **/
    public void setFileToDownload(View view){

        TransferObserver transferObserver = transferUtility.download(
                "test.numetric",     /* The bucket to download from */
                "MY",    /* The key for the object to download */
                fileToDownload        /* The file to download the object to */
        );

        transferObserverListener(transferObserver);

    }

    /**
     * This is listener method of the TransferObserver
     * Within this listener method, we get status of uploading and downloading file,
     * to display percentage of the part of file to be uploaded or downloaded to S3
     * It displays an error, when there is a problem in  uploading or downloading file to or from S3.
     * @param transferObserver
     */

    public void transferObserverListener(TransferObserver transferObserver){

        transferObserver.setTransferListener(new TransferListener(){

            @Override
            public void onStateChanged(int id, TransferState state) {
                Log.e("statechange", state+"");
            }

            @Override
            public void onProgressChanged(int id, long bytesCurrent, long bytesTotal) {
                int percentage = (int) (bytesCurrent/bytesTotal * 100);
                Log.e("percentage",percentage +"");
            }

            @Override
            public void onError(int id, Exception ex) {
                Log.e("error","error");
            }

        });
    }
}

Udemy Generic 728x90

Spread the word. Share this post!

  • Lucas Ferraz

    Thank you, it helped me =)

  • Sardor Islomov

    Thank you it is great !!!!

  • Buvaneswari

    Thank you,

  • Rohan

    How to see the uploaded files in AWS bucket. please explain like above example

  • Arun Rajagopalan

    God bless you guys!

  • Krishna Ch

    app.s3amazon D/CognitoCachingCredentialsProvider: No valid credentials found in SharedPreferences im having problm with my cognito upload process

    • Snehal Tembare

      Use CognitoCredentialsProvider instead of CognitoCachingCredentialsProvider

  • Suparat Yongjoh

    Thank you so much for this tutorial <3

  • Common Sense

    This is not secure, why would you use unauthorized users? It means anyone with the Cognito endpoint can put viruses on your bucket. BAD IDEA!!

  • http://dajver.blogspot.com/ Gleb Kravchenko

    Very very good article, thank you very much! But I have some issue – “com.amazonaws.AmazonClientException: Unable to execute HTTP request: Write error: ssl=0xb8e450e8: I/O error during system call, Connection reset by peer” and I don’t understand what to do :(

    • Snehal Tembare

      Check Regions for the Bucket you are created and Identity pool

  • Hardik

    Hey! Nice article. How to set public ACL in sample code?

  • God

    I would like to ask how you would know if the uploaded file is finish? If we look on onProgressChanged percentage is equivalent to 100 then i have found some bugs that it is triggered twice. Is there someone who is experienceing the same?