Image processing android app with Java Native Access (JNA)

Repository

https://github.com/java-native-access/jna

Overview

In this tutorial we will create simple image processing app in which user will capture image from camera and then color image will be converted to gray scale image using C++. Image processing in java is expensive task, using C++ we can greatly improve the performance. App will use Native Access(JNA) library to call native code from Java side.
We will use some techniques that we learnt in previous tutorials on Java Native Access(JNA) library. For more information on Java Native Access(JNA) library please read previous tutorials series

• Calling android C/C++ code with Java Native Access (JNA)
• Building android shared library and calling with Java Native Access (JNA)
• Building and using multiple android shared libraries
• Handling C++ callbacks, Logging and exceptions with Java Native Access (JNA)
• Mapping primitive, structure, array, NIO buffer, class and object types with Java Native Access (JNA)

Requirements

• Android Studio 3.0 or higher
• Android NDK

Difficulty

• Intermediate

Algorithm

• Get individual color channels from pixel
• Calculate gray value
• Create gray pixel

Example below uses red pixel

         ALPHA    RED      GREEN    BLUE
PIXEL = 11111111 11111111 00000000 00000000

// Right shift by 24 moves Byte 0(ALPHA) to Byte 4 position
// If MSB (most significant bit) is 1 then 1's will be appended from left side
// if MSB is 0 then 0's will be appended from left side
// & operation with 0xFF makes Byte 0, 1 and 2  = 0 and we get ALPHA value

ALPHA = (PIXEL >> 24) & 0xFF
11111111 11111111 11111111 11111111
00000000 00000000 00000000 11111111 &
--------------------------------------
00000000 00000000 00000000 11111111

// Right shift by 16 moves Byte 1(RED) to Byte 4 position
// If MSB (most significant bit) is 1 then 1's will be appended from left side
// if MSB is 0 then 0's will be appended from left side
// & operation with 0xFF makes Byte 0, 1 and 2  = 0 and we get RED value

RED = (PIXEL >> 16) & 0xFF
11111111 11111111 11111111 11111111
00000000 00000000 00000000 11111111 &
--------------------------------------
00000000 00000000 00000000 11111111

// Right shift by 8 moves Byte 2(GREEN) to Byte 4 position
// If MSB (most significant bit) is 1 then 1's will be appended from left side
// if MSB is 0 then 0's will be appended from left side
// & operation with 0xFF makes Byte 0, 1 and 2  = 0 and we get GREEN value

GREEN = (PIXEL >> 8) & 0xFF
11111111 11111111 11111111 00000000
00000000 00000000 00000000 11111111 &
--------------------------------------
00000000 00000000 00000000 00000000

// BLUE byte is already on Byte 4 position no need to shift
// & operation with 0xFF makes Byte 0, 1 and 2  = 0 and we get BLUE value

BLUE = (PIXEL) & 0xFF
11111111 11111111 00000000 00000000
00000000 00000000 00000000 11111111 &
--------------------------------------
00000000 00000000 00000000 00000000

// Average of RED, GREEN and BLUE is GRAY

GRAY = (RED + GREEN + BLUE) / 3

//We need to assign same GRAY value to RED,GREEN and BLUE
// ALPHA remains same

GRAY_PIXEL =  (ALPHA << 24) | (GRAY << 16) | (GRAY << 8) | GRAY

Tutorial covers

• Creating Android Studio project
• Configuring JNA AAR library
• Creating UI
• Capture image and showing in ImageView
• Convert bitmap image to pixels
• Convert color pixels to gray scale in C++
• Loading and mapping C++ shared library in Java
• Calling native method and showing gray scale image

Guide

1. Creating Android Studio project

Create new android project and change Application name you can also change Company domain and Package name according to requirements, select Include C++ support and click next

Select minimum SDK version and click next

Select Empty Activity and click next

Change Activity Name and Layout Name according to requirements and click Next

Select C++ Standard as Toolchain Default and click Finish to create project

If you get NDK not configured error in Messages window then click on File menu and select Project Structure and set Android NDK location.

2. Configuring JNA AAR library

Download jna.aar and create New Module and select Import JAR/AAR Package and click Next

Select jna.aar file from file system and click Finish

jna module added to project, open build.gradle file and add jna module under dependencies

dependencies {
    implementation project(':jna')
    ....
    ....
}

3. Creating UI

Our app will use camera so lets add camera permission in AndroidManifest.xml file

<uses-feature android:name="android.hardware.camera" android:required="true" />

Main layout contains only Button and ImageView user will click on Button to capture image from camera and resulting gray scale image will be shown in ImageView. Here is layout xml

<?xml version="1.0" encoding="utf-8"?>
<android.support.constraint.ConstraintLayout 
    xmlns:android="http://schemas.android.com/apk/res/android"
    xmlns:app="http://schemas.android.com/apk/res-auto"
    xmlns:tools="http://schemas.android.com/tools"
    android:layout_width="match_parent"
    android:layout_height="match_parent"
    tools:context=".MainActivity">

    <Button
        android:id="@+id/button"
        android:layout_width="wrap_content"
        android:layout_height="wrap_content"
        android:layout_marginEnd="8dp"
        android:layout_marginLeft="8dp"
        android:layout_marginRight="8dp"
        android:layout_marginStart="8dp"
        android:layout_marginTop="8dp"
        android:text="Capture Image"
        app:layout_constraintEnd_toEndOf="parent"
        app:layout_constraintStart_toStartOf="parent"
        app:layout_constraintTop_toTopOf="parent" />

    <ImageView
        android:id="@+id/imageView"
        android:layout_width="0dp"
        android:layout_height="0dp"
        android:layout_marginBottom="8dp"
        android:layout_marginEnd="8dp"
        android:layout_marginLeft="8dp"
        android:layout_marginRight="8dp"
        android:layout_marginStart="8dp"
        android:layout_marginTop="8dp"
        android:background="#e5e5e5"
        app:layout_constraintBottom_toBottomOf="parent"
        app:layout_constraintEnd_toEndOf="parent"
        app:layout_constraintStart_toStartOf="parent"
        app:layout_constraintTop_toBottomOf="@+id/button" />
</android.support.constraint.ConstraintLayout>

Main screen should look like this

To access Button and ImageView in Java

public class MainActivity extends AppCompatActivity {
    private Button button;
    private ImageView imageView;

    @Override
    protected void onCreate(Bundle savedInstanceState) {
        super.onCreate(savedInstanceState);
        setContentView(R.layout.activity_main);

        button = findViewById(R.id.button);
        imageView = findViewById(R.id.imageView);
    }
}

4. Capture image and showing in ImageView

In this section first we will add click listener on button in onCreate() method, click to this button will call captureImage() method

button.setOnClickListener(new View.OnClickListener() {
    @Override
    public void onClick(View v) {
        captureImage();
    }
});

Call to captureImage() method will starts new camera activity with image capture intent

//will open camera activity
private void captureImage() {
    Intent intent = new Intent(MediaStore.ACTION_IMAGE_CAPTURE);
    if (intent.resolveActivity(getPackageManager()) != null) {
        startActivityForResult(intent, REQUEST_IMAGE_CAPTURE);
    }
}

Result of camera activity will be handled in onActivityResult() method. We need to override this method, in this method we will get image result and assign it to ImageView

//result of camera activity handled here
@Override
protected void onActivityResult(int requestCode, int resultCode, Intent data) {
    if (requestCode == REQUEST_IMAGE_CAPTURE && resultCode == RESULT_OK) {
        //getting mutable bitmap from intent and set it to ImageView
        Bundle extras = data.getExtras();
        Bitmap bitmap = ((Bitmap) extras.get("data"))
                .copy(Bitmap.Config.ARGB_8888, true);
        imageView.setImageBitmap(bitmap);
    }
}

Here is complete code for image capture and showing captured image in ImageView

public class MainActivity extends AppCompatActivity {
    private Button button;
    private ImageView imageView;
    private static final int REQUEST_IMAGE_CAPTURE = 1;

    @Override
    protected void onCreate(Bundle savedInstanceState) {
        super.onCreate(savedInstanceState);
        setContentView(R.layout.activity_main);

        button = findViewById(R.id.button);
        imageView = findViewById(R.id.imageView);

        button.setOnClickListener(new View.OnClickListener() {
            @Override
            public void onClick(View v) {
                captureImage();
            }
        });
    }

    //will open camera activity
    private void captureImage() {
        Intent intent = new Intent(MediaStore.ACTION_IMAGE_CAPTURE);
        if (intent.resolveActivity(getPackageManager()) != null) {
            startActivityForResult(intent, REQUEST_IMAGE_CAPTURE);
        }
    }

    //result of camera activity handled here
    @Override
    protected void onActivityResult(int requestCode, int resultCode, Intent data) {
        if (requestCode == REQUEST_IMAGE_CAPTURE && resultCode == RESULT_OK) {
            //getting mutable bitmap from intent and set it to ImageView
            Bundle extras = data.getExtras();
            Bitmap bitmap = ((Bitmap) extras.get("data"))
                    .copy(Bitmap.Config.ARGB_8888, true);
            imageView.setImageBitmap(bitmap);
        }
    }
}

5. Convert bitmap image to pixels

We now have bitmap image, to convert bitmap image to pixels

int width = bitmap.getWidth();
int height = bitmap.getHeight();

int[] pixels = new int[width * height];
bitmap.getPixels(pixels, 0, width, 0, 0, width, height);

6. Convert color pixels to gray scale in C++

Open native-lib.cpp file in cpp folder and remove everything. Method we want to export is marked with extern "C" to avoid name mangling. It accepts two arguments one is pixels array and other one is length of pixels array.
For more information of passing Java arrays to C++ please read tutorial T5.

extern "C"
void toGrayScale(int *pixels, int len) {
    for (int i = 0; i < len; i++) {

        //getting individual color values from each pixel
        int A = (pixels[i] >> 24) & 0xFF;
        int R = (pixels[i] >> 16) & 0xFF;
        int G = (pixels[i] >> 8) & 0xFF;
        int B = pixels[i] & 0xFF;

        //averaging Red, Green and Blue value to get gray scale value
        int gray = (R + G + B) / 3;

        //assign same gray value to Red, Green and Blue.
        //alpha value is unchanged
        pixels[i] = (A << 24) | (gray << 16) | (gray << 8) | gray;
    }
}

7. Loading and mapping C++ shared library in Java

static {
    Native.register(MainActivity.class, "native-lib");
}

In static block of a class we load our shared library, first argument to Native.register() method is the class in which we defined our native methods and 2nd argument is the name of native shared library. Name of shared library is native-lib we can change this default name in CMakeLists.txt file which is available in app folder. Loading should be done in static block which will load shared library during class loading time.
To map native method void toGrayScale(int *pixels, int len) in Java we need to add native keyword which tells compiler that method is implemented in native code.

public native void toGrayScale(int pixels[], int len);

To call native method in Java is same as calling other Java methods

toGrayScale(pixels, pixels.length);

8. Calling native method and showing gray scale image

//calling native method
toGrayScale(pixels, pixels.length);

//updating grayscale pixels of bitmap and showing in ImageView
bitmap.setPixels(pixels, 0, width, 0, 0, width, height);
imageView.setImageBitmap(bitmap);

Complete code

Layout xml file

<?xml version="1.0" encoding="utf-8"?>
<android.support.constraint.ConstraintLayout
    xmlns:android="http://schemas.android.com/apk/res/android"
    xmlns:app="http://schemas.android.com/apk/res-auto"
    xmlns:tools="http://schemas.android.com/tools"
    android:layout_width="match_parent"
    android:layout_height="match_parent"
    tools:context=".MainActivity">

    <Button
        android:id="@+id/button"
        android:layout_width="wrap_content"
        android:layout_height="wrap_content"
        android:layout_marginEnd="8dp"
        android:layout_marginLeft="8dp"
        android:layout_marginRight="8dp"
        android:layout_marginStart="8dp"
        android:layout_marginTop="8dp"
        android:text="Capture Image"
        app:layout_constraintEnd_toEndOf="parent"
        app:layout_constraintStart_toStartOf="parent"
        app:layout_constraintTop_toTopOf="parent" />

    <ImageView
        android:id="@+id/imageView"
        android:layout_width="0dp"
        android:layout_height="0dp"
        android:layout_marginBottom="8dp"
        android:layout_marginEnd="8dp"
        android:layout_marginLeft="8dp"
        android:layout_marginRight="8dp"
        android:layout_marginStart="8dp"
        android:layout_marginTop="8dp"
        android:background="#e5e5e5"
        app:layout_constraintBottom_toBottomOf="parent"
        app:layout_constraintEnd_toEndOf="parent"
        app:layout_constraintStart_toStartOf="parent"
        app:layout_constraintTop_toBottomOf="@+id/button" />
</android.support.constraint.ConstraintLayout>

C++ code

extern "C"
void toGrayScale(int *pixels, int len) {
    for (int i = 0; i < len; i++) {
        //getting individual color values from each pixel
        int A = (pixels[i] >> 24) & 0xFF;
        int R = (pixels[i] >> 16) & 0xFF;
        int G = (pixels[i] >> 8) & 0xFF;
        int B = pixels[i] & 0xFF;

        //averaging Red, Green and Blue value to get gray scale value
        int gray = (R + G + B) / 3;

        //assign same gray value to Red, Green and Blue.
        //alpha value is unchanged
        pixels[i] = (A << 24) | (gray << 16) | (gray << 8) | gray;
    }
}

Java code

public class MainActivity extends AppCompatActivity {
    static {
        Native.register(MainActivity.class, "native-lib");
    }

    private Button button;
    private ImageView imageView;
    private static final int REQUEST_IMAGE_CAPTURE = 1;

    public native void toGrayScale(int pixels[], int len);

    @Override
    protected void onCreate(Bundle savedInstanceState) {
        super.onCreate(savedInstanceState);
        setContentView(R.layout.activity_main);

        button = findViewById(R.id.button);
        imageView = findViewById(R.id.imageView);

        button.setOnClickListener(new View.OnClickListener() {
            @Override
            public void onClick(View v) {
                captureImage();
            }
        });
    }

    //will open camera activity
    private void captureImage() {
        Intent intent = new Intent(MediaStore.ACTION_IMAGE_CAPTURE);
        if (intent.resolveActivity(getPackageManager()) != null) {
            startActivityForResult(intent, REQUEST_IMAGE_CAPTURE);
        }
    }

    //result of camera activity handled here
    @Override
    protected void onActivityResult(int requestCode, int resultCode, Intent data) {
        if (requestCode == REQUEST_IMAGE_CAPTURE && resultCode == RESULT_OK) {
            //getting mutable bitmap from intent and set it to ImageView
            Bundle extras = data.getExtras();
            Bitmap bitmap = ((Bitmap) extras.get("data"))
                    .copy(Bitmap.Config.ARGB_8888, true);

            //getting pixels from bitmap
            int width = bitmap.getWidth();
            int height = bitmap.getHeight();

            int[] pixels = new int[width * height];
            bitmap.getPixels(pixels, 0, width, 0, 0, width, height);

            //calling native method
            toGrayScale(pixels, pixels.length);

            //updating grayscale pixels of bitmap and showing in ImageView
            bitmap.setPixels(pixels, 0, width, 0, 0, width, height);
            imageView.setImageBitmap(bitmap);
        }
    }
}

Output

Github

Complete project available on github. Clone repo and open project name T6.