Pixel Playground using Renderscript Compute API


Today I’ll show how you can take advantage of the renderscript compute API to manipulate pixels in an image. Previously, I’m doing all my pixel manipulations using OpenCV ports for android (here and the official one), NDK and JNI since there are no direct API that you can use to call OpenCV functions from the Java layer. It is a painful process because you need to write in C for all the pixel manipulations and then call the C functions in Java via JNI.

The following post will show you how easy it is to perform some simple image processing without writing complex C codes and JNI bindings.

We will use the existing HelloCompute example as our base code and we will just modify it a little bit to give us more control on the pixels of the image.

In our main activity, HelloCompute.java:



protected void onCreate(Bundle savedInstanceState) {
    super.onCreate(savedInstanceState);
    setContentView(R.layout.main);

    mBitmapIn = loadBitmap(R.drawable.face2);
    mBitmapOut = Bitmap.createBitmap(mBitmapIn.getWidth(), mBitmapIn.getHeight(), mBitmapIn.getConfig());

    ImageView in = (ImageView) findViewById(R.id.displayin);
    in.setImageBitmap(mBitmapIn);

    out = (ImageView) findViewById(R.id.displayout);
    out.setImageBitmap(mBitmapOut);

    showOriginal = (CheckBox) findViewById(R.id.original);
    showOriginal.setOnCheckedChangeListener(new OnCheckedChangeListener() {

        @Override
        public void onCheckedChanged(CompoundButton buttonView, boolean isChecked) {
        if (isChecked) {
            mScript.set_original(true);
        } else {
            mScript.set_original(false);
        }

        mScript.invoke_filter();
        mOutAllocation.copyTo(mBitmapOut);
        out.invalidate();
        }
    });

    redChannel = (SeekBar) findViewById(R.id.red);
    greenChannel = (SeekBar) findViewById(R.id.green);
    blueChannel = (SeekBar) findViewById(R.id.blue);

    redChannel.setOnSeekBarChangeListener(this);
    greenChannel.setOnSeekBarChangeListener(this);
    blueChannel.setOnSeekBarChangeListener(this);

    createScript();
    }
 

Here, after loading the input bitmap and the output bitmap, we then retrieve handles to our controls. Basically we will have a checkbox to reset our image back to its original state and 3 sliders to control the RGB values of the image. The OnCheckedChangedListener on the checkbox simply sets the flag on the renderscript code to render the original image, then update the output bitmap and invalidate our imageview to refresh the display.

Damn it I accidentally deleted the images!

Now, the juiciest part is the createScript method.



private void createScript() {
    mRS = RenderScript.create(this);

    mInAllocation = Allocation.createFromBitmap(mRS, mBitmapIn, Allocation.MipmapControl.MIPMAP_NONE, Allocation.USAGE_SCRIPT);
    mOutAllocation = Allocation.createTyped(mRS, mInAllocation.getType());

    mScript = new ScriptC_mono(mRS, getResources(), R.raw.mono);

    mScript.set_width(mBitmapIn.getWidth());
    mScript.set_height(mBitmapIn.getHeight());
    mScript.set_gIn(mInAllocation);
    mScript.set_gOut(mOutAllocation);
    mScript.set_gScript(mScript);
    mScript.invoke_filter();
    mOutAllocation.copyTo(mBitmapOut);
    }

First, we need to allocate our input bitmap and the output bitmap which will give us easy access to the bitmaps from our renderscript code. Then we need to load our renderscript code stored in mono.rs. The class ScriptC_mono is actually an auto-generated class created by the LLVM front-end compiler. Renderscript API uses two compilers, one in the front-end that generates the support classes and the runtime compiler that is actually being shipped inside the Android devices (Honeycomb 3.0+). The methods set_width, set_height, invoke_filter, etc. are also generated based on the variables and methods declared inside mono.rs. Now let’s take a look at mono.rs


#pragma version(1)
#pragma rs java_package_name(com.example.android.rs.hellocompute)

rs_allocation gIn;
rs_allocation gOut;
int width;
int height;
float red;
float blue;
float green;
bool original = true;
rs_script gScript;

const static float3 gMonoMult = {0.299f, 0.587f, 0.114f};

void root(const uchar4 *v_in, uchar4 *v_out, const void *usrData, uint32_t x, uint32_t y) {
    const uchar4 *element = rsGetElementAt(gIn, x > width/2 ? (width-x):x, y);
    float4 test = rsUnpackColor8888(*element);

    float r = test.r;
    float g = test.g;
    float b = test.b;   

    if(!original){
        r = red > -1  ? red : r;
        g = green > -1  ? green : g;
        b = blue > -1  ? blue : b;
    }

    float3 mono = {r, g, b};
    *v_out = rsPackColorTo8888(mono);
}

void filter() {
    rsForEach(gScript, gIn, gOut, 0);
}

First line declares the Renderscript version that we want to use. Use 1 for now. Then line 2 declares the packagename where this script is going to be used. Then we declare all our variables. The generated code will have accessors and mutators for the variables and then invoke_methodName for methods. Next, the root method is like the main method in C or in Java. In this example, the root function will be called for every pixel of the bitmap. This is because we used the renderscript function rsForEach which simply loops through the allocated bitmap’s pixels and call the root function for each one of it. One thing to take note is that, according to Romain Guy and Chet Haase, this function (rsForEach) can take advantage of multiple cores which means that it can process the bitmap’s pixels in parallel and this obviously means a significant boost in terms of speed.

Although there’s a steep learning curve, the possibility of writing your own photo filters (your app can be the next Instagram :D) and the performance boost that you can gain from using Renderscript in your app are just worth it.

comments powered by Disqus