SDL2: Converting an Image to Grayscale

Hello folks! :)

[Update 2015-11-14: This article is out of date. Check out the latest version at Gigi Labs.]

In the previous article, "SDL2: Pixel Drawing", we saw how to draw pixels onto a blank texture that we created in code. Today, on the other hand, we'll see how we can manipulate pixels on an existing image, such as a photo we loaded from disk. We'll also learn how to manipulate individual bits in an integer using what are called bitwise operators, and ultimately we'll convert an image to grayscale.

We first need to set up an SDL2 project. After following the steps in "SDL2: Setting up SDL2 in Visual Studio (2013 or any other)", you will also need to add SDL2_image.lib to your Linker Input (so that it reads "SDL2.lib; SDL2main.lib; SDL2_image.lib", and place the Visual C++ development libraries obtained from the SDL_image 2.0 homepage into the appropriate folders of your SDL2 directory. After you build your project the first time, make sure you also place all the necessary DLLs (including SDL2.dll, SDL2_image.dll and all the rest) into the same folder as your executable - see "SDL2: Loading Images with SDL_image" in case you need a refresher.

And in fact we're going to start with the code from "SDL2: Loading Images with SDL_image", which is the following (adapted a little bit):

#include <SDL.h>
#include <SDL_image.h>

int main(int argc, char ** argv)
{
    bool quit = false;
    SDL_Event event;

    SDL_Init(SDL_INIT_VIDEO);
    IMG_Init(IMG_INIT_JPG);

    SDL_Window * window = SDL_CreateWindow("SDL2 Grayscale",
        SDL_WINDOWPOS_UNDEFINED, SDL_WINDOWPOS_UNDEFINED, 640, 480, 0);
    SDL_Renderer * renderer = SDL_CreateRenderer(window, -1, 0);
    SDL_Surface * image = IMG_Load("PICT3159.JPG");
    SDL_Texture * texture = SDL_CreateTextureFromSurface(renderer,
        image);

    while (!quit)
    {
        SDL_WaitEvent(&event);

        switch (event.type)
        {
        case SDL_QUIT:
            quit = true;
            break;
        }

        SDL_RenderCopy(renderer, texture, NULL, NULL);
        SDL_RenderPresent(renderer);
    }

    SDL_DestroyTexture(texture);
    SDL_FreeSurface(image);
    SDL_DestroyRenderer(renderer);
    SDL_DestroyWindow(window);
    IMG_Quit();
    SDL_Quit();

    return 0;
}

We can also use the same sample image as in that project, so make sure it's in the appropriate folders where Visual Studio can find it.

Now, let's get down to business.

You see, the problem here is that we can't quite touch the texture pixels directly. So instead, we need to work a little bit similarly to "SDL2: Pixel Drawing": we create our own texture, and then copy the surface pixels over to it. So we throw out the line calling SDL_CreateTextureFromSurface(), and replace it with the following:

    SDL_Texture * texture = SDL_CreateTexture(renderer,
        SDL_PIXELFORMAT_ARGB8888, SDL_TEXTUREACCESS_STATIC,
        image->w, image->h);

Then, at the beginning of the while loop, add this:

        SDL_UpdateTexture(texture, NULL, image->pixels,
            image->w * sizeof(Uint32));

If you try and run program now, it will pretty much explode. That's because our code is assuming that our image uses 4 bytes per pixel (ARGB - see "SDL2: Pixel Drawing"). That's something that depends on the image, and this particular JPG image is most likely 3 bytes per pixel. I don't know much about the JPG format, but I'm certain that it doesn't support transparency, so the alpha channel is out.

The good news is that it's possible to convert the surface into one that has a familiar pixel format. To do this, we use SDL_ConvertSurfaceFormat(). Add the following before the while loop:

    image = SDL_ConvertSurfaceFormat(image, SDL_PIXELFORMAT_ARGB8888, 0);

What this does is take our surface (in this case the one that image points to) and return an equivalent surface with the pixel format we specify. Now that the new image has the familiar ARGB format, we can easily access and manipulate the pixels. Add the following after the line you just added (before the while loop) to typecast the surface pixels from void * to Uint32 * which we can work with:

    Uint32 * pixels = (Uint32 *)image->pixels;

So far so good:

Now, let's add some code do our grayscale conversion. We're going to convert the image to grayscale when the user presses the 'G' key, so let us first add some code within the switch statement to handle that:

        case SDL_KEYDOWN:
            switch (event.key.keysym.sym)
            {
            case SDLK_g:
                for (int y = 0; y < image->h; y++)
                {
                    for (int x = 0; x < image->w; x++)
                    {
                        Uint32 pixel = pixels[y * image->w + x];
                        // TODO convert pixel to grayscale here
                    }
                }
                break;
            }
            break;

This is where bit manipulation comes in. You see, each pixel is a 32-bit integer which in concept looks something like this (actual values are invented, just for illustration):

Alpha	Red	Green	Blue
11111111	10110101	10101000	01101111

So let's say we want to extract the Red component. Its value is 10110101 in binary, or 181 in decimal. But since it's in the third byte from right, its value is much greater than that. So we first shift the bits to the right by 16 spaces to move it to the first byte from right:

		Alpha	Red
00000000	00000000	11111111	10110101

...but we still can't interpret the integer as just red, since the alpha value is still there. We want to extract just that last byte. To do that, we perform a bitwise AND operation:

Pixel	11111111 10110101
Mask	00000000 11111111
Red AND Mask	00000000 10110101

We do an AND operation between our pixel value and a value where only the last byte worth of bits are set to 1. That allows us to extract our red value.

In code, this is how it works:

                        Uint8 r = pixel >> 16 & 0xFF;
                        Uint8 g = pixel >> 8 & 0xFF;
                        Uint8 b = pixel & 0xFF;

The >> operator shifts bits to the right, and the & is a bitwise AND operator. Each colour byte is shifted to the last byte and then ANDed with the value 0xFF, which is hexadecimal notation for what would be 255 in decimal, or 11111111 in binary. That way, we can extract all three colours individually.

We can finally perform the actual grayscaling operation. A simple way to do this might be to average the three colours and set each component to that average:

                        Uint8 v = (r + g + b) / 3;

Then, we pack the individual colour bytes back into a 32-bit integer. We follow the opposite method that we used to extract them in the first place: they are each already at the last byte, so all we need to do is left-shift them into position. Once that is done, we replace the actual pixel in the surface with the grayscaled one:

                        pixel = (0xFF << 24) | (v << 16) | (v << 8) | v;
                        pixels[y * image->w + x] = pixel;

If we now run the program and press the 'G' key, this is what we get:

It looks right, doesn't it? Well, it's not. There's an actual formula for calculating the correct grayscale value (v in our code), which according to Real-Time Rendering is:

Y = 0.212671R + 0.715160G + 0.072169B

The origin of this formula is beyond the scope of this article, but it's due to the fact that humans are sensitive to different colours in different ways - in fact there is a particular affinity to green, hence why it is allocated the greatest portion of the pixel colour. So now all we have to do is replace the declaration of v with the following:

                        Uint8 v = 0.212671f * r
                            + 0.715160f * g
                            + 0.072169f * b;

And with this, the image appears somewhat different:

This approach gives us a more even distribution of grey shades - in particular certain areas such as the trees are much lighter and we can make out the details more easily.

That's all, folks! :) In this article, we learned how to convert an image to grayscale by working on each individual pixel. To do this, we had to resort to converting an image surface to a pixel format we could work with, and then copy the pixels over to a texture for display in the window. To actually perform the grayscale conversion, we learned about bitwise operators which assisted us in dealing with the individual colours. Finally, although averaging the colour channels gives us something in terms of shades of grey, there is a formula that is used for proper grayscale conversion.

Thanks for reading. Come back for the next article! :)

SDL2: Keyboard and Mouse Movement (Events)

Hi there! :)

[Update 2015-11-14: This article is out of date. Check out the latest version at Gigi Labs.]

In this article, we'll learn how to handle keyboard and mouse events, and we'll use them to move an object around the window. Hooray! :)

We'll start with an image. I made this 64x64 bitmap:

As you can see, I can't draw to save my life. But since this is a bitmap, we don't need the SDL_image extension.

Once you have an image, you'll want to create a new Visual Studio project, wire it up to work with SDL2, and then add some code to display a window and the spaceship in it. If you don't remember how, these past articles should help:

1. "SDL2: Setting up SDL2 in Visual Studio (2013 or any other)"
2. "SDL2: Empty Window"
3. "SDL2: Displaying an Image in the Window"

You should end up with code that looks something like this:

#include <SDL.h>

int main(int argc, char ** argv)
{
    // variables

    bool quit = false;
    SDL_Event event;
    int x = 288;
    int y = 208;

    // init SDL

    SDL_Init(SDL_INIT_VIDEO);
    SDL_Window * window = SDL_CreateWindow("SDL2 Keyboard/Mouse events",
        SDL_WINDOWPOS_UNDEFINED, SDL_WINDOWPOS_UNDEFINED, 640, 480, 0);
    SDL_Renderer * renderer = SDL_CreateRenderer(window, -1, 0);

    SDL_Surface * image = SDL_LoadBMP("spaceship.bmp");
    SDL_Texture * texture = SDL_CreateTextureFromSurface(renderer,
        image);
    SDL_FreeSurface(image);

    SDL_SetRenderDrawColor(renderer, 255, 255, 255, 255);

    // handle events

    while (!quit)
    {
        SDL_WaitEvent(&event);

        switch (event.type)
        {
        case SDL_QUIT:
            quit = true;
            break;
        }

        SDL_Rect dstrect = { x, y, 64, 64 };

        SDL_RenderClear(renderer);
        SDL_RenderCopy(renderer, texture, NULL, &dstrect);
        SDL_RenderPresent(renderer);
    }

    // cleanup SDL

    SDL_DestroyTexture(texture);
    SDL_DestroyRenderer(renderer);
    SDL_DestroyWindow(window);
    SDL_Quit();

    return 0;
}

You'll also want to place spaceship.bmp into the same folder as where your main.cpp, for when you run by pressing F5 (or in the same folder as your executable if you're running it directly from there).

Once you actually run the program, you should see this:

I set the window's background to white to match the spaceship's white background by setting the clear colour using SDL_SetRenderDrawColor(), and then calling SDL_RenderClear() to clear the window to that colour.

In order to handle keyboard and mouse events, we need an SDL_Event. Well, what do you know: we have been using one all along, in order to take action when the window is closed. What we need now is to handle a different type of event type. So let's add a new case statement after the one that handles SDL_QUIT:

        case SDL_KEYDOWN:

            break;

Within this case statement, let us now determine which key was actually pressed, and move the spaceship accordingly:

        case SDL_KEYDOWN:
            switch (event.key.keysym.sym)
            {
            case SDLK_LEFT:  x--; break;
            case SDLK_RIGHT: x++; break;
            case SDLK_UP:    y--; break;
            case SDLK_DOWN:  y++; break;
            }
            break;
        }

If you now run the program, you'll find that you can move the spaceship using the arrow keys:

You'll notice that it seems to move pretty slowly, and you have to keep pressing for quite a while to make any significant movement. Now, in your code, replace SDL_WaitEvent with SDL_PollEvent:

        SDL_WaitEvent(&event);

Now, try running it again:

Swoosh! In less than half a second, the spaceship hits the edge of the window. It's actually too fast. To get this down to something manageable, add a small delay at the beginning of your while loop:

        SDL_Delay(20);

SDL_PollEvent() is better than SDL_WaitEvent() when you want to continuously check (i.e. poll) for events, but it consumes more CPU power (you can see this if you open Task Manager). SDL_WaitEvent() is okay when your window is mostly sitting idle so you don't need to check for events that often.

Handling mouse events is also very easy. All you need to do is handle the appropriate event. Let's see how to handle a mouse click:

        case SDL_MOUSEBUTTONDOWN:
            switch (event.button.button)
            {
            case SDL_BUTTON_LEFT:
                SDL_ShowSimpleMessageBox(0, "Mouse", "Left button was pressed!", window);
                break;
            case SDL_BUTTON_RIGHT:
                SDL_ShowSimpleMessageBox(0, "Mouse", "Right button was pressed!", window);
                break;
            default:
                SDL_ShowSimpleMessageBox(0, "Mouse", "Some other button was pressed!", window);
                break;
            }

And this is what happens when you run it, and click somewhere in the window:

You can also track mouse motion and obtain the current coordinates of the mouse pointer. This is useful when moving things with the mouse (e.g. moving an object by mouse-dragging it). The following code obtains the mouse coordinates and displays then in the window title:

        case SDL_MOUSEMOTION:
            int mouseX = event.motion.x;
            int mouseY = event.motion.y;

            std::stringstream ss;
            ss << "X: " << mouseX << " Y: " << mouseY;

            SDL_SetWindowTitle(window, ss.str().c_str());
            break;

Note that you'll need to add the following at the top of your main.cpp to make the above code work:

#include <sstream>

You will now notice the mouse coordinates in the window title:

Wonderful! You should now have an idea of how to capture and handle keyboard and mouse events in SDL2. We will see more of these in an upcoming article dealing with drawing pixels in the window, so stay tuned to learn more! :)