SDL2: Converting an Image to Grayscale

Hello folks! :)

[Update 2015-11-14: This article is out of date. Check out the latest version at Gigi Labs.]

In the previous article, "SDL2: Pixel Drawing", we saw how to draw pixels onto a blank texture that we created in code. Today, on the other hand, we'll see how we can manipulate pixels on an existing image, such as a photo we loaded from disk. We'll also learn how to manipulate individual bits in an integer using what are called bitwise operators, and ultimately we'll convert an image to grayscale.

We first need to set up an SDL2 project. After following the steps in "SDL2: Setting up SDL2 in Visual Studio (2013 or any other)", you will also need to add SDL2_image.lib to your Linker Input (so that it reads "SDL2.lib; SDL2main.lib; SDL2_image.lib", and place the Visual C++ development libraries obtained from the SDL_image 2.0 homepage into the appropriate folders of your SDL2 directory. After you build your project the first time, make sure you also place all the necessary DLLs (including SDL2.dll, SDL2_image.dll and all the rest) into the same folder as your executable - see "SDL2: Loading Images with SDL_image" in case you need a refresher.

And in fact we're going to start with the code from "SDL2: Loading Images with SDL_image", which is the following (adapted a little bit):

#include <SDL.h>
#include <SDL_image.h>

int main(int argc, char ** argv)
{
    bool quit = false;
    SDL_Event event;

    SDL_Init(SDL_INIT_VIDEO);
    IMG_Init(IMG_INIT_JPG);

    SDL_Window * window = SDL_CreateWindow("SDL2 Grayscale",
        SDL_WINDOWPOS_UNDEFINED, SDL_WINDOWPOS_UNDEFINED, 640, 480, 0);
    SDL_Renderer * renderer = SDL_CreateRenderer(window, -1, 0);
    SDL_Surface * image = IMG_Load("PICT3159.JPG");
    SDL_Texture * texture = SDL_CreateTextureFromSurface(renderer,
        image);

    while (!quit)
    {
        SDL_WaitEvent(&event);

        switch (event.type)
        {
        case SDL_QUIT:
            quit = true;
            break;
        }

        SDL_RenderCopy(renderer, texture, NULL, NULL);
        SDL_RenderPresent(renderer);
    }

    SDL_DestroyTexture(texture);
    SDL_FreeSurface(image);
    SDL_DestroyRenderer(renderer);
    SDL_DestroyWindow(window);
    IMG_Quit();
    SDL_Quit();

    return 0;
}

We can also use the same sample image as in that project, so make sure it's in the appropriate folders where Visual Studio can find it.

Now, let's get down to business.

You see, the problem here is that we can't quite touch the texture pixels directly. So instead, we need to work a little bit similarly to "SDL2: Pixel Drawing": we create our own texture, and then copy the surface pixels over to it. So we throw out the line calling SDL_CreateTextureFromSurface(), and replace it with the following:

    SDL_Texture * texture = SDL_CreateTexture(renderer,
        SDL_PIXELFORMAT_ARGB8888, SDL_TEXTUREACCESS_STATIC,
        image->w, image->h);

Then, at the beginning of the while loop, add this:

        SDL_UpdateTexture(texture, NULL, image->pixels,
            image->w * sizeof(Uint32));

If you try and run program now, it will pretty much explode. That's because our code is assuming that our image uses 4 bytes per pixel (ARGB - see "SDL2: Pixel Drawing"). That's something that depends on the image, and this particular JPG image is most likely 3 bytes per pixel. I don't know much about the JPG format, but I'm certain that it doesn't support transparency, so the alpha channel is out.

The good news is that it's possible to convert the surface into one that has a familiar pixel format. To do this, we use SDL_ConvertSurfaceFormat(). Add the following before the while loop:

    image = SDL_ConvertSurfaceFormat(image, SDL_PIXELFORMAT_ARGB8888, 0);

What this does is take our surface (in this case the one that image points to) and return an equivalent surface with the pixel format we specify. Now that the new image has the familiar ARGB format, we can easily access and manipulate the pixels. Add the following after the line you just added (before the while loop) to typecast the surface pixels from void * to Uint32 * which we can work with:

    Uint32 * pixels = (Uint32 *)image->pixels;

So far so good:

Now, let's add some code do our grayscale conversion. We're going to convert the image to grayscale when the user presses the 'G' key, so let us first add some code within the switch statement to handle that:

        case SDL_KEYDOWN:
            switch (event.key.keysym.sym)
            {
            case SDLK_g:
                for (int y = 0; y < image->h; y++)
                {
                    for (int x = 0; x < image->w; x++)
                    {
                        Uint32 pixel = pixels[y * image->w + x];
                        // TODO convert pixel to grayscale here
                    }
                }
                break;
            }
            break;

This is where bit manipulation comes in. You see, each pixel is a 32-bit integer which in concept looks something like this (actual values are invented, just for illustration):

Alpha	Red	Green	Blue
11111111	10110101	10101000	01101111

So let's say we want to extract the Red component. Its value is 10110101 in binary, or 181 in decimal. But since it's in the third byte from right, its value is much greater than that. So we first shift the bits to the right by 16 spaces to move it to the first byte from right:

		Alpha	Red
00000000	00000000	11111111	10110101

...but we still can't interpret the integer as just red, since the alpha value is still there. We want to extract just that last byte. To do that, we perform a bitwise AND operation:

Pixel	11111111 10110101
Mask	00000000 11111111
Red AND Mask	00000000 10110101

We do an AND operation between our pixel value and a value where only the last byte worth of bits are set to 1. That allows us to extract our red value.

In code, this is how it works:

                        Uint8 r = pixel >> 16 & 0xFF;
                        Uint8 g = pixel >> 8 & 0xFF;
                        Uint8 b = pixel & 0xFF;

The >> operator shifts bits to the right, and the & is a bitwise AND operator. Each colour byte is shifted to the last byte and then ANDed with the value 0xFF, which is hexadecimal notation for what would be 255 in decimal, or 11111111 in binary. That way, we can extract all three colours individually.

We can finally perform the actual grayscaling operation. A simple way to do this might be to average the three colours and set each component to that average:

                        Uint8 v = (r + g + b) / 3;

Then, we pack the individual colour bytes back into a 32-bit integer. We follow the opposite method that we used to extract them in the first place: they are each already at the last byte, so all we need to do is left-shift them into position. Once that is done, we replace the actual pixel in the surface with the grayscaled one:

                        pixel = (0xFF << 24) | (v << 16) | (v << 8) | v;
                        pixels[y * image->w + x] = pixel;

If we now run the program and press the 'G' key, this is what we get:

It looks right, doesn't it? Well, it's not. There's an actual formula for calculating the correct grayscale value (v in our code), which according to Real-Time Rendering is:

Y = 0.212671R + 0.715160G + 0.072169B

The origin of this formula is beyond the scope of this article, but it's due to the fact that humans are sensitive to different colours in different ways - in fact there is a particular affinity to green, hence why it is allocated the greatest portion of the pixel colour. So now all we have to do is replace the declaration of v with the following:

                        Uint8 v = 0.212671f * r
                            + 0.715160f * g
                            + 0.072169f * b;

And with this, the image appears somewhat different:

This approach gives us a more even distribution of grey shades - in particular certain areas such as the trees are much lighter and we can make out the details more easily.

That's all, folks! :) In this article, we learned how to convert an image to grayscale by working on each individual pixel. To do this, we had to resort to converting an image surface to a pixel format we could work with, and then copy the pixels over to a texture for display in the window. To actually perform the grayscale conversion, we learned about bitwise operators which assisted us in dealing with the individual colours. Finally, although averaging the colour channels gives us something in terms of shades of grey, there is a formula that is used for proper grayscale conversion.

Thanks for reading. Come back for the next article! :)

VB .NET Basics: Conditionals, Logical Operators and Short-Circuiting

Hello again! :)

Two months ago, I wrote the article "VB .NET Basics: Input and Output", which has recently been gaining a lot of attention. In today's article, we're going to see how we can write conditional statements in VB .NET to let the program take different actions depending on user input. More importantly, we're going to see how to use logical ANDs and ORs in 'If' conditions - something that is likely to catch you off guard if you're coming from some other language.

First, create a new VB .NET Console Application. If you're using SharpDevelop, the project will be created with the following initial code:

Module Program
    Sub Main()
        Console.WriteLine("Hello World!")
       
        ' TODO: Implement Functionality Here
       
        Console.Write("Press any key to continue . . . ")
        Console.ReadKey(True)
    End Sub
End Module

Let's start off with a simple conditional. Replace the code in Main() wit the following:

        Console.WriteLine("Are you male or female?")
        Dim genderStr As String = Console.ReadLine();

This just asks the user about his/her gender and collects their input in a string. Now, we'll use that input as follows:

        If genderStr.Substring(0, 1).ToLower() = "m" Then
            Console.WriteLine("Hello sir!")
        End If

We're taking the first character from the user's input (thanks to the Substring() method), converting it to lowercase. That way, whether the user types "Male", "male", or "m126", it's still going to interpret that as being male.

There are a couple of things to note here. First of all, in VB .NET, the = operator (which is the same one used in assigning values to variables) is used for equality comparison. This is quite different from C-like languages which use = for assignment and == for comparison.

Secondly, VB .NET is sensitive to spacing. If you try to put the Then on the next line, like this:

        If genderStr.Substring(0, 1).ToLower() = "m"
            Then
            Console.WriteLine("Hello sir!")
        End If

...you'll get a syntax error. You can, however, stuff the conditional in one line and omit the End If, like this:

        If genderStr.Substring(0, 1).ToLower() = "m" Then Console.WriteLine("Hello sir!")

Okay, since the feminists must be grumbling at this point, let's replace our conditional with this:

        If genderStr.Substring(0, 1).ToLower() = "m" Then
            Console.WriteLine("Hello sir!")
        ElseIf genderStr.Substring(0, 1).ToLower() = "f" Then
            Console.WriteLine("Hello miss!")
        Else
            Console.WriteLine("Hello alien!")
        End If

You might be familiar with Else from other languages, but this ElseIf might possibly be new. ElseIf makes its condition (the one checking for female) run only if the initial If returned false - it's the equivalent of writing the following:

        If genderStr.Substring(0, 1).ToLower() = "m" Then
            Console.WriteLine("Hello sir!")
        Else
            If genderStr.Substring(0, 1).ToLower() = "f" Then
                Console.WriteLine("Hello miss!")
            Else
                Console.WriteLine("Hello alien!")
            End If
        End If

...but when you have lots of conditions, it saves you from having to indent indefinitely. ElseIf exists in other languages, but can vary quite a lot between them (e.g. elif in Python, elsif in Perl, or even just else if in the C-like languages).

Of course, when we are checking the same variable over different values, we know we should actually be using a Select Case statement, which is the equivalent of a switch statement in C# and other C-like languages. We can change the above conditional into a Select Case as follows:

        Select Case genderStr.Substring(0, 1).ToLower()
            Case "m":
                Console.WriteLine("Hello sir!")
            Case "f":
                Console.WriteLine("Hello miss!")
            Case Else
                Console.WriteLine("Hello alien!")
        End Select

Select Case is syntactically quite different from switch statements. In particular, there are no break statements, and Case Else is the equivalent of the default statement in C-like languages: its statements are executed if none of the earlier cases are matched.

We can now add the following to wait for user input before exiting the application:

        Console.ReadLine()

...and then test it (I used Ctrl+F5 to run, instead of debug, the application - this lets you run several instances at the same time):

Now, let's see how to write more interesting conditional statements. To do this, let us first collect the user's age:

        Dim message As String = Nothing
        Dim ageStr As String = Console.ReadLine()
        Dim age As Integer = 0
        Dim converted As Boolean = Integer.TryParse(ageStr, age)

So, first we're declaring a message variable, which we'll set in a minute. It's initially set to Nothing, which is equivalent to C# null. Then we're accepting user input, and converting it to a number. We use Integer.TryParse() so that if the conversion fails, it simply returns false and age is not set - this saves us from having to do exception handling as in "C# Basics: Arithmetic and Exceptions".

You might guess that we can logically combine conditional statements using the And and Or statements. Let's try that out:

        If converted Then
            If genderStr.Substring(0, 1).ToLower() = "m" And age < 30 Then
                message = "You are a young man!"
            ElseIf genderStr.Substring(0, 1).ToLower() = "m" And age >= 30 Then
                message = "You are an old man!"
            ElseIf genderStr.Substring(0, 1).ToLower() = "f" And age < 30 Then
                message = "You are a young lady!"
            ElseIf genderStr.Substring(0, 1).ToLower() = "f" And age >= 30 Then
                message = "You are an old lady!"
            ElseIf genderStr.Substring(0, 1).ToLower() = "a" Then
                message = "Whatever your age, you are an alien."
            End If
        End If

Heh, if you're 30 or over, you're probably shaking your fists. But anyway, since we're dealing with multiple variables, we can't use a Select Case statement here. Instead, we are checking different combinations of gender and age. Note that I haven't included an Else statement, and this means that if the age conversion fails, then message remains Nothing.

Now, let's show the message. We can handle the case where message is Nothing with another conditional:

        If message IsNot Nothing And message.Length > 0 Then
            Console.WriteLine(message)
        End If

Okay. Let's begin testing the application:

That worked pretty well, didn't it? Now let's try an invalid age:

Uh oh. What just happened here? Despite the check against Nothing, accessing message.Length still caused a NullReferenceException to be thrown. Why is that? Let's look at our condition again:

         If message IsNot Nothing And message.Length > 0 Then

If message is Nothing, then the first condition resolves to false. At this point we normally expect the second expression to be ignored (which is called short circuiting), but that is not what is happening. The And operator evaluates both expressions, and only then ANDs them together.

In other words, we're using the wrong operator. We need to use AndAlso instead of And, and OrElse instead of Or. AndAlso and OrElse will short circuit (i.e. they won't evaluate both expressions if they don't have to), while And and Or are bitwise operators and mean something different (which we won't go into at this stage). If you're coming from any C-style language such as C#, then you can understand this as follows: AndAlso and OrElse are the equivalents of && and || respectively; while And and Or are the equivalents of & and | respectively.

We can now change our code to use AndAlso instead of And:

        If converted Then
            If genderStr.Substring(0, 1).ToLower() = "m" AndAlso age < 30 Then
                message = "You are a young man!"
            ElseIf genderStr.Substring(0, 1).ToLower() = "m" AndAlso age >= 30 Then
                message = "You are an old man!"
            ElseIf genderStr.Substring(0, 1).ToLower() = "f" AndAlso age < 30 Then
                message = "You are a young lady!"
            ElseIf genderStr.Substring(0, 1).ToLower() = "f" AndAlso age >= 30 Then
                message = "You are an old lady!"
            ElseIf genderStr.Substring(0, 1).ToLower() = "a" Then
                message = "Whatever your age, you are an alien."
            End If
        End If
       
        If message IsNot Nothing AndAlso message.Length > 0 Then
            Console.WriteLine(message)
        End If

And now, this code doesn't raise any exceptions (there is no message written because that's the way the logic works):

Great. In this article, we learned how to use conditional statements in Visual Basic .NET. We started by looking at simple If and Select Case statements, and then discovered how to set up more complex conditions by using Boolean operators to combine simple conditions. The main takeaway from this is that you should always use AndAlso and OrElse in your conditions, not And and Or. This will allow your conditions to short circuit, i.e. if your condition is (A AND B) and A is false, then the condition is false regardless of the value of B, so B is not evaluated. Short circuiting is very important when the second condition is dependent on the first, as we have seen in the above code.