SimpleCV Blog moved.

The community has grown and so has the website.  The blog is now hosted as part of the website at:

http://www.simplecv.org

OpenCV vs. Matlab vs. SimpleCV.

Reference Article:

http://blog.fixational.com/post/19177752599/opencv-vs-matlab

Okay, I decided it’s about time I write this article.  I came across a well written OpenCV vs. Matlab article the other day and felt I needed to voice my opinion. Now you may think this article is a little biased being that I’m a SimpleCV core developer, but anyone is more than welcome to test my theories for yourself and write a rebuttal if you find anything I write to be non-truthful.  I’ve heard so many times “Why would I use simplecv, just use opencv’s python bindings”.  Okay, before I rant, I’m going to suggest you take a look at the article I mentioned before as I will basically be using that as a guide and just extending each section with my thoughts on SimpleCV.  Also the neckbeard index method is awesome as a benchmark!  I am using the ratings the previous author gave in each of the categories only adding my addition which is simplecv and where I feel it fits in this paradigm.

Before I get into the differences I will give a little background of what SimpleCV is.  It’s meant to be simple first and foremost.  It wasn’t designed for speed in use, but in actuality it runs pretty darn fast, fast enough for anyone that is new to vision to get up and running in near real time if not real time.  The best part is that it wrappers openCV, so there is no reason that you couldn’t get it working in SimpleCV first to prove it out then port the code to OpenCV so it’s blazing fast.  I will also state this now and again, we are not aiming to replace OpenCV, only make the awesome sauce that took them 10+ years to build easier to use.


*Note: I am using the same rating system from the previous article and just stating where I believe simplecv fits into that schema.


Easy of Use:

Well this is where SimpleCV reigns supreme.  It’s meant to be simple and easy. Best part is it’s free and open source, cross platform and has 1-click installers. OpenCV has 1-click installers, but doesn’t provide a development environment for you to get started in, Matlab runs cross platform as well but will cost you a shiny penny (or a whole lot of them), the code is higher level and nice to read.  Although it runs cross platform it has quite a few issues running seamlessly on apple and linux based computers.  Enough ranting, on to the code.  Here is how you load an image in simplecv.

img = Image(‘myimage.jpg’)
img.show()


SimpleCV even has a few nifty features that matlab or opencv probably don’t have like loading images directly from the web:

img = Image(‘http://www.simplecv.org/randomimg.png’)

It also supports really large images, and at the time of this article even supports newer formats like Webm/Webp which I don’t believe neither opencv nor matlab have.

Or if you want to 1-up it you get neat things for lazy people like me, load a directory of images.

imgs = ImageSet(‘/path/to/image/dir’)
imgs.show()


Neckbeard Index:
Matlab: 9/10
OpenCV: 3/10
SimpleCV: 10/10


Speed:
This is something that is typically a big factor in computer vision.  People like things to run in real time for the most part,  and although it’s possible to do in SimpleCV it is also very dependent on the image size as well as the skills of the programmer in addition to the hardware of the machine it’s running on.  When comparing to matlab, matlab is extremely slow.  OpenCV is indeed much faster as it’s written directly in C, while SimpleCV is python based (don’t underestimate python though: http://morepypy.blogspot.com/2012/02/almost-there-pypys-arm-backend_01.html).  SimpleCV although not quite as fast as OpenCV it can render a 5 megapixel image from a webcam at 30 frames a second.  If you want to get more into things like feature point detection (SURF, SIFT, etc), then you will notice a slow down, although a trick is you can always scale the image down to speed things up.  Another up side in SimpleCV is it has timing functions built in (%timeit img.findBlobs()). To give you a general idea of how long it will take.

As mentioned, the good part of SimpleCV is that it wrappers OpenCV (amongst other libraries) so you could always get your application up and running and then port it to openCV if you need the speed up.


Neckbeardindex:
Matlab: 2/10
OpenCV: 9/10
SimpleCV: 5/10



Resources Needed:
As mentioned in the previous article matlab is a resource hog.  You will need a computer that can do some processing.  As for SimpleCV that is completely the opposite.  While it may not be as fast as OpenCV out of the box, but it doesn’t add much to it’s resource usage.  In fact SimpleCV was designed to be very scalable.  We have SimpleCV running on ARM and Intel atom based devices doing real time applications as well as running on amazon EC2 instances.

Neckbeardindex:
Matlab: 4/10
OpenCV: 9/10
SimpleCV: 8/10

Cost
Matlab cost about $2,000.00 per license.  OpenCV is of course free. As well as SimpleCV.  The only advantage to using SimpleCV in this case is the cost of your time.  We have 1-click installers that will have you up and running in minutes.  And at the time of writing this article we have our web based demos coming shortly, so essentially you can try before you buy in your web browser (or spend any time installing).


Neckbeardindex:
Matlab: 4/10
OpenCV: 10/10
SimpleCV: 10/10


Development Environment:
Matlab comes with it’s own environment.  While this is nice, it also has a downside that if you don’t know the interface then there is a partial learning curve.  OpenCV is the complete opposite where you basically install the libraries and you are responsible for setting up your environment.  SimpleCV has come to a middle ground where you are provided with a bare bones environment and you can use the environment you are comfortable with (i.e. Eclipse, Textmate, Xcode, VIM, etc). as well if you want.  The barebones in SimpleCV is it’s built in shell interface.  You just type ‘simplecv’ from anywhere at your normal OS prompt and it will automatically start your development environment.  From here you can just start coding, or even follow the on screen instructions which provide you with help.  Below is an screenshot of what the shell looks like:



It also has built in support for just displaying images on the fly.  Then you don’t have to worry about if you are going to use GTK, QT, etc (although we have examples for that too).  You literally just use:

img = Image(‘simplecv’)
img.show()

and we do the rest.  Or if you want you can use built in alternative methods and display images in the browser.

img.show(‘browser’)



We also have the ability to display data in plots, similar to what you would see in matlab.

hist = img.histogram()

plot(hist)

And as of recently we now have support for ipython notebooks, which allow you to use our shell in the browser, with inline documentation as well as a bunch of other awesome libraries and tools combined.  You can rapidly prototype and test vision algorithms and code very quickly this way.




We also use native editors.  Some of us use Geany daily for development, others emacs,komodo, some vim.  In theory any editor that supports python should work fine.  We have found that the basic shell is actually very convenient and efficient in addition to a lightweight code editor.  As we progress forward we are moving towards a more out of the box friendly web support with inline shell so it will be even more like a standard matlab environment you would expect, although it’s not that far off using ipython notebooks.

Neckbeard Index:

Matlab: 8/10

OpenCV: 6/10

SimpleCV: 7/10


Memory Management

Here SimpleCV also stands proud.   Since it’s built on python, pretty much all the memory management is done for you.  This is not quite the same in opencv where it’s very easy to lose track of memory and cause the system to segmentation fault (*gulp*).  SimpleCV is much more forgiving when boo-boo’s happen in your code, it handles it gracefully like a ballerina.  So it’s very similar to matlab in the since you can be more risky and try things and fail less.

Neckbeard Index:
Matlab: 9/10
OpenCV: 4/10
SimpleCV: 9/10



Portability:
We have SimpleCV running on ARM and Intel atom based devices doing real time applications. We even have the kinect working on the pandaboard with SimpleCV in realtime, here is a picture:

As mentioned SimpleCV is meant to scale, so it also runs on Amazon EC2 instances (or any other various cloud based service should in theory work, i.e. heroku).   SimpleCV was also designed to decouple the image acquisition from processing so very soon we should have working mobile demos as well, where the image is acquired and pushed up to a EC2 instance running simplecv for processing and then sends the results back down to the mobile device.  We don’t offer as many platforms as opencv, such as native support on android or iphone. But in theory it could be ran if someone ported simplecv in jython.


Neckbeard Index:

Matlab: 3/10
OpenCV: 8/10
SimpleCV: 6/10


Development of Programming Skills

Python is hands down the glue language.  You can use python to talk to C programs, or to talk to web applications, or embedded devices.  Python is used in web frameworks, amongst many other professional grade applications (such as for google).  Of any of the tools used for coding I would state python is a very desirable programming skill to have even outside of vision circles.  While C is also a very useful language to know, I think python reaches a greater breadth of areas of possible job opportunities.


Neckbeard Index:
Matlab: 3/10
OpenCV: 8/10
SimpleCV: 9/10



Debugging

Once again I’m going to tout SimpleCV.  The simpleCV shell is your debugging environment.  In fact I come from the world of setting breakpoints, heck I even knew how to do it in PHP.  When I started writing python I wanted to code like I used to.  You open your text editor, you import your libraries at the top, you write a few functions, set your break points, save, then go to the command line, compile, hope it builds, then run it and wait until it hits a breakpoint.  Python was much different, especially using the shell that is built into SimpleCV.  You are always “in” the program while in the shell.  You don’t set break points because you are the break point.  You can define variables, functions, etc. on the fly, run them see if they break, if they do just redefine the function and run again.  You can just view variables in memory. It is by far the fastest form of coding I have ever done in my 15+ years of coding (C, C++, Java, PHP, etc).  The easiest way for me to explain it is just use an image.  Notice how I can search for the value I’m looking for quickly without recompiling, etc.




Neckbeard Index:

Matlab: 9/10
OpenCV: 5/10
SimpleCV: 9/10


Help and Sample Code:

SimpleCV aims at keeping true to it’s name, ‘simple’.  In that sense we have taken a decent amount of time at trying to help people in any way possible.  We have our help forum which is very similar to stackoverflow.  We have a mailing list.  We have a big section of examples (http://examples.simplecv.org).  We have extensive documentation (http://www.simplecv.org/doc/).  We try to update our blog frequently with various code examples or updates on progress.  We had a bunch of demonstration videos that were accidentally deleted so we are in process of redoing. We also use twitter to post updates frequently as well (http://www.twitter.com/simple_cv). All of this is accessible through the website (http://www.simplecv.org) with 1-click.

The best but last part I forgot to mention was our documentation is built into the shell as well. So you can just type ‘help function’ and see what you need without ever leaving the shell.  I don’t have much experience with matlab, but from what I do I couldn’t complain about their documentation, OpenCV has examples all over, but that’s the problem is that they are all over the place via the internet.  We are trying consolidate everything and just make it simpler and easier to use and we make it very easy for anyone to contribute to either code or documentation via github.

Neckbeard Index:
Matlab: 8/10
OpenCV: 9/10
SimpleCV: 10/10


Conclusion (TLDR; Icing on the Cake):
Well I’m a big fan of puns, so I decide to wrap up with these final words.  First a thank you to fixational for writing the previous article, which inspired me to write this one.  To say the least, matlab has it’s place, personally I don’t think it’s in the vision space, but time may prove me wrong.  Once again I’m going to state we are not trying to replace OpenCV.  OpenCV powers google’s autonomous car, it powers the vision code behind the Robot Operating system, it powers drones, humanoid robots, etc.  We ain’t gonna derail that train.  With that being said, you can think of opencv as the cake and simplecv as the icing.  We add the bling.  I suggest just giving simplecv a try, it’s quick, free, and easy, and if you don’t like it you can always switch brands.  We are a small team of programmers who love computer vision and like to have fun, which I’m very proud to be a part of.  We have fun demos like a mustachinator, animation station, kinect coloring, etc.  With that I’ll present my pun, which is a compliment to both the fun nature of SimpleCV and to the neckbeard rating system.  Someone made a beard detector with SimpleCV:
http://labs.radiantmachines.com/beard/

*Note: Some of the features mentioned don’t exist in 1.2 which only has the 1-click installers available.  1.3 will be released soon with all the above features (or can use development version). When 1.3 is released it will have 1-click installers.

If you made it this far, thanks for reading.  Feel free to find me on #simplecv irc.freednode.net, if you have questions.

-Anthony (https://github.com/xamox)

Cool New Features on Github!

In the past week we added a few cool new features to the github repo. We wanted to take a minute to tell our users about them. Most of the current work focused on getting a system for doing alpha matting. SimpleCV now has a new blit function that supports binary masks, alpha transparencies, and full alpha masks. This lets you combine two images in new and interesting ways, just like you would in your favorite image editor. Here are a few cool examples of this code in action. You can find these examples in our test suite on github. The first example is just pure binary masking.

In the image above we have two images of a couple of furry friends and black and white image. The image on the left if the top image and the image in the center is the bottom image. We are going to blit (or copy) the top image onto the bottom image, only where the mask is white. Here is what the code looks like:

 bottom.blit(top,mask=mask)

Where bottom, top, and mask are all images. This is cool and all, but you’ve been able to do this using photo editors forever. What if we wanted to create a new image the looks like a transparent copy of the top image floating on the bottom image. That is easy we can call:

bottom.blit(top,alpha=0.5)

Here alpha is a measure of how transparent the top layer will be, one is completely opaque, and zero is completely clear. That looks like this: 

But what if we wanted to vary the amount of blending between the image? To do this we can use what we call an alpha mask. An alpha mask is just a grayscale image. The whiter an area in the alpha mask is the more we will use the top image; the darker the area on the mask the more we will see the bottom image. Our alpha mask looks like this:

and the code to do this looks like this:

bottom.blit(top,alphaMask=amask) 

And the end result looks like this:

So what other cool things can you do with these new features?  Well, you can create old-school 3D images for one. The red/blue 3D movies of yore essentially used an alpha blend to place two slightly offset images on top of the other with a slight color modification for each channel. This type of 3D is called anaglyph stereo. I wrote a small script to demonstrate this principal. The inputs are two png images called left and right (from the left and right cameras or views), the result is stored in a file called threedee.png. Here are two input images I gave the script:

And here is the result (requires 3D glasses). 

We also added functions to extract alpha masks (createAlphaMask) a and binary masks (createBinaryMask) from images and then apply those masks to an image so you can get at just the data you want. 

For example, lets say we wanted to extract the Ingenuitas logo from an image. We can just create a binary mask and apply it like so:

img = Image(“logo.png”)

mask = img.createBinaryMask(color1=(0,128,128),color2=(255,255,255))

img.applyBinaryMask(mask,bg_color=Color.RED)

The result looks like this:

We could also use this technique to blit the logo onto another image. 

We’ve had a number of requests for us to add a circle feature extractor in SimpleCV. We managed to squeeze this in a week ago. It couldn’t be easier to user. Just load and image and do the following:

img = Image(“circles.png”)

circs = img.findCircles()

circs.draw()

Here is an action shot of that function at work: 

We also added a few handy utilities in the past month. For example we added a single shot white balance routine. There are two different, rather simple approaches, but they get the job done. You can access those by calling

img.whiteBalance()

The results looks like this:

Here the input image is on the right, an implementation of the gray world white balancing algorithm is in the center, and our default white balance approach is on the right. 

Finally, we added a convenience method called sideBySide. SideBySide allows you to stack images next to each other so you can really show your imaging pipeline or generate before and after pictures. For example, calling 

img = Image(“logo.png”)

img3 = Image(“plane.png”)

img.sideBySide(img3,side=’right’,scale=True)

will generate the following image:

There are a lot more features that we will hopefully cover in the next few weeks. 

The anatomy of an OpenCV Bug.

I was working on the 1.3 release of SimpleCV the other day and when I came across a problem in the OpenCV python bindings. I think this problem illustrates why new users have such a hard time with computer vision. I was trying to write a routine that generates an binary mask given either one color, or two colors that define a range of values. The algorithm is really straightforward, for the single color case you just apply binary threshold to each channel of the image and then repeat the process by incrementing the values by one, and subtracting second image from the first image. For the two color case you do the same, but the first threshold is the lesser of the two channel values, and the second threshold is the greater of the two values. I checked the OpenCV documentation before writing my routine:

http://opencv.willowgarage.com/documentation/python/imgproc_miscellaneous_image_transformations.html?highlight=threshold#Threshold

You can see here that the values for the threshold should be a float, and the maximum value (in our case 255) should also be a float. I thought this was curious, but I kept working the algorithm. I tossed it in our test suite using our logo as the test image.

My goal was to pull out each of the three color ranges in the images; the black background, the white foreground, and the blue foreground of the gear. I got some results, but not something I thought was correct. So, thinking it was an algorithm problem, I went back and looked at the algorithm and tweaked it a bit. Still the problem persisted. I then moved to storing some debugging images so I could get a feeling for where the problem was. The funny thing was that these debugging images appeared to be pretty close to correct. I worked the debugging cycle, looking at each part of the code. Finally I broke out my trusty copy of GIMP to validate what I was seeing. Since I was just applying a threshold GIMP works pretty well, here is what that looks like:

The images below are the results I was seeing. The leftmost image is the input channel, the next image to the right is the low threshold, followed by the high threshold, and finally the subtraction result.  They look close to correct, as in they get most of the colored region, and the images are different, but the subtraction is obviously wrong. 

So I start hacking the method to shreds. Here is the source code that I used for debugging (note that this is about 5x longer than it should be as I was trying to debug the entire processing pipeline):. 

https://gist.github.com/1591988

Finally, in exasperation I hardcode the threshold values to something I knew worked from using GIMP. Low and behold everything worked peachy keen. Here is an example of the red channel with hard coded values (note, I am intentionally not trying to grab the whole image):

So it turns out, after a solid hour and a half of fiddling, that the documentation is incorrect, and the method really does require integer values for its parameters. Had the method checked these values and threw and error, seg faulted, or returned a blank image, I could have had this fixed in a jiffy. Because the results were close enough to correct it took me forever to figure out the problem. I can sleep well tonight knowing that my time will hopefully make computer vision a lot easier for other SimpleCV developers. The other valuable lesson here is that if we want to make SimpleCV truly simple, we really need to provide our customers/users with documentation that is correct. It is these kind of bugs that infuriate me to no end, and every once and awhile I need a reminder about why even the “boring” stuff like writing documentation needs be done correctly.  Furthermore, seeing these errors in the wild makes me realize that our error catching needs to be bullet proof. If a user tosses us a value that doesn’t fit (and god knows python will let you do this), we need to throw a warning or error.