OpenCV is considered by many to be the best open source computer vision library. It has versions for Windows, Linux, Mac, iOS and Android. As Windows installation is usually easier, I’ll focus on Linux/Mac.

OpenCV uses cmake (Cross Platform Make) to configure and build the library. You can download cmake for the relevant platforms here (unless you’re interested in compiling it, I suggest downloading the binary distribution).

Once cmake is installed (make sure its in the path by running cmake from the command line), download the OpenCV sources for Linux/Mac (a tar’ed bz2 archive) and extract them to a directory. Next, cd to that directory and issue:

cmake .
make
sudo make install

if you need to uninstall for any reason just issue sudo make uninstall from the same directory.

On OS/X a simpler (and possibly better option) is brew install opencv (brew), which will automate all the above.