Monday, December 3, 2012

How to Install, Build and Use the Boost C++ libraries for development with the Eclipse IDE using the MinGW compiler on a Windows Platform

h1

How to Install, Build and Use the Boost C++ libraries for development with the Eclipse IDE using the MinGW compiler on a Windows Platform


December 04, 2012
This article outlines how to install, build and use the Boost C++ libraries for development with the Eclipse IDE using the MinGW compiler on a Windows platform.  The article will cover all steps including:
  • Downloading and installing the Eclipse C++ IDE.
  • Downloading and installing the MinGW (GCC, G++, GDB, etc) compiler toolchain.
  • Downloading and installing the Boost library sources.
  • Compiling and running a simple program that uses the Boost header files only.
  • How to build the complete set of Boost binary libraries.
  • How to configure a C++ project in Eclipse IDE to include and link the Boost C++ binary libraries.
  • Finally, we will compile and run a simple program to verify the Boost libraries are correctly installed, built and can be used by your C++ projects.

Overview

We begin by providing a brief overview of the various software components we use in this article.  All of the software components mentioned in this article are free to use for opensource and commercial development.  There are some excellent resources and tutorials included in the references section at the end of this article.  In particular, I have provided references to free online tutorials for C++ programming, and cookbook recipes for C++ and the Boost Libraries.  A cookbook is a collection of hundreds of practical solutions (recipes) to problems that programmers face, with the goal of demonstrating a language’s capabilities and the relative simplicity of implementing them.
What are the Boost C++ libraries?
The Boost C++ Libraries are a collection of free libraries that extend the functionality of C++.  The current Boost release contains over 80 individual libraries (an overview of the libraries are provided here).
Why should a developer/organization use the Boost Libraries?
  • In a word, Productivity. High-quality libraries like Boost speeds initial development, results in fewer bugs, reduces reinvention-of-the-wheel, and cuts long-term maintenance costs.
  • The Boost C++ libraries are open-source and peer-reviewed.
  • The Boost C++ libraries are platform independent and work on almost any modern operating system, including UNIX and Windows variants.
  • The Boost C++ libraries complements the Standard Template Library (STL) rather than replaces it.
  • The Boost C++ libraries are well documented.
  • Most of the Boost libraries are licensed under the Boost Software License, designed to allow Boost to be used with both free and proprietary software projects.
What is the Eclipse C++ IDE?
The Eclipse CDT Project provides a fully functional C and C++ Integrated Development Environment based on the Eclipse platform. Features include: support for project creation and managed build for various toolchains, standard make build, source navigation, various source knowledge tools, such as type hierarchy, call graph, include browser, macro definition browser, code editor with syntax highlighting, folding and hyperlink navigation, source code refactoring and code generation, visual debugging tools, including memory, registers, and disassembly viewers.
What is MinGW?
MinGW, a contraction of “Minimalist GNU for Windows”, is a port of theGNU Compiler Collection (GCC), and GNU Binutils, for use in the development of native Microsoft Windows applications. Offered in easily installed binary package format, for native deployment on MS-Windows, or user-built from source, for cross-hosted use on Unix or GNU/Linux, the suite exploits Microsoft’s standard system DLLs to provide the C-Runtime and Windows API.

1.  Installation of the Eclipse IDE

Download and install the Eclipse for C++ (CDT) IDE.
  1. Ensure you have the Java SE Runtime Environment (JRE) installed.  The Eclipse IDE requires that the Java JRE or JDK be installed in order to run.
  2. Download the Eclipse CDT IDE.  The downloaded file will have a name similar to:eclipse-cpp-helios-win32.zip
  3. Extract the downloaded file to the following folder: c:\software
  4. Thus, the full path to the eclipse installation should be:c:\software\eclipse
  5. The Eclipse IDE has an extensive inbuilt help menu with detailed user guides and tutorials.

2.  Installation of the MinGW Compiler Toolchain

Download and install the MinGW Compiler toolchain.  Previously, the various components of the GNU toolchain had to be installed individually and manually.  More recently, we can install the entire MinGW toolchain  in one operation from within the Eclipse IDE.
  1. Within the Eclipse IDE, from the menubar, select:  Help | Install New Software.
  2. Enter in the Wascana repository URL (the latest version may be obtained from here) :  http://svn.codespot.com/a/eclipselabs.org/wascana/repo
  3. Select the packages that appear and follow the on-screen prompts until the installation is complete.  The package displayed may be:Wascana C/C++ Developer for Windows
  4. The MinGW toolchain will be installed at the following location:c:\software\eclipse\mingwStep 2.1 of this article is illustrated in the following figure:
    MinGW Toolchain Installation (Wascana)
  5. Verify the Eclipse C++ IDE and MinGW toolchain are correct installed and configured by following the simple tutorial from the Eclipse IDE Welcome page.
    1. Within the Eclipse IDE, from the menubar, select: Help | Welcome.
    2. Select: Overview | C/C++ Development | C/C++ Development User Guide | Getting Started | Create a simple Application.
    3. Ensure you can complete the tutorial successfully, before proceeding on (Ensure you can successfully build and run the HelloWorld application).

3.  Installation of the Boost C++ Library Sources

Install the Boost C++ Library sources and header files.  The most important reference for the remainder of this article  is the Boost Getting Started on Windows guide.  You should read the guide in parallel with this article.  However, the guide is written with Visual Studio developers in mind.  In this article, we outline the steps for users of the Eclipse IDE with the MinGW compiler toolchain.  At the time of writing, the current version of the Boost C++ library is 1.44.0.  You should always choose to install the latest version.
  1. Download the latest version of the Boost C++ Libraries.
  2. Extract the zip file to the folder: c:\software
  3. Thus, the full path to the Boost C++ Library installation folder is: c:\software\boost_1_44_0

4.  Configure the Eclipse C++ projects to use the Boost C++ Library Headers

All Eclipse C++ projects using the Boost libraries must include the Boost C++ Library installation folder in the include paths.
Using the Getting Started on Windows guide, follow the tutorial Build a Simple Program Using Boost.  The example will use a header-only library.  The simple program reads a sequence of integers from standard input, uses Boost.Lambda to multiply each number by three, and writes them to standard output.
  1. Create a new project as previously illustrated in the HelloWorld Tutorial (suggested new project name is BoostLamdaExample1).
  2. Then create a new source file called example.cpp and cut-and-paste the sample source code from the section Build a Simple Program Using Boost of the guide.
  3. Include the Boost C++ Library installation folder in the include paths of the BoostLamdaExample1 project.
    1. Within the Eclipse IDE, within the project explorer on the left-hand-side, select the C++ project using the Boost libraries.
    2. Right-click over the C++ project and select properties from the context menu.
    3. From within the Properties dialog box, insert the full path of the boost C++ Library installation folder into the following:
      1. C/C++ Build | Settings | GCC C++ Compiler | Includes
      1. C/C++ Build | Settings | GCC G Compiler | Includes
        Step 4.3 of this article is illustrated in the following figure:
        Eclipse IDE Boost Include Path
  4. Build the project BoostLamdaExample1. (Ctrl-B)
  5. Verify the Run Configuration is correct.
    1. From the menubar, select Run | Run Configurations.
    2. Select the BoostLamdaExample1 application.
    3. Verify all the parameters appear correct.
      The Run Configuration is illustrated in the following figure:
      BoostLamdaExample1 Run Configuration
  6. Run the project BoostLamdaExample1.
  7. Enter the input data in the console tab:1 2 3
  8. Input to exit the program.
  9. If you are able to run the program successfully, then you have built and run a C++ application using a Boost header library.

5.  Compile and build the Boost C++ Binary Libraries

The Boost C++ Libraries come as source code.  Most of the Boost libraries are header-only libraries.  They consist of header files containing templates and inline functions and do not require separately compiled binary libraries or special treatment when linking.  Although most of the libraries consist solely of header files, there are some important binary libraries required and thus, they must be built.
There are prebuilt binaries of the Boost C++ library for Visual Studio (and only Visual Studio) available for free and may be downloaded from:  BoostPro
We are using the Eclipse C++ IDE with MinGW and so we must compile and build the Boost C++ binary libraries ourselves. We will use Bjam.  Bjam is the command-line tool that drives the Boost Build system. To build Boost binaries, we invoke bjam from the Boost C++ Library installation folder.  We will first update two batch files to ensure the build process will proceed smoothly.
  1. Set the toolset to GCC.
    1. Edit the file: c:\software\boost_1_44_0\bootstrap.bat
    2. Replace the following line:
      set toolset=msvcwith the line:set toolset=gcc
    3. Save and exit the file.
  2. Update the build batch file with the correct location of the MinGW toolchain (compiler and linker).
    1. Edit the file:c:\software\boost_1_44_0\tools\jam\src\build.bat
    2. Under the procedure :Guess_toolset, update the MinGW entry to point to the correct folder:
      call :Clear_Error
      if EXIST "C:\software\eclipse\mingw\bin\gcc.exe" (
      set "BOOST_JAM_TOOLSET=mingw"
      set "BOOST_JAM_TOOLSET_ROOT=C:\software\eclipse\mingw\"
      goto :eof)
    3. Optionally, remove all other blocks of code similar to the above codeblock from the procedure :Guess_toolset. This is advisable because you may find that the build procedure quickly exits with the error message saying it cannot find the compiler.  Removing all other codeblocks will ensure the compiler and toolset are set correctly.
      The following figure illustrates the complete :Guess_toolsetprocedure in my version of build.bat.
      GuessToolkit Procedure
  3. Compile and Build the Boost C++ Library binaries from the command prompt.
    1. Open up a command prompt at:  c:\software\boost_1_44_0
    2. Add the MinGW bin folder to the PATH.
      set PATH=c:\software\eclipse\mingw\bin;%PATH%
    3. Optionally, add python to the end of the PATH.  (Obviously, you may need to download and install it).
      set PATH=%PATH%;c:\python27;
    4. Enter the command:
      booststrap

      This should successfully build the executable Bjam.  If not, please review steps 5.1, 5.2 and 5.3.
    5. Build the Boost Library binaries by issuing the following command (This may take over 2hrs to complete):
      bjam --toolset=gcc  --build-dir=c:\software\boost_1_44_0\build  --build-type=complete  stage
      1. The newly built Boost C++ Binary Libraries are generated in the folder:c:\software\boost_1_44_0\stage\lib
      2. The option:--build-type=completewill generate both debug and release builds of the Boost C++ libraries.  This is suitable for developers.
      3. The option:--build-type=minimalwill generate release builds only of the Boost C++ libraries.   This may become an issue for developers who want to create debug builds of their projects.  Since the compiler automatically try to link against the debug builds of the Boost C++ Libraries, an error message is displayed.
      4. The declaration of stage or install specifies whether the Boost C++ Libraries are installed in a subfolder namedstage or system-wide. The meaning of system-wide depends on the operating system. In Windows, the target directory is c:\boost,  in Linux it is /usr/local. The target directory can also be explicitly specified using the –prefixoption.
      5. The intermediate files are generated in the build folder:c:\software\boost_1_44_0\buildYou may delete the build folder when the entire build process has completed.
      6. If you have a multi-core processor, you can accelerate the build process.  The option:-j Nwill run up to n shell commands concurrently. The default is 1.  I have an Extreme Quad-Core processor, and adding the option -j 4 reduced the build time from just over 2hrs to just under 40 minutes.

6.  Configure the Eclipse C++ projects to use the Boost C++ Binary Libraries

  1. All Eclipse C++ project using the Boost C++ binary libraries must include the Boost Binary Library folder in the Libraries (-L) settings.
    1. From with the Eclipse IDE, right-click on the selected project and select Properties.
    2. Place the Boost Binary Library folder path c:\software\boost_1_44_0\stage\lib in the following location:
      C/C++ Build | Settings | MinGW C++ Linker | Libraries | Library search path (-L).
  2. All Eclipse C++ projects using a specific Boost C++ binary library must include the specific binary library in the Libraries (-l) settings.
    1. From with the Eclipse IDE, right-click on the selected project and select Properties.
    2. Place the name of the specific binary library in the following location:
      C/C++ Build | Settings | MinGW C++ Linker | Libraries | Libraries (-l).
    3. IMPORTANT: To link a Boost C++ Binary library, e.g.:libboost_regex-mgw44-1_44.athe command line option should be:-lboost_regex-mgw44-1_44(i.e.: Remove the prefix lib and the postfix .a).
  3. Step 6 in this article is illustrated in the following figure:
    Boost Binary Library Folder Configuration

7.  Final step – Verify we can use the Boost C++ Binary Libraries from a sample application.

  1. Using the Getting Started on Windows guide, follow the example Link Your Program to a Boost Library.  To demonstrate linking with a Boost binary library, the sample program extracts the subject lines from emails. It uses the Boost.Regex library, which has a separately-compiled binary component.
  2. Create a new C++ project in Eclipse called BoostRegexExample1 and create a new source file called regex.cpp.
  3. Cut-and-paste the sample code from the section Link Your Program to a Boost Library of the guide into regex.cpp.
  4. Add the Boost installation folder to the Include Path of theBoostRegexExample1 project as indicated in step 4.3 of this article.
  5. Add the Boost Library binary folder to the Library (-L) Path of theBoostRegexExample1 project as indicated in step 6.1 of this article.
  6. Add the specific Boost Regex Binary Library boost_regex-mgw44-1_44to the library (-l) settings as indicated in step 6.2 of this article.
    1. To find the name of a specific library, please browse the Boost Library binary folder, and select the library filename such that it is the shortest filename with the prefix libboost_regex. The similar looking filenames refer to the same library compiled with different options (statically linked, dynamically link, release-build, debug-build, and so on).
  7. Compile and build (Ctrl-B) the project:  BoostRegexExample1.
  8. Verify the Run Configuration parameters are correct:
    1. From the Eclipse IDE menubar, select Run | Run Configurations.
    2. Ensure the BoostRegexExample1 C++ application is selected.
    3. Ensure the Build configuration is set to Debug.
    4. Click Run.
  9. Cut-and-paste the following sample email into the console as input:
    To: Bob
    From: Mary
    Subject: The test is successful
    Body: Great news, Boost Library integration works.
  10. If the following message is printed out, then the program ran successfully (using the Boost Binary Library Regex).
    The test is successful
  11. To exit the program, input Ctrl-Z to the console.
  12. Congratulations! You may now use the Boost Libraries in your C++ application from the Eclipse CDT IDE using the MinGW toolchain.

References

  • Eclipse C++ IDE – Free fully functional C and C++ Integrated Development Environment.
  • Boost C++ Libraries – Free peer-reviewed portable C++ source libraries.
  • MinGW -  Free port of the GNU Compiler Collection (GCC), and GNU Binutils, for use in the development of native Microsoft Windows applications.
  • Wascana - Wascana is the Eclipse C/C++ IDE for the Windows along with the MinGW GNU toolchain.
  • LearnCpp.com – Free website devoted to teaching you to program in C++.
  • Online C++ Cookbook. – Excellent resource for Java, C++, C#, Oracle and SQL developers.
  • The Boost C++ Libraries Book – An online book providing an overview on using the Boost C++ libraries.  The book is released under aCreative Commons License.
  • cdtdoug.blogspot.com – Personal blog of the Eclipse CDT project  leader providing insights into his mind and the future directions of the project.

Wednesday, January 18, 2012

Setting up a server for PXE network booting

If you're looking to perform a lot of system recovery, or system installation, then network booting with PXE is ideal. PXE allows you to boot up a system and have it automatically get an IP address via DHCP and start booting a kernel over the network.
PXE itself stands for "Pre-boot eXecution Environment", which describes how it works in the sense that the clients using it haven't booted in a traditional manner.
In order to use PXE you need to setup a boot-server which will allow client systems to :
  • Request an IP address (via DHCP)
  • Download a kernel (via TFTP)
With both of these services in place any system which supports PXE/network-booting (you might need to enable it in the BIOS) should be able to gain an IP address, fetch a kernel, and boot without an installed operating system.
(This is ideal for systems which can't be booted by a traditional approach; for example your new AMD-64 system which doesn't have a CD/DVD drive!)
Our Setup
For the purposes of this article we'll assume:
  • We're working with a small network 192.168.1.0/24
  • We'll allow all local machines to boot and get an IP address via DHCP from the range 196.168.1.70-192.168.1.100
  • Our "boot-server" is the host "prashant" with IP address 192.168.1.50
  • We will serve the same kernel to each host.
In our example we'll configure a PXE-server which will allow remote systems to run the Debian Etch installer, but nothing here is specific to that. PXE allows you to configure a system to boot from an arbitrary kernel (and matching ramdisk if you wish to use one). With the correct configuration you can even cause the clients to mount a remote file-system via NFS and have a diskless thin-client system.
TFTP Setup
TFTP is a very simple file-transfer protocol, which is very similar to FTP but which doesn't use any kind of authentication. If you're not already running a TFTP server you can install one by running:
root@prashant:~# apt-get install tftpd-hpa 
Once installed you'll need to enable it by editing the file /etc/default/tftpd-hpa. You should change RUN_DAEMON to yes, leaving you with contents like these:
#Defaults for tftpd-hpa RUN_DAEMON="yes" OPTIONS="-l -s /var/lib/tftpboot" 
Now create the root directory, if it is missing, and start the server:
root@prashant:~# mkdir -p /var/lib/tftpboot 
root@prashant:~# /etc/init.d/tftpd-hpa start Starting HPA's tftpd: in.tftpd. 
Once our systems have retrieved an IP address via DHCP they will request files from beneath the /var/lib/tftpboot root directory. We'll come back to the contents of this directory shortly.
DHCP Setup
If you don't already have a DHCP server configured upon your LAN you'll need to install one. If you're using a small home router, or similar, to provide local DHCP services you must disable this first. (Since we require the DHCP server to pass back some extra options to clients which the majority of routers won't allow).
Discussing a full DHCP installation is mostly beyond the scope of this introduction but the thing we're trying to do is fairly simple. The goal of the DHCP server in this setup is twofold:
  • We obviously want to use it to allow clients to request and receive an IP address.
  • We want to cause the DHCP "answer" to give some extra details to the clients which are requesting an address:
    • The address of the TFTP server.
    • The initial filename to request from the TFTP server.
The most common DHCP server is the dhcp-server package, and you can install this by running:
root@prashant:~# apt-get install dhcp3-server 
Once installed the server is configured in the file /etc/dhcp3/dhcpd.conf, and there are a lot of available options described there. For our example we'll use the following configuration:
option domain-name-servers 62.31.64.39, 62.31.112.39; 
default-lease-time 86400; max-lease-time 604800; authoritative;  

subnet 192.168.1.0 netmask 255.255.255.0 {range 192.168.1.70 192.168.1.100;

filename "pxelinux.0";

next-server 192.168.1.50;

option subnet-mask 255.255.255.0;

option broadcast-address 192.168.1.255;
option routers 192.168.1.1; } 
Here we've configured the server to give out IP addresses from the range 192.168.1.70-100, set the default gateway to be 192.168.1.1 and use our ISP's nameservers.
We've also used next-server to point to the TFTP server we're using (which is the same host as our DHCP server, but doesn't need to be). We've chosen the default name of pxelinux.0 as the name of the file for booting clients to request.
Using dnsmasq instead
Personally I use the dnsmasq package to provide DHCP services upon my LAN, since this is small and simple and provides other useful abilities, setting up PXE booting with dnsmasq just requires the addition of the following line to /etc/dnsmasq.conf:
dhcp-boot=pxelinux.0,prashant,192.168.1.50 
(Again we've setup the filename along with the name and IP address of the TFTP server which is "prashant" / 192.168.1.50 in this example)
Restarting the service after this change is as simple as:
root@prashant:~# /etc/init.d/dnsmasq restart Restarting DNS forwarder and DHCP server: dnsmasq. 
PXE Configuration
Now that we've configured the TFTP and DHCP servers we need to go back and complete the configuration. By default when a client boots up it will use its own MAC address to specify which configuration file to read - however after trying several options it will fall back to requesting a default file.
We need to create that that file, which will contain the list of kernels which are available to boot, we'll firstly need to create a directory to hold it:
root@prashant:~# mkdir /var/lib/tftpboot/pxelinux.cfg 
Now save the following as /var/lib/tftpboot/pxelinux.cfg/default:
DISPLAY boot.txt  DEFAULT etch_i386_install  LABEL etch_i386_install         

kernel debian/etch/i386/linux         

append vga=normal initrd=debian/etch/i386/initrd.gz  -- LABEL etch_i386_linux         

kernel debian/etch/i386/linux         append vga=normal initrd=debian/etch/i386/initrd.gz  --  LABEL etch_i386_expert         

kernel debian/etch/i386/linux         append priority=low vga=normal initrd=debian/etch/i386/initrd.gz  --  LABEL etch_i386_rescue         

kernel debian/etch/i386/linux         append vga=normal initrd=debian/etch/i386/initrd.gz  rescue/enable=true --  PROMPT 1 TIMEOUT 0 
This file instructs the client to display the contents of the file boot.txt so create that too:
- Boot Menu - =============  etch_i386_install etch_i386_linux etch_i386_expert etch_i386_rescue 
The only remaining job is to download the official Etch installer kernel and associated files and save them in the directories specified in the default file we created:
root@prashant:~# cd /var/lib/tftpboot/ 

root@prashant:~# wget http://ftp.uk.debian.org/debian/dists/etch/main/installer-i386/current/images/netboot/debian-installer/i386/pxelinux.0  

root@prashant:~# mkdir -p /var/lib/tftpboot/debian/etch/i386 

root@prashant:~# cd /var/lib/tftpboot/debian/etch/i386 

root@prashant:~# wget http://ftp.uk.debian.org/debian/dists/etch/main/installer-i386/current/images/netboot/debian-installer/i386/linux 

root@prashant:~# wget http://ftp.uk.debian.org/debian/dists/etch/main/installer-i386/current/images/netboot/debian-installer/i386/initrd.gz 
When these commands have been completed we'll have the following structure:
root@prashant:~# tree /var/lib/tftpboot/

 /var/lib/tftpboot/ 

|-- boot.txt 

|-- debian 

|   `-- etch 

|       `-- i386 

|           |-- initrd.gz 

|           `-- linux 

|-- pxelinux.0 

`-- pxelinux.cfg     

`-- default   
4 directories, 5 files

 
(We only used debian/etch here in case we want to offer other installers in the future. You can put everything in one directory if you wish, just update pxelinux.cfg/default to match.)
We should now be ready to test the setup.
We've installed a pxelinux.0 file which will instruct booting clients to request pxelinux.cfg/default. This will then make a list of boot options available, which are displayed by the simple boot menu file we created.
The files which are used for booting are stored beneath the TFTP root directory and thus accessible to booting clients.
Sample Run
A sample remote boot looks like this:
PXE entry point found (we hope) at 9AE5:00D6 

My IP address seems to be C0A80146 192.168.1.70 

FTFTP prefix: 

Trying to load: pxelinux.cfg/01-00-14-22-a1-53-85 

Trying to load: pxelinux.cfg/C0A80146 

Trying to load: pxelinux.cfg/C0A8014 

Trying to load: pxelinux.cfg/C0A801 

Trying to load: pxelinux.cfg/C0A80 

Trying to load: pxelinux.cfg/C0A8 

Trying to load: pxelinux.cfg/C0A 

Trying to load: pxelinux.cfg/C0 

Trying to load: pxelinux.cfg/C 

Trying to load: pxelinux.cfg/default 

- Boot Menu - 

=============  

etch_i386_install 

etch_i386_linux 

etch_i386_expert 

etch_i386_rescue 
As you can see the system here attempted to load several configuration files, based upon its MAC address (01-00-14-22-a1-53-85) initially then falling back to the octets in the IP address it was given by DHCP (192.167.1.70).
Finally it managed to load a working configuration using the last-chance default file we created. This in turn instructed it to show the boot menu we created.
From here on the system will boot into whichever kernel you specify. (We could configure the system to time-out here and just boot into a default option, but we didn't.)
From here on you should understand how PXE can be used to boot an arbitrary kernel and initial ram-disk. Later we'll look at mounting a remote file-system over NFS to provide a disk-less thin-client.