Building with oi-build

illumosoi-userland is build is OpenIndiana's primary build framework for post-oi_151 development. It is set to replace all existing consolidations, vastly simplifying how we build the operating system.

illumosoi-userland build is also tied into our continuous integration platform. When an update is committed to the illumosoi-userland build mercurial repository, an automated build is kicked off. This will then automatically publish the built package to the the /experimental repo, or generate an email alert if the build failed (to be completed).

...

oi-

...

build overview

illumosoi-userland build is a fork of Oracle's userland-gate, which we have extended and added additional software to. The layout is very similar.

Inside illumosoi-userland build is a directory called "components", under which lives a directory for each software package. Inside each of these software package directories is a main main Makefile, as well as one or more more .p5m IPS manifest files, and there may also be license files and patches.

The The Makefile essentially contains a "build recipe". To build a piece of software, you simply simply cd into the directory of the software, and type "make TARGET", where where TARGET can be one of:

• prep : Download, extract and patch the software archive
• build : Build the software
• install : Install the software into the prototype directory
• sample-manifest : Create a sample manifest file in the build directory
• publish : Publish the software to the local userland IPS repo

...

Setting up your build environment

We strongly recommend building packages inside a fresh local zone set up exclusively for building. A  A good example howto example howto on creating a build-zone can be found in the crossbow CrossBow instructions, see Building in zones.

Some suggestions about build environment (dataset layouts, etc.) can also be found in the illumos wiki: How to build illumos (note however that compilers required for the core OS may be different than those acceptable for userland software).

Ensuring your system is up to date

illumosoi-userland build requires that your system be updated with the latest software from the pkg.openindiana.org/experimental repository. If you are currently on oi_151, updating to /experimental will primarily deliver you a newer pkg5 release (currently based on 0.166).

When you want to test this out inside a virtualbox image or inside your global zone:
sudo pkg uninstall package-manager

You can do this by running

The procedure to do this is as follows:

sudo /usr/bin/pkg set-publisher -p http://pkg.openindiana.org/experimental/
sudo /usr/bin/pkg set-publisher -P oi-experimental
sudo /usr/bin/pkg set-publisher --non-sticky openindiana.org
sudo /usr/bin/pkg install -v package/pkg
sudo /usr/bin/pkg update -v

The first line adds the experimental repo to your package publisher list, the second line sets it as the primary publisher, and the third line allows packages from the experimental repo to replace those installed from the from the openindiana.org repo. The last line updates your system with the newer packages.

Note: If you are doing this in a zone and encounter an error from pkg about being unable to clone the current boot environment, you will need to update the zone from the global zone by doing:

ZONE=yourzonename
ZONEROOT=`zonecfg -z $ZONE info zonepath | awk '{print $NF}'`
sudo zoneadm -z $ZONE halt
sudo zoneadm -z $ZONE ready
export PYTHONPATH=${ZONEROOT}/root/usr/lib/python2.6/vendor-packages
sudo ${ZONEROOT}/root/usr/bin/pkg -R ${ZONEROOT}/root set-publisher -p http://pkg.openindiana.org/experimental/
sudo ${ZONEROOT}/root/usr/bin/pkg -R ${ZONEROOT}/root set-publisher -P oi-experimental
sudo ${ZONEROOT}/root/usr/bin/pkg -R ${ZONEROOT}/root set-publisher --non-sticky openindiana.org
sudo ${ZONEROOT}/root/usr/bin/pkg -R ${ZONEROOT}/root install -v package/pkg
sudo ${ZONEROOT}/root/usr/bin/pkg -R ${ZONEROOT}/root update -v
unset PYTHONPATH ZONE ZONEROOT

Installing pre-requisites

First, lets install the required software list:

sudo /usr/bin/pkg install pkg:/archiver/gnu-tar pkg:/compress/p7zip pkg:/compress/unzip \
 pkg:/developer/build/ant pkg:/developer/build/autoconf pkg:/developer/build/automake-110 \
 pkg:/developer/build/gnu-make pkg:/developer/build/libtool pkg:/developer/build/make \
 pkg:/developer/gcc-3 pkg:/developer/gnome/gettext pkg:/developer/java/jdk \
 pkg:/developer/java/junit pkg:/developer/lexer/flex pkg:/developer/macro/cpp \
 pkg:/developer/macro/gnu-m4 pkg:/developer/object-file pkg:/developer/parser/bison \
 pkg:/developer/versioning/mercurial pkg:/file/gnu-coreutils pkg:/file/gnu-findutils \
 pkg:/library/libtool/libltdl pkg:/library/libxslt pkg:/library/pcre \
 pkg:/runtime/perl-512 pkg:/system/library/math/header-math pkg:/text/gawk \
 pkg:/text/gnu-diffutils pkg:/text/gnu-gettext pkg:/text/gnu-grep \
 pkg:/text/gnu-patch pkg:/text/gnu-sed pkg:/text/groff \
 pkg:/text/texinfo pkg:/library/neon pkg:/library/apr-util-13 \
 pkg:/developer/library/lint pkg:/system/header pkg:/developer/build/onbld

Install the Sun Studio compilers (obsolete see below)

We are currently working to replace the Sun Studio compilers with gcc. For the time being, most of the software imported from Oracle's userland-gate builds with Sun Studio.

The Sun Studio compilers can be found here: http://hub.opensolaris.org/bin/view/Community+Group+tools/sun_studio_tools
if above links don't track, this direct link is available: http://download.oracle.com/otn/solaris/studio/SunStudio12u1-SunOS-x86-tar-ML.tar.bz2

export DOWNLOAD_LOCATION=$HOME
sudo mkdir /opt/SUNWspro
cd /opt/SUNWspro
sudo tar xvf $DOWNLOAD_LOCATION/sunstudio12-patched-ii-2009Sep-sol-x86.tar.bz2
sudo tar xvf $DOWNLOAD_LOCATION/sunstudio12u1-patched-ii-2010Feb-sol-x86.tar.gz \
 pkg:/data/docbook

Changes on Sun Studio compilers

Oracle just moved the Studio Compiler. Now you need to login to access "Oracle Solaris Studio 12.3" they can be found at

...

The following steps are needed to get it running:

export DOWNLOAD_LOCATION=$HOME
cd /opt
tar xjf $DOWNLOAD_LOCATION/SolarisStudio12.3-solaris-x86-bin.tar.bz2 -C ./
ln -s /opt/SolarisStudio12.3-solaris-x86-bin/solarisstudio12.3/ ./SUNWspro

To use the sun compilers you may want to

export CC=/opt/SUNWspro/bin/suncc 
export CXX=/opt/SUNWspro/bin/sunCC

where suncc is the c compiler and sunCC is the c++ compiler.

...

Downloading oi-build and preparing for first use

See the illumos wiki instructions How to build illumos for optional suggestions about build environment (dedicated ZFS dataset layouts, etc.)

Check out the repository (subdirectory oi-build must not pre-exist):

cd ~
hg clone sshhttps://anonhg@hghg.illumosopenindiana.org/illumosoi-userland/build
cd illumosoi-userlandbuild

Now we will run the setup stage, which will prepare some tools, and create an IPS IPS pkg5 repository for first use under the the i386/build directory:

cd $HOME/illumosoi-userland
gmake setup

build

# If you have the Sun Studio compilers installed
gmake setup
 
# If you don't have (access to) the Studio compilers or want to use GCC
gmake setup COMPILER=gcc

Important: Checking your environment

Make the check-environment target to check your environment is set up correctly:

...

This will report back any issues.

Adding the local repository to your publisher list

sudo /usr/bin/pkg set-publisher -p file://$HOME/oi-build/i386/repo
sudo /usr/bin/pkg set-publisher -P illumos-userland
sudo /usr/bin/pkg set-publisher --non-sticky oi-experimentalbuild

You will now be able to to pkg install the software you have built and published via via oi-build.

Optional: Running a local pkg server for installation on other zones/hosts

If you would like to use your illumosoi-userland build repository on other zones or hosts, you can run a pkg server:

/usr/lib/pkg.depotd -d $HOME/illumosoi-userlandbuild/i386/repo -p 10000

On other hosts, you can then specify http://hostname:10000 instead of the file:// address above.

See the illumos wiki instructions How to build illumos for optional suggestions about running a pkg server instance as an SMF service.

Ready to build and install your first package

...

Simply descend into the illumosthe oi-userlandbuild/components/SOFTWARE directory (where SOFTWARE is the name of the bit of software you wish to build):

Code Block
cd $HOME/illumosoi-userlandbuild/components/SOFTWARE gmake publish sudo pkg install SOFTWARE

Contributing changes back to

...

oi-

...

build

Preparing you changes for review

When you think your changes are ready for review you will need to generate a diff. The most common way to do this is '"hg diff'". It's recommended that you explain the implications of your changeset if it's not immediately obvious, and how you've tested it.

Committing changes

If you changeset has had either "2 thumbs up and 3 days open for review" or "4 thumbs up" then you are now ready to push you changes to the repo. Add and commit all files for your new component using:

cd ~/illumosoi-userlandbuild/components/SOFTWARE
hg add Makefile *.p5m <etc.>
hg commit .

...

When committing changes on behalf of someone else the commit message remains the same but you must pass the committers committer's name to to hg commit.

hg commit -u 'Contributor Name <contributor.name@fake.net>'