In this writeup I'll try to document how to use CrossBow virtual networking and some other components to create a LAN-in-a-box.
We will set up internet access for local zones (LZ), such as when you're Building in zones, for cases when your corporate or home LAN does not provide for these zones' individual network addresses and only your machine's global zone (GZ) has an "external" IP address assigned. The same setup can be used for your testbed virtual machines, as well as for easier creation of networked local zones on an OI installation implemented as a VM in VirtualBox (which sets some limitations on networking choices).
Here are the major steps:
It is arguable whether the building local zones need the network access at all, but it might be needed for example if you maintain a golden source-code repository in the global zone and want to distribute the changesets to local zones' cloned workspaces (see Building in zones, Working on several bugs at once and Making a LAN mirror of global package repositories), or if you want to upload webrevs straight from your build zones. YMMV, this here is just a technology POC
It is also worth noting that some best-practice setups use a similar approach, with a dedicated local zone being the router/firewall/NAT and it has the external IP address, and the global zone "hypervisor" being one of the privately-addressed nodes which have the internet access through such local zone. In particular, this may be the only way to have ip-type=shared
local zones to access the external network using NAT through the same box. It is not difficult to transition from the setup described below to such a best-practice one by swapping some commands and VNICs (Virtual Network Interface Cards), though the adventure is left for the curious and won't be detailed in this page.
In particular, this setup will focus on:
in.dhcpd
) with text-file storage format;(So far) this document does not deal with configuration of naming services, though some pointers can be given anyway
One solution is to use a private DNS server on the host machine for the zones to know each other's names and private IP addresses, which would forward requests for other names to the LAN DNS servers. See Solaris DHCP Administration Guide Chapter 4 for details on enabling the registration of DHCP clients in the ISC BIND server by the Sun DHCP server, and optionally allowing the DHCP client to push its desired hostname via DHCP server to DNS.
An alternate useful configuration might be to use DNS for remote hostname look-ups, and allow the DHCP server to update its host's /etc/hosts
file with information about client hostnames, but this is not explored here either.
In this page it is assumed that the local zones receive their default hostnames via zoneadm install
(`zonename`
or ultimately /etc/nodename
), and can be configured to use external DNS servers for internet/LAN access, and they can address each other and their GZ via /etc/hosts
file if that is at all needed.
For the purposes of the setup described below:
192.168.127.0/24
, on a dedicated etherstub, with192.168.127.1
, and192.168.1.40
on a e1000g0
interface;In many of the examples below, samples of commands to run as "root
" (or via pfexec
) are prefixed with a ":;
" prompt.
Basically this prepends a call of "true
" to the sample command and does nothing, unlike the typically used "hash" (#
) prompt which acts as a comment and precludes the copy-pasted command blocks from running.
This chapter focuses on creating the host-only network, which allows using a private IP address range for local zones to communicate with each other and with the global zone.
This by itself does not provide communications with the outside world, and you may have your own reasons to stop the network configuration at that point, and proceed to setting up the local zone(s). In that case the local zones would only be able to access each other and the global zone, using the VNICs connected to a common "etherstub".
:; dladm create-etherstub localstub127
This provides a host-only Ethernet segment, acting like a virtual switch, named "localstub127
".
:; dladm create-vnic -l localstub127 vnic127001 :; dladm create-vnic -l localstub127 -m 00:12:71:27:01:01 vnic127101
The first line defines a VNIC for GZ named "vnic127001
" with a random MAC address, attached to the virtual switch created above.
The second line creates a VNIC for an LZ, attached to the same switch with a predefined MAC address, others can be defined similarly.
Here I use a simple naming scheme, with 127
standing for the subnet and 001
standing for the static IP address (192.168.127.1
) that will be assigned to the VNIC for the global zone.
Similarly the 101
characters in the vnic127101
name and 00:12:71:27:01:01
in the MAC address stand for the IP address (192.168.127.101
) that will be assigned to the VNIC via DHCP – fixed addressing, even if distributed with dynamic methods, is useful for setting up /etc/hosts
and the publishing of services via NAT.
A simple construct like the one below would pre-create some 9 VNICs to this pattern quickly:
:; for IP in 01 02 03 04 05 06 07 08 09; do \ dladm create-vnic -l localstub127 -m 00:12:71:27:01:${IP} vnic1271${IP}; done
:; grep "192\.168\.127\." /etc/netmasks || \ echo "192.168.127.0 255.255.255.0" >> /etc/netmasks :; ipadm create-if vnic127001 && \ ipadm create-addr -T static -alocal=192.168.127.1 vnic127001/v4 ### Legacy ways: # echo "192.168.127.1" > /etc/hostname.vnic127001 # ifconfig vnic127001 plumb 192.168.127.1/24 up
The first two lines take care of saving the VNIC addressing for the future. The last line brings up the new IPv4 interface and sets up its addressing.
Quick check (note that etherstub default MTU is 9000, equivalent of Jumbo Frames):
:; ifconfig vnic127001 vnic127001: flags=1000843<UP,BROADCAST,RUNNING,MULTICAST,IPv4> mtu 9000 index 3 inet 192.168.127.1 netmask ffffff00 broadcast 192.168.127.255 ether 2:8:20:87:e4:43
First of all, the DHCP server software should be installed. In this example I'll use Sun BOOTP/DHCP server software (alternately ISC DHCP can be used, for example).
:; pkg install service/network/dhcp :; svccfg import /lib/svc/manifest/network/dhcp-server.xml :; svcs -a | grep -i dhcp disabled 16:51:37 svc:/network/dhcp-server:default
binfiles
storage backend for DHCP configuration to be available, but I find inspecting and manipulating text files to be easier :; pkg install service/network/dhcp/datastore/binfiles
:; pkg install service/network/dhcp network/dhcp/dhcpmgr
The management of the server can be done in a couple of ways: properly via command-line tools including /usr/sbin/dhcpconfig
, /usr/sbin/pntadm
and /usr/sbin/dhtadm
, or with the GUI applet /usr/sadm/admin/bin/dhcpmgr
(also symlinked as /usr/sbin/dhcpmgr
), and somewhat improperly by manipulating the config files (/etc/inet/dhcpsvc.conf
) and DHCP text-formatted database files /var/dhcp/SUNW*
directly – this is not encouraged, but is often faster, more straightforward, and "just plain works"TM.
First of all, to enable the DHCP server, its configuration file should exist and say that the server is enabled (for more details see init-script /etc/init.d/dhcp
which wraps the SMF service).
:; dhcpconfig -r SUNWfiles -p /var/dhcp -D -d `domainname` -a 8.8.8.8 -l 86400 Created DHCP configuration file. Created dhcptab. Added "Locale" macro to dhcptab. Added server macro to dhcptab - openindiana. DHCP server started. :; cat /etc/inet/dhcpsvc.conf DAEMON_ENABLED=TRUE RESOURCE=SUNWfiles RUN_MODE=server PATH=/var/dhcp CONVER=1
In the example above, the -d
option sets the local domain name, and the -a
option sets the DNS servers to be used (that would likely be your LAN's DNS server address, or 192.168.127.1
if you set up a DNS server instance on this box). We've also requested to use the less performant but more convenient text-formatted database files "resource".
The wizard also took some freedom to add a "server macro" based on `hostname`
and IP addressing of this zone, which may be acceptable or invalid for our uses – we'll see.
:; echo "INTERFACES=vnic127001" >> /etc/inet/dhcpsvc.conf :; /etc/init.d/dhcp stop; /etc/init.d/dhcp start :; svcs -p dhcp-server STATE STIME FMRI online 17:44:51 svc:/network/dhcp-server:default 17:44:51 26974 in.dhcpd :; netstat -an | grep -w 67 255.255.255.255.67 Idle 192.168.127.0.67 Idle 192.168.127.1.67 Idle
You would likely want the DHCP server to log its events, errors and other messages into syslog
to ease the debugging.
Enable logging on the DHCP server:
:; echo "LOGGING_FACILITY=0" >> /etc/inet/dhcpsvc.conf :; svcadm restart dhcp-server
Note: You can use any local syslog
facility ranging from 0 to 7.
Add this line to /etc/syslog.conf
to enable saving of these messages into a particular file (NOTE: the two parts must be separated by TAB characters):
local0.notice /var/log/dhcpsvc
Note that other local0
events would get into this log file, like ipmon
does on my system. Well, while debugging THIS setup, it is only a bonus to have the two logs co-located
Now you can touch the file, and restart the syslog
:
:; touch /var/log/dhcpsvc :; svcadm restart system-log
Note that syslog
does not create log files by itself, and complains if one is not present at the moment of the daemon's startup or restart.
Ultimately, enable log rotation to restrain the disk space requirements:
:; cat << EOF >> /etc/logadm.conf ### Rotate DHCP/ipmon logs /var/log/dhcpsvc -C 4 -s 1m -a '/usr/sbin/svcadm refresh system-log' EOF
This uses the currently default log rotation engine logadm
regularly called from cron
; if you use something else (newsyslog
, logrotate.d
, etc.) – configure that engine appropriately.
Here are some sample syslog
ged entries about DHCP events:
Jun 6 23:48:26 openindiana in.dhcpd[6041]: [ID 771465 local0.notice] DHCP ASSIGN 1339012106 -000000001 192.168.127.101 192.168.1.40 001271270101 N/A 001271270101 Jun 7 14:24:05 openindiana in.dhcpd[6041]: [ID 771465 local0.notice] DHCP EXTEND 1339064645 -000000001 192.168.127.101 192.168.1.40 001271270101 N/A 001271270101
Now that we have a server running, we can define a subnet macro:
:; dhcpconfig -N 192.168.127.0 -m 255.255.255.0 -t 192.168.127.1 Added network macro to dhcptab - 192.168.127.0. Created network table.
The macros are used to basically group some settings for a client, and can be nested, as we'll do below.
Under The Hood
Let's see under the hood – what have we defined in the previous step?
:; ls -la /var/dhcp/SUNW* -rw-r--r-- 1 root root 104 Jun 6 17:51 /var/dhcp/SUNWfiles1_192_168_127_0 -rw-r--r-- 1 root root 385 Jun 6 17:51 /var/dhcp/SUNWfiles1_dhcptab :; cat /var/dhcp/SUNWfiles1_dhcptab # SUNWfiles1_dhcptab # # Do NOT edit this file by hand -- use dhtadm(1M) or dhcpmgr(1M) instead # Locale|m|4785356079057862657|:UTCoffst=10800: openindiana|m|10017694421132247041|:Include=Locale:Timeserv=192.168.1.40:LeaseTim=86400:LeaseNeg:DNSdmain="domain.com":DNSserv=8.8.8.8: 192.168.127.0|m|2030560481990672385|:Subnet=255.255.255.0:Router=192.168.127.1:Broadcst=192.168.127.255:
The first two macros were predefined by initial dhcpconfig
based on what it knows about the naming and networking setup of the host and on our command-line options, while the last one was added by us explicitly.
The /var/dhcp/SUNWfiles1_192_168_127_0
file is currently empty (comments only), but it will soon contain the definitions which bind IP addresses, MAC addresses and DHCP Macros for particular clients.
Now, combine (nest) the macros into one DHCP profile which can be used to configure the clients:
:; dhtadm -A -m localstub127 -d ":Include=192.168.127.0:Include=`hostname`:" :; tail -1 /var/dhcp/SUNWfiles1_dhcptab localstub127|m|593912200859484161|:Include=192.168.127.0:Include=openindiana:
NOTE: This implies the Timeserv
value to be the GZ public address, but since local zones don't have individual clocks – we don't care. This can be a problem however if the DHCP clients include different machines which have clocks and can't reach the router's external address, or are zones which host VMs; in this case the `hostname`
-based macro (the openindiana
above) should be redefined, or another similar one can be defined and Include
d by our localstub127
macro instead. In the latter case I'd make several macros – for time servers, for DNS servers, etc. and combine them all as suitable for a particular client profile.
More information about Sun DHCP macros can be found here:
Some client addresses can be predefined, following our IP/MAC-address naming pattern explained above. This simple construct would create some 9 entries for the VNICs made in examples above:
:; OWNERIP=$(getent hosts `hostname` | awk '{print $1}') :; for IP in 01 02 03 04 05 06 07 08 09; do \ pntadm -A 192.168.127.1${IP} -i 010012712701${IP} \ -s ${OWNERIP} -m localstub127 -f 01 \ -c "Build zone on localstub127" 192.168.127.0; done
Here we define the following pntadm
options:
-A
– the client IP address (or hostname known via /etc/hosts
, technically);-i
– the client identifier (MAC address prefixed by the ARP code for the type of network, usually 01
for Ethernet);-s
– DHCP server responsible for the leases;-m
– the client macro for common settings;-f
– the flag describing the lease type (01
= PERMANENT lease; announced via DHCP and never expires to be recycled for other clients);-c
– an optional comment describing this client;IMPORTANT NOTE about the OWNERIP construct used above and below: The Sun DHCP server is identified by the numeric IP address (i.e. 192.168.1.40
from the external network, for our example) which corresponds to the textual hostname (i.e. openindiana
), which in turn corresponds to this host's name set in /etc/nodename
. That is, even though you serve addresses on a 192.168.127.0/24
subnet, the "owner" of these leases must be 192.168.1.40
if that address corresponds to the nodename as resolvable via /etc/hosts
(or possibly DNS too). Otherwise this instance of the DHCP server will refuse to issue addresses "owned" by another server (even if the provided value resolves to this same machine).
Also keep this in mind if you ever rename the GZ hostname
and/or change its "primary" IP address.
You can also use these commands (likely with different options) to reserve some IP addresses for dynamic leasing for not-predefined hosts, i.e. to quickly provision VMs or test zones. You might use a different macro (i.e. with shorter LeaseTim
) in this case, as well.
For example, a range of 192.168.127.201-209
will be defined here, leasing addresses for 30 minutes to any client (except those who already have a reservation for the same MAC address):
:; OWNERIP=$(getent hosts `hostname` | awk '{print $1}') :; dhtadm -A -m localstub127rnd -d \ ':Include=192.168.127.0:Include=Locale:LeaseTim=1800:LeaseNeg:DNSdmain="'"`domainname`"'":DNSserv=8.8.8.8:' :; for IP in 01 02 03 04 05 06 07 08 09; do \ pntadm -A 192.168.127.2${IP} -i 00 \ -s ${OWNERIP} -m localstub127rnd -f 00 \ -c "Quick-test hosts on localstub127" 192.168.127.0; done
You might want to use a local mirror of the OpenIndiana package repository hosted in the global zone, as detailed in Making a LAN mirror of global package repositories.
However, after completing the procedure described on that page, you might need to spawn another instance of the package depot service to listen on the more-specific IP address (than the default 0.0.0.0
) and do HTTP-redirects to it, with these commands:
:; svccfg -s pkg/server add dev127 select dev127 addpg pkg application setprop pkg/inst_root = astring: "/export/pkg/dev/" setprop pkg/port = count: 10002 setprop pkg/address = net_address: 192.168.127.1 setprop pkg/proxy_base = astring: http://192.168.127.1:10002 exit :; svccfg -s pkg/server select dev addpg dev dependency setprop dev/entities = fmri: svc:/application/pkg/server:dev127 setprop dev/grouping = astring: optional_all setprop dev/restart_on = astring: restart setprop dev/type = astring: service exit :; svcadm refresh pkg/server:dev127 pkg/server:dev :; svcadm disable -st pkg/server:dev127 pkg/server:dev :; svcadm enable pkg/server:dev127 pkg/server:dev
NOTE that the default listener dev
created in that procedure would now depend on this new service instance dev127
, otherwise dev127
can fail to start due to its own internal checks (technically: pkg.depotd
checks if it can connect to the specified address:port
first, and the default listener of dev
is considered a busy port by dev127
). Any restarts of dev127
should cause shutdown and restart of dev
as well, for the same reasons.
These steps allow networking clients in the LZ to contact the outside world and allow networking servers in the LZ to be accessible from the outside world.
You can set up "forwarding" and "routing" in Solaris with routeadm
. Forwarding is the process of relaying packets not originated by nor destined to this host's IP addresses, from one interface to another. Routing encompasses special daemons which manage the kernel's routing table dynamically and/or announce this box as a router. For our purposes, we only need forwarding enabled (and we don't want to announce our host-only subnet to the external LAN).
Enable IPv4 packet forwarding and update the running configuration:
:; routeadm -e ipv4-forwarding :; routeadm -u
Here we can see that the interfaces got a ROUTER
flag enabled:
:; ifconfig -a lo0: flags=2001000849<UP,LOOPBACK,RUNNING,MULTICAST,IPv4,VIRTUAL> mtu 8232 index 1 inet 127.0.0.1 netmask ff000000 e1000g0: flags=1100843<UP,BROADCAST,RUNNING,MULTICAST,ROUTER,IPv4> mtu 1500 index 2 inet 192.168.1.40 netmask ffffff00 broadcast 192.168.1.255 ether 0:15:17:5b:d9:64 vnic127001: flags=1100843<UP,BROADCAST,RUNNING,MULTICAST,ROUTER,IPv4> mtu 9000 index 3 inet 192.168.127.1 netmask ffffff00 broadcast 192.168.127.255 ether 2:8:20:87:e4:43 lo0: flags=2002000849<UP,LOOPBACK,RUNNING,MULTICAST,IPv6,VIRTUAL> mtu 8252 index 1 inet6 ::1/128
To enable NAT we need to set up translation rules in /etc/ipf/ipnat.conf
and allow connections on the packet filter in /etc/ipf/ipf.conf
. Rough-cut examples follow.
Also note that local zones with ip-type=exclusive
can define and use their own firewall settings, although it won't likely be required for the setup we're making here.
### ipnat.conf for zone-builder host ### Interfaces: ### e1000g0 = physical net (192.168.1.40) ### vnic127001 = build zones in 192.168.127.0/24 ### Allow incoming access to local zones (incoming on e1000g0) rdr e1000g0 0.0.0.0/0 port 22101 -> 192.168.127.101 port 22 tcp ### Allow local zones to access external network (outgoing packets from e1000g0) map e1000g0 192.168.127.0/24 -> 0/32 proxy port ftp ftp/tcp age 600 map e1000g0 192.168.127.0/24 -> 0/32 portmap tcp/udp auto age 600 map e1000g0 192.168.127.0/24 -> 0/32 age 600
0/32
in the map
rules above and 0/0
in rdr
rules should automagically substitute the interface's active address(es); this is a mode originally used for NATing by hosts with dial-up or otherwise dynamic external addresses.# # ipf.conf for zone-builder host # # IP Filter rules to be loaded during startup # # See ipf(4) manpage for more information on # IP Filter rules syntax. ### Interfaces: ### e1000g0 = physical net ### vnic127001 = build zones in 192.168.127.0/24 ### Allow all on internal interfaces pass in quick on lo0 all keep state keep frags pass out quick on lo0 all keep state keep frags pass in quick on vnic127001 all keep state keep frags pass out quick on vnic127001 all keep state keep frags pass in quick on vnic127001 pass out quick on vnic127001 ### Allow PING pass out quick proto icmp from any to any icmp-type 8 code 0 pass in quick proto icmp from any to any icmp-type 0 code 0 pass in quick proto icmp from any to any icmp-type 8 code 0 pass out quick proto icmp from any to any icmp-type 0 code 0 ### Allow TRACEROUTE pass out quick proto icmp from any to any icmp-type 11 code 0 pass in quick proto icmp from any to any icmp-type 3 code 3 pass in quick proto icmp from any to any icmp-type 11 code 0 pass out quick proto icmp from any to any icmp-type 3 code 3 ### Default rule block in log all ### Start new rule groups for the physical interface, ### see man for the special syntax block out log quick on e1000g0 all head 100 block in log quick on e1000g0 all head 101 ##################################################################### ### Specific rules can follow here, for now we just allow anything ### NOTE: rules for NATed connections use internal/private addresses ##################################################################### ### Rules for outgoing packets pass out quick proto tcp from any to any flags S keep state keep frags group 100 pass out quick proto udp from any to any keep state group 100 pass out quick from any to any group 100 ### Rules for incoming packets pass in quick proto tcp from any to any flags S keep state keep frags group 101 pass in quick proto udp from any to any keep state group 101 pass in quick from any to any group 101
group
" rules at the end of the file. The default action of these groups is to log
the packets (so you can trace them with ipmon
) and block
them, denying the communication. However, the defined actions above simply allow all packets to enter and exit the external interface. If you further tune the firewall, these rules should be commented out and replaced by specific permissions – and don't forget to allow your SSH Have your console access ready, just in case you forbid yourself the pleasures of networking with the host!
Alternately, you can use screen
or VNC to set a timer which would "(sleep 20; svcadm disable ipfilter)&
" – just in case...
First of all, reconfigure the ipfilter
SMF service to use traditional configuration methods (with the manually created files instead of rule snippets added by some services):
:; svccfg -s svc:/network/ipfilter:default setprop firewall_config_default/policy=custom :; svccfg -s svc:/network/ipfilter:default setprop firewall_config_default/custom_policy_file=/etc/ipf/ipf.conf :; svcadm refresh ipfilter
Apply the settings – but first see if ipfilter
service is running and enable or restart it accordingly:
### If ipfilter was disabled -- start it: :; svcs ipfilter STATE STIME FMRI disabled May_03 svc:/network/ipfilter:default :; svcadm enable ipfilter ### If ipfilter was enabled -- restart it: :; svcs ipfilter STATE STIME FMRI online May_16 svc:/network/ipfilter:default :; svcadm restart ipfilter
Test if the settings got applied:
:; ipnat -l List of active MAP/Redirect filters: rdr e1000g0 0.0.0.0/0 port 22101 -> 192.168.127.101 port 22 tcp map e1000g0 192.168.127.0/24 -> 0.0.0.0/32 proxy port ftp ftp/tcp age 600/600 map e1000g0 192.168.127.0/24 -> 0.0.0.0/32 portmap tcp/udp auto age 600/600 map e1000g0 192.168.127.0/24 -> 0.0.0.0/32 age 600/600 List of active sessions: :; ipfstat -hion 0 @1 pass out quick on lo0 all keep state keep frags 5 @2 pass out quick on vnic127001 all keep state keep frags 0 @3 pass out quick on vnic127001 all 0 @4 pass out quick proto icmp from any to any icmp-type echo code 0 1059 @5 pass out quick proto icmp from any to any icmp-type echorep code 0 0 @6 pass out quick proto icmp from any to any icmp-type timex code 0 0 @7 pass out quick proto icmp from any to any icmp-type unreach code 3 57 @8 block out log quick on e1000g0 all head 100 0 @1 pass out quick proto tcp from any to any flags S/FSRPAU keep state keep frags group 100 7 @2 pass out quick proto udp from any to any keep state group 100 127 @3 pass out quick from any to any group 100 0 @1 pass in quick on lo0 all keep state keep frags 4 @2 pass in quick on vnic127001 all keep state keep frags 0 @3 pass in quick on vnic127001 all 0 @4 pass in quick proto icmp from any to any icmp-type echorep code 0 1059 @5 pass in quick proto icmp from any to any icmp-type echo code 0 0 @6 pass in quick proto icmp from any to any icmp-type unreach code 3 0 @7 pass in quick proto icmp from any to any icmp-type timex code 0 205 @8 block in log all 205 @9 block in log quick on e1000g0 all head 101 0 @1 pass in quick proto tcp from any to any flags S/FSRPAU keep state keep frags group 101 79 @2 pass in quick proto udp from any to any keep state group 101 177 @3 pass in quick from any to any group 101
ipfstat -hion
output can differ on your machine, it is the rule-hit count.If you initially did just a host-only network and stopped at that, you might not have configured a router announcement in the macros, or the setting was/became invalid.
Make sure that this host (192.168.127.1
) is now announced as the router for the host-only network clients, per DHCP macro examples above.
The local zone setup is explained in more detail in other documents, such as Building in zones.
For the purposes of this document, we only overview the creation of the zone and the tests that it has network access both ways.
You might have set up some VNICs in the examples above. If you ran out of these addresses, follow the procedures above to add a VNIC and a DHCP reservation for its address.
In short:
:; dladm create-vnic -l localstub127 -m 00:12:71:27:02:34 vnic127234 :; OWNERIP=$(getent hosts `hostname` | awk '{print $1}') :; pntadm -A 192.168.127.234 -i 01001271270234 \ -s ${OWNERIP} -m localstub127 -f 01 \ -c "Build zone on localstub127" 192.168.127.0
Update /etc/ipf/ipnat.conf
rules if you need to publish something (i.e. SSH access) as you see fit, and svcadm restart ipfilter
in this case.
:; zonecfg -z zone1 create -b set zonepath=/zones/build/zone1 set brand=ipkg set autoboot=true set ip-type=exclusive add net set physical=vnic127101 end verify commit exit
NOTE: Be certain to use the VNIC name, zone name and zone path which you want and not the ones you've copy-pasted from this example
Beside installing the zone, we will use a /etc/sysidcfg
file to predefine the local zone's environmental configuration, so that the zone (and its possible clones) gets configured automatically.
Install the local zone:
:; zoneadm -z zone1 install ... Next Steps: Boot the zone, then log into the zone console (zlogin -C) to complete the configuration process. :; zfs snapshot -r rpool/zones/build/zone1@1
NOTE about using a local package mirror
If you are Making a LAN mirror of global package repositories on this machine's global zone, you might want to use that for LZ installation and subsequent upgrades, and bypass an HTTP proxy server that you might have configured in your environment. In that case you'd use the command like this for zone installation, instead of the one given above:
:; http_proxy= https_proxy= zoneadm -z zone1 install -P openindiana.org=http://192.168.127.1:10002/ :; zfs snapshot -r rpool/zones/build/zone1@1
... or likewise for the repo hosted on another box in your LAN – using the LAN's IP address (192.168.1.40
in our example). The remainder of the procedures below remains as described.
Prepare the /etc/sysidcfg
file with environmental presets for the zone (for hands-free boots), boot it and monitor how it goes:
:; zoneadm -z zone1 ready :; cat <<EOF > /zones/build/zone1/root/etc/sysidcfg terminal=vt100 network_interface=PRIMARY {dhcp protocol_ipv6=no} security_policy=none nfs4_domain=dynamic timezone=Europe/Moscow root_password=fto/dU8MKwQRI name_service=NONE EOF ### the encrypted root password shown here is: abc123 :; zoneadm -z zone1 halt :; zfs snapshot -r rpool/zones/build/zone1@2 :; zoneadm -z zone1 boot :; zlogin -C zone1
Unfortunately, I did not find a way for sysidcfg
to pick up DNS settings from DHCP, so they are to be either pushed explicitly with name_service=DNS...
configuration with a block like this:
name_service=DNS { domain_name=myhome.com name_server=192.168.1.1,8.8.8.8 search=myhome.com,mywork.com}
... or with name_service=NONE
and a hack described below in order to allow local zones to use the DNS settings from DHCP properties.
It seems that the ability for a local zone to get DNS settings from DHCP is crippled explicitly in /lib/svc/share/smf_include.sh
routine smf_netstrategy
:
... # The network boot strategy for a zone is always "none". # smf_netstrategy () { if smf_is_nonglobalzone; then _INIT_NET_STRATEGY="none" export _INIT_NET_STRATEGY return 0 fi ...
Commenting away the block for smf_is_nonglobalzone
(in the local zone's copy of the script) and doing svcadm restart svc://network/service:default
does the trick for auto-generation of /etc/resolv.conf
and fixup of /etc/nsswitch.conf
, but further impact is unknown, so this is for now a "use at your own risk" kind of workaround. Comments requested on mailing list.
More information about /etc/sysidcfg
options can be found here:
Note that with the /etc/sysidcfg
file in place, the local zone proceeds to auto-configuration and reboot (which may take a minute of it seemingly hanging).
If you get errors like: "Could not contact a dhcp server on the network interface vnic127101
" or "in.rdisc: No interfaces up
", possibly after several minutes of waiting, double-check the DHCP server and firewall settings. In particular, check that the "owner" of the addresses is set correctly in the dhcptab (see details above).
If the local zone does not receive an IP address, you can try to see if it even tries:
:; snoop -r -d vnic127001 Using device vnic127001 (promiscuous mode) OLD-BROADCAST -> BROADCAST DHCP/BOOTP DHCPREQUEST OLD-BROADCAST -> (broadcast) ARP C Who is 192.168.127.101, 192.168.127.101 ? 192.168.127.101 -> (broadcast) ARP C Who is 192.168.127.101, 192.168.127.101 ? OLD-BROADCAST -> (broadcast) RARP C Who is 0:12:71:27:1:1 ? 192.168.127.101 -> 192.168.127.255 BPARAM C WHOAMI? 192.168.127.101 192.168.127.101 -> 192.168.127.255 BPARAM C WHOAMI? 192.168.127.101 (retransmit)
There may be several repetitions of these lines in varied order. If the server does not give it the address, there would only be repetitions of the DHCPREQUEST
line every few seconds.
You may also want to check the services and configuration from inside the local zone.
While the zone has not yet been configured, direct attempts to zlogin
into it can fail, because the root
user is not yet activated:
:; zlogin zone1 [Connected to zone 'zone1' pts/3] Login incorrect [Connection to zone 'zone1' pts/3 closed]
In this case, use an administrative workaround:
:; zlogin -S zone1 [Connected to zone 'zone1' pts/3] @zone1:~$ bash bash-4.0#
Now you can see what the zone has configured for itself by using ifconfig -a
, what it is running with ps -ef
, the state of services with svcs
and so on.
Now that your local zone seems to be configured and running, zlogin
into it and see what networking settings it has grabbed:
:; ifconfig -a lo0: flags=2001000849<UP,LOOPBACK,RUNNING,MULTICAST,IPv4,VIRTUAL> mtu 8232 index 1 inet 127.0.0.1 netmask ff000000 vnic127101: flags=1004843<UP,BROADCAST,RUNNING,MULTICAST,DHCP,IPv4> mtu 9000 index 2 inet 192.168.127.101 netmask ffffff00 broadcast 192.168.127.255 ether 0:12:71:27:1:1 lo0: flags=2002000849<UP,LOOPBACK,RUNNING,MULTICAST,IPv6,VIRTUAL> mtu 8252 index 1 inet6 ::1/128 :; netstat -rn Routing Table: IPv4 Destination Gateway Flags Ref Use Interface -------------------- -------------------- ----- ----- ---------- --------- default 192.168.127.1 UG 1 0 vnic127101 127.0.0.1 127.0.0.1 UH 6 56 lo0 192.168.127.0 192.168.127.101 U 2 0 vnic127101 Routing Table: IPv6 Destination/Mask Gateway Flags Ref Use If --------------------------- --------------------------- ----- --- ------- ----- ::1 ::1 UH 2 0 lo0 :; traceroute -nI www.ru traceroute: unknown host www.ru ### Oops... here DNS settings were not defined explicitly nor via DHCP with the workaround above. ### If DNS config is applied in any manner, name resolution and internet should work: :; ping -ns pkg.openindiana.org PING pkg.openindiana.org (91.194.74.133): 56 data bytes 64 bytes from 91.194.74.133: icmp_seq=0. time=53.891 ms 64 bytes from 91.194.74.133: icmp_seq=1. time=52.602 ms 64 bytes from 91.194.74.133: icmp_seq=2. time=53.448 ms ^C ----pkg.openindiana.org PING Statistics---- 3 packets transmitted, 3 packets received, 0% packet loss round-trip (ms) min/avg/max/stddev = 52.602/53.314/53.891/0.655 :; traceroute -nI pkg.openindiana.org traceroute to pkg.openindiana.org (91.194.74.133), 30 hops max, 40 byte packets 1 192.168.127.1 0.342 ms 0.086 ms 0.071 ms 2 192.168.1.1 11.741 ms 2.199 ms 1.384 ms ... 12 87.245.245.10 61.736 ms 95.524 ms 96.185 ms 13 195.72.129.157 50.344 ms 54.766 ms 49.147 ms 14 91.194.74.133 53.484 ms 52.423 ms 51.936 ms
Rough outline of the routine, detailed steps are in the chapters above:
ipnat.conf
in the GZ;ipfilter
settings;Test accessibility of the service from your external LAN host (different from the routing GZ):
### Testing from another host :; (echo ""; sleep 1) | telnet 192.168.1.40 22101 Trying 192.168.1.40... Connected to 192.168.1.40. Escape character is '^]'. SSH-2.0-Sun_SSH_1.5 Protocol mismatch. Connection to 192.168.1.40 closed by foreign host.
If you rename the VNICs dedicated to a zone, such as when you clone a working zone and give it a new identity (in the example below I'll be changing 101 to 109), keep in mind the following configuration files:
/etc/hostname.vnic*
– empty file, its existence notifies the OS that the named interface should get "plumbed" at startup.
:; mv /etc/hostname.vnic127101 /etc/hostname.vnic127109
If this file is not empty, it can contain additional parameters for ifconfig
, such as static addressing or MTU size – in case of DHCP we likely want it empty.
/etc/dhcp.vnic*
– empty file, notifies the OS that the named interface should get its settings via DHCP.
:; mv /etc/dhcp.vnic127101 /etc/dhcp.vnic127109
/etc/hosts
– matches IP addresses to textual names in absence of DNS or a different naming server; one of the lines should be the current host's name and address. The entry should be changed, i.e.
-192.168.127.101 zone1 +192.168.127.109 build-gcc444
Without this things that reference the current host can fail, such as sendmail
and some Solaris_audit
tasks.
/etc/nodename
– if you've also changed the local zone's name (i.e. while cloning it), you want to change its perception of its name.
:; zonename > /etc/nodename
SSH keys – to be pedantic, you might want each cloned zone to use different SSH server keys, though this is not technically required.
:; rm -f /etc/ssh/ssh_host_*_key* && \ svcadm restart ssh
Now you can restart (init 6
) the cloned/renamed zone to apply the new settings. Simple svcadm restart p
hysical:default
can also suffice for networking setup.