The goal of this project is to build enterprise-grade iSCSI storage that is modular enough to meet any iSCSI needs.

I chose OpenSolaris for the flexibility we get from ZFS, which everyone has at least heard of, but also for its Common SCSI Target (COMSTAR) project.   I’ll only be discussing the iSCSI target portion of this project, but I recommend reading more on the capabilities of COMSTAR outside of the iSCSI space.

HARDWARE

Since we talking about building our own storage array, lets look at some hardware options. My personal preference for a chassis is the Supermicro SC846 for the redundant power, an option for two internal disks so that you can use all 24 hot-swap for just storage, and the ability to use 3.5″ and 2.5″ drives.

If you prefer only go for 2.5″ drives you might want to check out SC216 which will also provide 24 disks, but in only 2 Rack Units of space.

The next important decision is the HBA(s) you will be using. Keep in mind that since we’ll be using ZFS for this project, we do not want a RAID card, but instead a JBOD card. Trust me on this on, RAID cards can turn out to be a nightmare, plus JBOD HBA’s are cheaper. My personal preference in HBA for OpenSolaris is the LSI 3081 card for the Fusion-MPT chip. You don’t have to buy LSI brand, but I definitely recommend an HBA with Fusion-MPT. Since these provide 8 SATA ports, you’ll need 3 of them to support 24 disks.

The last major hardware decision is networking. My preference for 1GB is a NIC with the 82571EB chip. Intel offers a single port card, the Intel PRO/1000 MT, and a dual-port version, the Intel PRO/1000 PT.

For 10GB I recommend a NIC with the 82598EB chip. For a dual-port CX-4 version I use the EXPX9502CX4 card or if you prefer dual port SR fiber go with the EXPX9502AFXSR card.

Just to be clear, all of the hardware recommendation I have made I currently use with OpenSolaris servers and are confirmed by Sun to be supported in OpenSolaris.

CONFIGURATION

Let’s start with a fresh install of OpenSolaris.

Mirror OS disk

First, lets mirror our OS disk for added reliability.  In this example, OpenSolaris was install on disk c9d0s0.  Our second OS disk is c10d0s0.

1. Create a solaris disk label on the second disk

# format c10d0s0

Select “fdisk” then “create 100% Standard Solaris Partition over the full Disk”

2. Next, we need to copy the Solaris slice layout from the OS disk to the second disk. (note we use s2, this is very important)

# prtvtoc /dev/rdsk/c9d0s2 | fmthard -s - /dev/rdsk/c10d0s2

3. Next, we’ll attach the mirror disk to the OS zpool

#  zpool attach -f rpool c9d0s0 c10d0s0

4. Last, we need to make the second disk bootable

# installgrub -m /boot/grub/stage1 /boot/grub/stage2 /dev/rdsk/c10d0s0

Static IP
If you don’t want to rely on always getting the same DHCP IP, you’ll probably want to statically configure the IP of your storage server.

First, we need to disable the NetworkAutomagic service

# svcadm disable network/physical:nwam

Next, enable the config file-based networking service

# svcadm enable network/physical:default

Now we must configure the IP statically. This is done by creating a /etc/hostname. file. In this example I’ll use the e1000g0 interface.

 vi /etc/hostname.e1000g0
192.168.1.200

Configure the netmask for the management IP

# vi /etc/netmasks
192.168.1.0 255.255.255.0

Configure the default gateway

# vi /etc/defaultrouter
192.168.1.1

Tell system to use standard file-based DNS

# cp /etc/nsswitch.dns /etc/nsswitch.conf

Now, Configure DNS servers

# vi /etc/resolv.conf
nameserver 192.168.1.4

Configure IP Multi-Pathing (IPMP)
If you went with a dual-port card, or two cards, it’s advisable to use IPMP so that a single link down doesn’t make your iSCSI volumes unaccessible.

In this example I’m using two e1000g interfaces and creating the IPMP interface iscsi0.

# vi /etc/hostname.iscsi0
ipmp group san0 192.168.1.200 up

The primary interface of the IPMP group is e1000g0

# vi /etc/hostname.e1000g0
group san0 -failover up

The backup interface is e1000g1

# vi /etc/hostname.e1000g1
group san0 -failover standby up

Enable COMSTAR
Install stmf (library and service for COMSTAR)

# pkg install SUNWstmf

Now install the iSCSI toolset

# pkg install SUNWiscsit

At this point, reboot your machine before continuing on.

After rebooting, we will enable the stmf service

# svcadm enable stmf

Creating your zpool
I went with a chassis that supports up to 24 disks to build in room for expansion. Based on you needs, you can fill all 24 hot-swap trays with raw storage to be exported as one or more iSCSI volumes, or you can use some to take advantage of some of the performance advantages of creating a hybrid pool.

If you are unfamiliar with the term hybrid pool, I suggest reading up on ZIL and L2ARC. Here are a few links to get you started:
ZIL: SLOG BLOG
L2ARC: ZFS L2ARC

So for purposes of this example, I’ll presume to save four drive bays for SSDs, a pair for ZIL and a pair for L2ARC, leaving 20 disks. We can then use these 20 drive slots for 4 RAIDZ of 5 disks. I’m going to start with configuring one, then I’ll explain how to grow your ZPOOL when adding your second RAIDZ for storage expansion.

Before we go any further, now is a good time to demonstrate two useful commands. The first, we can use devfsadm to scan for newly added disks.

# devfsadm -Cv

Second, we can use the format command to list all recognized disks.

# format < /dev/null

For the first 5 storage disks, mine were recognized on channel 7. I'll create my initial zpool named "iSCSIdisks" as a RAIDZ using all 5 disks.

# zpool create iSCSIdisks raidz c7t0d0 c7t1d0 c7t2d0 c7t3d0 c7t4d0

There we go, we now have our storage to start creating iSCSI volumes. I'm going to now create a 20GB zvol (target volume) that will be used as the disk for a virtual machine.

# zfs create -V 20G iSCSIdisks/vm1_hdd

Next, I need to make a LUN (Logical Unit) out of this volume.

# sbdadm create-lu /dev/zvol/rdsk/iSCSIdisks/vm1_hdd

Now that we have create a logical unit, we need to find out the GUID of this volume so that we can provide it to COMSTAR for iSCSI access. Here's how you list all LUNs that have been created.

# sbdadm list-lu

Now, if you don't already have the iscsit server enabled, now would be a good time to do so.

# svcadm enable -r svc:/network/iscsi/target:default

I'm going to create a basic iSCSI target configuration here that leaves this storage wide-open to be accessed by anyone, I suggest you secure yours. To do so, read up on itadm in the man page.

# itadm create-target

You can now see your newly created iSCSI target, and all previously created ones, using the itadm command.

# itadm list-target

You're all set to access this storage remotely.

The last thing I want to come back to is how we will grow our underlying storage as we need to expand. Following the previous example of a 5 disk RAIDZ, I'll just add a second 5 disk RAIDZ to the zpool iSCSIdisks.

Since I have 3 LSI HBAs, each with 8 ports, my next 5 disks will consume the last 3 ports of my first HBA and the first 2 ports of my second one. I plug in the 5 new disks, run "devfsadm -Cv" then run "format < /dev/null" to ensure they have been recognized. Now I'm ready to add them.

# zpool add iSCSIdisks raidz c7t5d0 c7t6d0 c7t7d0 c8t0d0 c8t1d0

And that’s it, your zpool is now grown and ready to be sliced up into more iSCSI targets.

Enjoy your new enterprise iSCSI array, and don’t for get to check out ZIL and L2ARC!

WebVirt has become a very exciting project, so I’ve been spending almost all of my computing time coding. I have posted some screenshots but I have to admit that they’re already a bit dated with some of the new features I’ve implemented.

Current WebVirt can connect to remote libvirt nodes, however only with zero authentication. What this mean is that you’re limited to using a connection string like this:

qemu+tcp://192.168.1.2/?name=qemu:///session

That being said, I’m planning on using the Redhat package python-nss for key creation and management. So that should follow “hopefully” shortly.

Once connected, you can start and destroy (stop) both virtual networks and domains that are currently defined on a remote physical node. I should point out that there is still a bug in libvirt that “may” undefine a network on the remote machine when you destroy it. Undefining is removing the config from the remote node. This does not effect domains, however.

You can also push network and domain configurations created in WebVirt to your remote nodes.

You can toggle whether or not each virtual domain and network is to autostart. This means that when the libvirt daemon is started/restarted on the remote node, these virtuals domains and networks will automatically start.

Lastly, importing previously defined virtual domains and networks from remote nodes.  The elaborate on this feature, let’s say you’re like me. You’re so excited about libvirt, that you’re already using it to manage virtual domains and networks, and you just can’t wait for a full release of WebVirt.

Fear not!  Currently WebVirt can only import the very basic virtual networks and the import of virtuals domains is all but a place holder (only the name and UUID are read as a proof of concept as up CVS check-in 14), this feature is of utmost importance to the project and will be fully implemented in time.

Since the architecture of the project has been laid out. I determined it was time to allow a little more attention than my website can provide.

Since I’ve been developing it on Fedora and intend on using it with Fedora, RHEL, and CentOS, I went with FedoraHosted.

Go check out the project Trac page: fedorahosted.org/webvirt

I started playing with creating xml configs for creating new domains, networks and storage. I had been planning on 1. learning pythong and 2. playing with django, so this jointly inspired me to start WebVirt, a web-based front-end to libvirt for managing virtual machines.

I spent this weekend completing my first two goals, coding basic xml generation for libvirt resources, GPLing and creating the repo.

The Multi Router Traffic Grapher (MRTG)

Reference: Tobi Oetiker’s MRTG – The Multi Router Traffic Grapher

To borrow a phrase from Tobi Oetiker, “You have a router, you want to know what it does all day long? Then MRTG is for you.” The goal here is to track the actions of the OpenBSD router over time. This practice is important for detecting trends in traffic, helpful for finding bottlenecks, and even identify a baseline to recognize abnormal changes in traffic.

So let’s get on with it. For this example, I use OpenBSD’s MRTG package. I’ll also install two packages required for IPv6, OpenBSD MRTG package requires them:

mschenck ~# sudo pkg_add http://mirror.rit.edu/pub/OpenBSD/4.3/packages/i386/mrtg-2.15.2p1.tgz
mschenck ~# sudo pkg_add http://mirror.rit.edu/pub/OpenBSD/4.3/packages/i386/p5-Socket6-0.19.tgz
mschenck ~# sudo pkg_add http://mirror.rit.edu/pub/OpenBSD/4.3/packages/i386/p5-IO-INET6-2.01p0.tgz

I should point out that you need an snmp daemon running for MRTG to pole stats from your router. The enable this first you must add the following to “/etc/rc.conf.local”:

snmpd_flags=""          # for normal use: ""

This will enable snmpd to automatically start on reboot. In the meantime, lets start it ourself

mschenck ~#  sudo  /usr/sbin/snmpd

By default, OpenBSD’s snmp daemon (snmpd(8)) only listens on localhost and the default community string is “public“.  You can change these settings by modifying ” /etc/snmpd.conf” (see snmpd.conf(5)), however for this example we’ll stick with these default settings.

Now lets get an http daemon running to display these graphs.  OpenBSD come with Apache, lets enable and start it up. Add the following line to /etc/rc.conf.local:

httpd_flags=""          # for normal use: "" (or "-DSSL" after reading ssl(8))

Again, lets avoid the reboot and just start the daemon manually:

mschenck ~# sudo /usr/sbin/httpd

Now, lets backup to original document root for apache and create a new one for displaying our mrtg graphs:

mschenck ~# sudo mv /var/www/htdocs /var/www/htdocs-orig
mschenck ~# sudo mkdir -p /var/www/htdocs/cfg

So now that we have MRTG and, the perl modules it requires, and an snmp daemon running and an http daemon up to display our graphs; we’re ready to start configuring.

mschenck ~# sudo cfgmaker --global 'WorkDir: /var/www/htdocs'  \
          --global 'Options[_]: bits,growright' \
          --output /var/www/htdocs/cfg/mrtg.cfg    \
           public@localhost

Now lets schedule the polling of our NICs’ stats for the mrtg graphs. I’m going to put the task to root’s crontab:

mschenck ~# sudo crontab -u root -e

And then add the following cron schedule:

*/5 * * * *  /usr/local/bin/mrtg /var/www/htdocs/cfg/mrtg.cfg --logging /var/log/mrtg.log

Now, just watch the data start collecting.

Section "ServerLayout"
    Identifier     "X.org Configured"
    Screen      0  "Screen0" 0 0
    InputDevice    "Mouse0" "CorePointer"
    InputDevice    "Keyboard0" "CoreKeyboard"
EndSection

Section "Files"
    RgbPath      "/usr/X11/lib/X11/rgb"
    ModulePath   "/usr/X11/lib/modules/amd64"
    FontPath     "/usr/X11/lib/X11/fonts/TrueType/"
    FontPath     "/usr/X11/lib/X11/fonts/Type1/"
    FontPath     "/usr/X11/lib/X11/fonts/Type1/sun/"
    FontPath     "/usr/X11/lib/X11/fonts/F3bitmaps/"
    FontPath     "/usr/X11/lib/X11/fonts/misc/"
    FontPath     "/usr/X11/lib/X11/fonts/100dpi/"
    FontPath     "/usr/X11/lib/X11/fonts/75dpi/"
EndSection

Section "Module"
    Load  "IA"
    Load  "dbe"
    Load  "extmod"
    Load  "record"
    Load  "xtrap"
    Load  "GLcore"
    Load  "glx"
    Load  "dri"
    Load  "xtsol"
    Load  "freetype"
EndSection

Section "InputDevice"
    Identifier  "Keyboard0"
    Driver      "kbd"
EndSection

Section "InputDevice"
    Identifier  "Mouse0"
    Driver      "mouse"
    Option        "Protocol" "auto"
    Option        "Device" "/dev/mouse"
    Option        "ZAxisMapping" "4 5 6 7"
EndSection

Section "Monitor"
    Identifier    "Monitor0"
    VendorName    "Monitor Vendor"
    ModelName    "Monitor Model"

    HorizSync 30-80
    VertRefresh 60
    Option "DPMS"

    ModeLine "1920x1080" 148.50 1920 2008 2052 2200 1080 1084 1089 1125 +hsync +vsync

EndSection

Section "Device"
    Identifier  "Card0"
    Driver      "nvidia"
    VendorName  "nVidia Corporation"
    BoardName   "NV44 [GeForce 6200 LE]"
    BusID       "PCI:4:0:0"

        Option      "RenderAccel" "true"
        Option      "AllowGLXWithComposite" "true"

EndSection

Section "Screen"
    Identifier "Screen0"
    Device     "Card0"
    Monitor    "Monitor0"
    DefaultDepth 24

    SubSection "Display"
        Viewport   0 0
        Depth     1
        Modes    "1920x1080"
    EndSubSection
    SubSection "Display"
        Viewport   0 0
        Depth     4
        Modes    "1920x1080"
    EndSubSection
    SubSection "Display"
        Viewport   0 0
        Depth     8
        Modes    "1920x1080"
    EndSubSection
    SubSection "Display"
        Viewport   0 0
        Depth     15
        Modes    "1920x1080"
    EndSubSection
    SubSection "Display"
        Viewport   0 0
        Depth     16
        Modes    "1920x1080"
    EndSubSection
    SubSection "Display"
        Viewport   0 0
        Depth     24
        Modes    "1920x1080"
    EndSubSection
EndSection

Section "DRI"
        Mode         0666
EndSection

Then I had to fix gdm to play well with opensolaris
/etc/X11/gdm/gdm.conf

SystemMenu=true
RebootCommand=/usr/sbin/init 6
HaltCommand=/usr/sbin/init 5

I Switched from dtlogin to gdm for graphical login

# /usr/dt/bin/dtconfig -d
# svccfg
svc:> select gdm
svc:/application/graphical-login/gdm> setprop general/enabled = boolean: "true"
svc:/application/graphical-login/gdm> listprop general/enabled
general/enabled  boolean  true
svc:/application/graphical-login/gdm>
# usr/dt/bin/dtconfig -d
# /usr/dt/bin/dtconfig -d & svcadm enable gdm2-login

Add the following to /etc/X11/gdm/custom.conf under the “[daemon]” to enable auto-login for “media” user account
# sudo vi /etc/X11/gdm/custom.conf

AutomaticLoginEnable=true
AutomaticLogin=media

I had to add the following to the end of /etc/pam.conf to allow user “media” to be allowed local graphical auto-login

gdm-autologin auth required pam_unix_cred.so.1
gdm-autologin auth sufficient pam_allow.so.1
gdm-autologin account sufficient pam_allow.so.1
gdm-autologin session sufficient pam_allow.so.1
gdm-autologin password sufficient pam_allow.so.1

Add the following to the end of ~media/.dmrc to let gdm know the default window manager for user “media”

[Desktop]
Session=gnome

Xbox 360 and File Server access

Reference: The Book of PF

In this post I’ll describe how I accomplished two goals, as well as talk about a little trick for debugging what is blocked. The two goals are allowing my XBox 360 connect to Xbox Live through my OpenBSD firewall, and Port-forward to my file server so I can access my files remotely.

Let’s start with the Xbox 360. This is very minimal change from Part 4. In Part 4 we created two macros for allowed client services, the services we allow hosts on our local network to utilize, see here:

client_tcp_services = "{ ssh, smtp, domain, www, pop3, auth, https, pop3s, imap,
imaps, 8000, 8080, 5190, 5222 }"
client_udp_services = "{ domain, bootps, 67 }"

The Xbox 360 uses TCP port 3074, and UDP ports 88 and 3074 so we just need to add these to those macros, seen here:

client_tcp_services = "{ ssh, smtp, domain, www, pop3, auth, https, pop3s, imap, imaps, 8000, 8080, 5190, 5222, 3074 }"
client_udp_services = "{ domain, bootps, 67, 88, 3074  }"

Next, I set up NAPT or Network Address Port Translation. I redirect port 5022 on the external interface of my OpenBSD router to port 22 of my file server. This will allow me to SSH, SFTP and RSYNC to/from it when I’m away from home. To do this I need to add a single PF RDR rules, seen here:

# file server
rdr on $ext_if proto { tcp, udp } from any to $ext_if port 5022 -> $file_server port 22

Now I just reload my rules, first checking with pfctl -nf /etc/pf.conf as previously described, then loading with pfctl -f /etc/pf.conf.

One last thing is to describe a method for debugging why connections might be failing. In Part 4 I described blocking all unintended traffic with “block all”. For debugging I’m going to change this to “block log all”.

Now, when ever connectivity issues occur, debugging is simple running the following tcpdump:

# tcpdump -netvvi pflog0

As log as no other lines are configured to log, all output from this command will just describe traffic being blocked.

Our final /etc/pf.conf for this part of the project:

# Interface Globals
ext_if = "rl0"
int_if = "xl0"
wifi_if = "rum0"

# Static machines
file_server = "192.168.0.2"
xbox = "192.168.0.3"

# Protocol Globals
router_daemons = "{ ssh, domain, ntp, bootps, 8080, 5022 }"
client_tcp_services = "{ ssh, smtp, domain, www, pop3, auth, https, pop3s, imap, imaps, 8000, 8080, 5190, 5222, 3074 }"
client_udp_services = "{ domain, bootps, 67, 88, 3074  }"

# Provide NATing for my local subnets
nat on $ext_if from $wifi_if:network to any -> ($ext_if) static-port
nat on $ext_if from $int_if:network to any -> ($ext_if)

# file server
rdr on $ext_if proto { tcp, udp } from any to $ext_if port 5022 -> $file_server port 22

block log all
set skip on lo

# Allowed Client traffic
pass out on $ext_if proto tcp to any port $client_tcp_services
pass out on $ext_if proto udp to any port $client_udp_services

# Router services
pass proto icmp
pass quick inet proto { tcp, udp } to any port $router_daemons

Continue to Step 6.

Configuring the Interfaces

First step is to configure the interfaces. My external interface is rl0 (“Realtek 8139″ – you can see dmesg below), my internal wired interface is xl0 (“3Com 3c905B 100Base-TX”), and my wireless interface is rum0 (“Ralink 802.11 bg WLAN”).

As I stated in Part 1 the external interface (rl0) will be a dhcp client to my cable ISP. To configure this, I created /etc/hostname.rl0 with the following information:

dhcp NONE NONE description "Global Uplink"

ifconfig rl0 looks like this:

rl0: flags=8843 mtu 1500
lladdr 00:0d:87:07:07:56
description: Global Uplink
groups: egress
media: Ethernet autoselect (100baseTX full-duplex)
status: active
inet6 fe80::20d:87ff:fe07:756%rl0 prefixlen 64 scopeid 0x2
inet  netmask 0xfffffe00 broadcast 255.255.255.255

Next the internal wired interface (xl0); will be statically configured with the address 192.168.0.1 with a 24 bit subnet mask, later this interface while be advertising DHCP service for the wired local network. I created /etc/hostname.xl0 like this:

inet 192.168.0.1 255.255.255.0 192.168.0.255 description "Local Wired"

ifconfig xl0:

xl0: flags=8843 mtu 1500
lladdr 00:10:5a:0e:da:9a
description: Local Wired
media: Ethernet autoselect (100baseTX full-duplex)
status: active
inet 192.168.0.1 netmask 0xffffff00 broadcast 192.168.0.255
inet6 fe80::210:5aff:fe0e:da9a%xl0 prefixlen 64 scopeid 0x1

And lastly the wireless interface (rum0). This interface is being set up as a wireless access point (“hostap”) configured as follows:

• Will be set to only support 802.11B for the time being
• Utilizing channel 6
• Use a hex WEP key
• Configured as 192.168.1.1 with a 24 bit netmask
• And later will be advertising DHCP for the local wireless network

My /etc/hostname.rum0 looks like this:

up mediaopt hostap mode 11b chan 6 nwid Puffy nwkey 0x
inet 192.168.1.1

ifconfig rum0:

rum0: flags=8843 mtu 1500
lladdr 00:1c:f0:90:82:22
groups: wlan
media: IEEE802.11 autoselect mode 11b hostap
status: active
ieee80211: nwid Puffy chan 6 bssid 00:1c:f0:90:82:22 nwkey 0x 100dBm
inet6 fe80::21c:f0ff:fe90:8222%rum0 prefixlen 64 scopeid 0x5
inet 192.168.1.1 netmask 0xffffff00 broadcast 192.168.1.255

OpenBSD 4.3 (GENERIC) #698: Wed Mar 12 11:07:05 MDT 2008
deraadt@i386.openbsd.org:/usr/src/sys/arch/i386/compile/GENERIC
cpu0: VIA Ezra ("CentaurHauls" 686-class) 801 MHz
cpu0: FPU,DE,TSC,MSR,MTRR,PGE,MMX
real mem  = 259555328 (247MB)
avail mem = 242909184 (231MB)
mainbus0 at root
bios0 at mainbus0: AT/286+ BIOS, date 03/17/03, BIOS32 rev. 0 @ 0xfb390, SMBIOS rev. 2.2 @ 0xf0800 (43 entries)
bios0: vendor Phoenix Technologies, LTD version "6.00 PG" date 03/17/2003
bios0: VIA Technologies, Inc. VT8601
apm0 at bios0: Power Management spec V1.2 (slowidle)
apm0: AC on, battery charge unknown
acpi at bios0 function 0x0 not configured
pcibios0 at bios0: rev 2.1 @ 0xf0000/0xdd54
pcibios0: PCI IRQ Routing Table rev 1.0 @ 0xfdce0/112 (5 entries)
pcibios0: PCI Exclusive IRQs: 5 10 11 12
pcibios0: PCI Interrupt Router at 000:07:0 ("VIA VT82C596A ISA" rev 0x00)
pcibios0: PCI bus #1 is the last bus
bios0: ROM list: 0xc0000/0xc000 0xcc000/0x4000! 0xd0000/0x4000
cpu0 at mainbus0
pci0 at mainbus0 bus 0: configuration mode 1 (no bios)
pchb0 at pci0 dev 0 function 0 "VIA VT8601 PCI" rev 0x05
agp0 at pchb0: v2, aperture at 0xe0000000, size 0x10000000
ppb0 at pci0 dev 1 function 0 "VIA VT82C601 AGP" rev 0x00
pci1 at ppb0 bus 1
vga1 at pci1 dev 0 function 0 "Trident CyberBlade i1" rev 0x6a
wsdisplay0 at vga1 mux 1: console (80x25, vt100 emulation)
wsdisplay0: screen 1-5 added (80x25, vt100 emulation)
pcib0 at pci0 dev 7 function 0 "VIA VT82C686 ISA" rev 0x40
pciide0 at pci0 dev 7 function 1 "VIA VT82C571 IDE" rev 0x06: ATA100, channel 0 configured to compatibility, channel 1 configured to compatibility
wd0 at pciide0 channel 0 drive 1: <QUANTUM FIREBALLlct15 15>
wd0: 16-sector PIO, LBA, 14324MB, 29336832 sectors
wd0(pciide0:0:1): using PIO mode 4, Ultra-DMA mode 4
pciide0: channel 1 ignored (disabled)
uhci0 at pci0 dev 7 function 2 "VIA VT83C572 USB" rev 0x1a: irq 11
uhci1 at pci0 dev 7 function 3 "VIA VT83C572 USB" rev 0x1a: irq 11
viaenv0 at pci0 dev 7 function 4 "VIA VT82C686 SMBus" rev 0x40: 24-bit timer at 3579545Hz
xl0 at pci0 dev 8 function 0 "3Com 3c905B 100Base-TX" rev 0x24: irq 10, address 00:10:5a:0e:da:9a
exphy0 at xl0 phy 24: 3Com internal media interface
rl0 at pci0 dev 14 function 0 "Realtek 8139" rev 0x10: irq 11, address 00:0d:87:07:07:56
rlphy0 at rl0 phy 0: RTL internal PHY
cmpci0 at pci0 dev 15 function 0 "C-Media Electronics CMI8738/C3DX Audio" rev 0x10: irq 12
audio0 at cmpci0
opl0 at cmpci0: model OPL3
midi0 at opl0: <CMPCI Yamaha OPL3>
mpu at cmpci0 not configured
"C-Media Electronics HSP56 AMR" rev 0x20 at pci0 dev 15 function 1 not configured
isa0 at pcib0
isadma0 at isa0
pckbc0 at isa0 port 0x60/5
pckbd0 at pckbc0 (kbd slot)
pckbc0: using irq 1 for kbd slot
wskbd0 at pckbd0: console keyboard, using wsdisplay0
pcppi0 at isa0 port 0x61
midi1 at pcppi0: <PC speaker>
spkr0 at pcppi0
npx0 at isa0 port 0xf0/16: reported by CPUID; using exception 16
pccom0 at isa0 port 0x3f8/8 irq 4: ns16550a, 16 byte fifo
usb0 at uhci0: USB revision 1.0
uhub0 at usb0 "VIA UHCI root hub" rev 1.00/1.00 addr 1
usb1 at uhci1: USB revision 1.0
uhub1 at usb1 "VIA UHCI root hub" rev 1.00/1.00 addr 1
biomask ebed netmask efed ttymask efef
rum0 at uhub0 port 2 "Ralink 802.11 bg WLAN" rev 2.00/0.01 addr 2
rum0: MAC/BBP RT2573 (rev 0x2573a), RF RT2528, address 00:1c:f0:90:82:22
softraid0 at root
root on wd0a swap on wd0b dump on wd0b

Continue to Step 3.

Goals:

- Two local subnets, Wired and Wireless

- Support Xbox Live

- Provide external access to my file server

- Limit all communication out of my network strictly to the applications and services I choose to allow (web traffic, mail traffic, ssh, xbox, torrents, etc)

- Improved stability over the over-the-counter home routers

- Shape traffic using ALTQ

Resources:

- For the Operating System, I’m going with OpenBSD 4.3 (STABLE)

- Putting to use my MaxTerm 8300B

• EVE-M 800 mhz (x86)
• 256 MB of RAM
• 15 GB IDE hard drive (spare one sitting around)
• D-Link USB  Wireless Card (rum0 driver in OpenBSD) for local wireless traffic
• Spare old 3com 10/100 Mbit NIC for local wired traffic

The Plan:

• Local wired traffic: 192.168.0.X/24
• Local wireless traffic: 192.168.1.X/24
• Provide DHCP services on both local interfaces
• External Interface running DHCP
• Starting with blocking everything [block all]
• Designate IPs per server and personal computer on my network
• Designate torrent ports to each of the personal computers
• Initially just use WEP for proof of concept, later use WPA or even an authpf with a possible web interface for authenticating
• Perform MRTG graphing if for nothing more than to see what I spend my bandwidth doing

Continue to Part 2.

First disable automating; this may be volfs or hal depending on the version of solaris 10/11

 # svcadm disable volfs 

Now plug in the thumbdrive and determine the location for the logical node

bash-3.00# rmformat -l

Looking for devices...

1. Logical Node: /dev/rdsk/c0t0d0p0

Physical Node: /pci@0,0/pci-ide@1f,1/ide@0/sd@0,0

Connected Device: TEAC     CD-224E-N        1.AA

Device Type: CD Reader

2. Logical Node: /dev/rdsk/c4t0d0p0

Physical Node: /pci@0,0/pci15d9,7980@1d,7/hub@5/storage@1/disk@0,0

Connected Device: Corsair  Flash Voyager    1.00

Device Type: Removable

bash-3.00#

/dev/rdsk/c4t0d0 will be the root of what we’re working with in this example

Now run:

# fdisk -B /dev/rdsk/c4t0d0s0

Now you can get the geometry using the following:

# fdisk -W - /dev/rdsk/c4t0d0s0

Now, look at output from previous step and find max MB (MAXSIZE) of usable storage. You do this by multiplying bytes/sector (usually 512 at the top) with num sectors at the bottom of output (in my case, a 4 GB drive had roughly 7.9Million sections or so) divide by (1024*1024) to yield MB (should be 3900 MB or so for a 4GB flash stick.

slices: 0 = 2MB, {$MAXSIZE-3}MB, "wm", "root" :

1 = 0, 1MB, "wu", "boot" :

2 = 0, {$MAXSIZE - 1}MB, "wm", "backup"

Here is a “slices.txt” file for a 256 mb thumbdrive

slices: 0 = 2MB, 244MB, "wm", "root" :

1 = 0, 1MB, "wu", "boot" :

2 = 0, 246MB, "wm", "backup"

Now put this label to the drive

# rmformat -s ./path/to/slices.txt /dev/rdsk/c4t0d0p0

Now to refresh Solaris devices:

# devfsadm; devfsadm -C

Now to create the filesystem, mount it and copy the necessary files

# newfs /dev/rdsk/c4t0d0s0
# mkdir /mnt/backup
# mount /dev/dsk/c4t0d0s0 /mnt/backup
# cd /mnt/backup
# cp -r /boot ./boot
# cp -r /platform ./platform

Now lets make this bootable

# fdisk /dev/rdsk/c4t0d0p0

Set partition 1 active using option 2

# cd /mnt/backup/boot/grub/
# installgrub stage1 stage2 /dev/rdsk/c4t0d0s0

Now safely unmount

 # cd /;umount /mnt/backup

references:

• Giving USB the boot – install

[Taken from: www.unixzone.dk]

Netatalk 2.0.3 requires some patching to compile on Solaris 10 (or OpenSolaris)

Requirements:
• netatalk
• Berkeley DB 4.2.52
• GCC compiler, Sun Studio didn’t work for me
• Patches: netatalk-2.0.3/sys/netatalk/at.h and netatalk-2.0.3/sys/solaris/tpi.c

Click <HERE> for build instructions for DB

On Solaris we don’t use ranlib, LDFLAGS adds /usr/local/lib to the
library search path where my Berkeley DB libs are , rest of the options
are self-explanatory.

 # gzip -cd netatalk-2.0.3.tar.gz | tar xf -
# gzip -cd patches.tar.gz | tar xf -
# cd netatalk-2.0.3
# RANLIB=echo CC=gcc LDFLAGS=-R/usr/local/BerkeleyDB.4.2/lib
./configure --prefix=/opt/netatalk --with-ssl-dir=/usr/sfw
--with-bdb=/usr/local/BerkeleyDB.4.2 --without-pam --disable-ddp
--disable-tcp-wrappers  --disable-srvloc --with-cnid-dbd-backend
# echo "#define SOLARIS2 10" >>config.h

Depending on the version of your Solaris installation, you’ll want to
change this to match, ie. 8, 9, 10, or 11 for OpenSolaris.

Patch the source to support x64 Solaris

 # patch -i ../patches/at.h.patch sys/netatalk/at.h
Looks like a unified context diff.
done
#  patch -i ../patches/config.h.patch ./config.h
Looks like a normal diff.
done
# patch -i ../patches/endian.h.patch sys/netatalk/endian.h
Looks like a normal diff.
done
# patch -i ../patches/tpi.c.patch sys/solaris/tpi.c
Looks like a unified context diff.
done

Build and install the software

 # make
# make install

Under Solaris, you must create atalkd.conf, since Solaris provides no
method for determining the names of the available interfaces. It is
sufficent to name the available interfaces in atalkd.conf, one per line.
E.g.
eri0
on a line by itself on many Suns, hme0 on Ultras etc. See atalkd(8).

Create init script and add to Sun’s svc system

 # cp distrib/initscripts/rc.atalk.sysv /opt/local/lib/svc/method/netatalk

Place netatalk.xml somewhere on the file system

 # svccfg import /path/to/netatalk.xml
# svcadm enable netatalk
# rm /path/to/netatalk.xml

Now for configuration:

 # cd /opt/netatalk/etc/netatalk/
# ls -l
total 96
-rw-r--r--   1 root     root       5066   Apr  4 15:21 AppleVolumes.default
-rw-r--r--   1 root     root       25124  Apr  2 14:49 AppleVolumes.system
-rw-r--r--   1 root     root       11259  Apr  4 14:59 afpd.conf
-rw-r--r--   1 root     root       1059   Apr  4 11:57 atalkd.conf
-rw-r--r--   1 root     root       1429   Apr  4 15:01 netatalk.conf
-rw-r--r--   1 root     root       1479   Apr  2 14:49 papd.conf
drwxr-xr-x   2 root     root       512    Apr  3 11:49 uams
#

Add the following to “afpd.conf”:

 "Solaris AFP" -uamlist uams_guest.so -loginmesg "Welcome, $u!" -transall -noddp -tcp

Configure “netatalk.conf” as seen here:

 # Appletalk configuration
# Change this to increase the maximum number of clients that can connect:
AFPD_MAX_CLIENTS=50

# Change this to set the machine’s atalk name and zone, the latter containing
# the ‘@’ sign as first character — compare with nbp_name(3) if in doubt
#
# NOTE: If Netatalk should register AppleTalk services in the standard zone
#       then you need not to specify a zone name here.

#
#       If your zone has spaces in it, you’re better off specifying
#       it in afpd.conf if you realize that your distribution doesn’t
#       handle spaces correctly in the startup script. Remember to use
#       quotes here if the zone name contains spaces.
#
#ATALK_ZONE=”@some zone”
ATALK_NAME=`echo ${HOSTNAME}|cut -d. -f1`
# specify the Mac and unix charsets to be used

ATALK_MAC_CHARSET=’MAC_ROMAN’
ATALK_UNIX_CHARSET=’LOCALE’
# specify this if you don’t want guest, clrtxt, and dhx
# available options: uams_guest.so, uams_clrtxt.so, uams_dhx.so,
#                    uams_randnum.so
#AFPD_UAMLIST=”-U uams_clrtxt.so,uams_dhx.so”
# Change this to set the id of the guest user
AFPD_GUEST=nobody
# Set which daemons to run (papd is dependent upon atalkd):

ATALKD_RUN=no
PAPD_RUN=no
CNID_METAD_RUN=yes
AFPD_RUN=yes
TIMELORD_RUN=no
A2BOOT_RUN=no
# Control whether the daemons are started in the background
ATALK_BGROUND=no
# export the charsets, read form ENV by apps

export ATALK_MAC_CHARSET
export ATALK_UNIX_CHARSET

Add the following to “AppleVolumes.default”:

 :DEFAULT: cnidscheme:dbd
/Storage "Storage" rwlist:nobody