Improve OpenSolaris COMSTAR iSCSI Server performance.

I setup OpenSolaris iSCSI and Linux Open iSCSI in Virtualbox to compare the performace. I can get 70~80% of local FS throughput on Linux iSCSI, but it is 10~20% on OpenSolaris iSCSI.
Some tuning can boost OpenSolaris iSCSI performance to be on par with Linux iSCSI

#Environment
Server: OpenSolaris 2009.06 VS Openfiler 2.3 on Virutalbox 3.1
Client: Centos 5.2
Regardless of the iSCSI Server type, the network has to be optimized by tuning network parameters
#OpenSolaris iSCSI tunning
1. Use the bock device(/dsk) as backstore, NOT character device (/rdsk), because the block device (/dev/dsk) has buffer.

$iscsitadm create target -b /dev/zvol/dsk/data/iscsi filera.dbvol

2. Turn on fast-write-ack

$iscsitadm modify admin --fast-write-ack enable

reactivate the zpool for existing target.

$svcadm disable iscsi/target iscsitgt
$zpool export poolname ;zpool import poolname
$svcadm enable iscsi/target iscsitgt

Use the option with care, from iscsitadm man pages:
Enables or disables fast-write acknowledgment. You should enable this option only if a system is connected to the power grid through a UPS.
Otherwise, data corruption could occur if power is lost and some writes that were acknowledged have not been completely flushed to the backing store.