Tag Archives: Linux

Replicating user accounts on linux

We have several linux servers based on CentOS. Since we have only a small number of these machines, we do not use LDAP but just simple username and passwords.

When we have a new person, we just add that person to the main server and then copy the necessary lines in the files (/etc/passwd, /etc/shadow, /etc/group, /etc/gshadow) to the other servers. Since we use NFS for the home directories, there is nothing to do on that front.

See this article (http://www.cyberciti.biz/faq/howto-move-migrate-user-accounts-old-to-new-server/) for more details.

USB 3.0 transfer speeds on Linux

Thinking that USB 3.0 might be a reasonable way to attach lots of drives, I got a USB 3.0 portable 2.5″ 1TB drive from Seagate to test its transfer speed.

The following is the bonnie++ output from the Seagate drive connected to USB 3.0 in Linux:

Version  1.96       ------Sequential Output------ --Sequential Input- --Random-
Concurrency   1     -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks--
Machine        Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP  /sec %CP
bob             31G   981  94 72831   9 39674   5  4059  94 117863   6 114.7   1
Latency             14671us     901ms     887ms    9356us     161ms   64699ms
Version  1.96       ------Sequential Create------ --------Random Create--------
bob                 -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete--
              files  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP
                 16 14349  10 +++++ +++ 10884   5 26680  17 +++++ +++ +++++ +++
Latency             14111us     470us     538us     288us      27us     570us

This is the output from the eSATA connected ZFS pool:

Version  1.96       ------Sequential Output------ --Sequential Input- --Random-
Concurrency   1     -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks--
Machine        Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP  /sec %CP
bob             31G   172  99 93448  11 61622  11   419  98 182779  11 191.6   7
Latency             74909us   12032us     903ms   94470us     316ms     233ms
Version  1.96       ------Sequential Create------ --------Random Create--------
bob                 -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete--
              files  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP
                 16 11745  45 +++++ +++ 16822  55 11020  50 +++++ +++ 14196  53
Latency             30405us     509us     292us   47021us      24us    2572us

It is clear that the eSATA ZFS pool is faster but not by that much. Write speeds are nearly the same but read speeds are rather different. One factor against the USB 3.0 is that it is probably a slow disk. But the eSATA ZFS pool is connected to a 1x PCIe so it takes a hit too.

All in all, not a bad performance from the USB 3.0 drive. For folks who favour ease of connection, USB 3.0 might not be too bad.

bonnie++ output

As the data that I work with gets ever larger, the need for more and faster disks get ever more important.

In the past, 1TB was a large amount of storage but in today’s world, that is the amount you get in a small portable drive.

For my work, we routinely need storage in the 10s of TBs and to get to that kind of storage we need to aggregate disk. To understand the performance of these arrays of disk, I typically use bonnie++ but its output is not the easiest to understand.

See this page for help on the bonnie++ output.

Redirecting within a bash script

I use bash scripts to tie together different programs to get the results that I need and it is nice to be able to redirect all the stdout and stderr of all the programs to a single output file. In a bash script, you can add the following lines to the top of the script and that will redirect everything to the specified file.

exec > log.out
exec 2>&1

See the following links from which I got the information from:

http://www.tldp.org/LDP/abs/html/x17891.html

http://stackoverflow.com/questions/3173131/redirect-copy-of-stdout-to-log-file-from-within-bash-script-itself