Linux : sheevaplug as a perfect NAS – Reloaded
Since more than 2 years now, I’m using a Sheevaplug as a low power NAS (see this article). Until now, I was using a Debian 6 “squeeze”, with an old 2.6 kernel.
I had some trouble with Samba, like crash with high usage on small files (eg. Kodi’s library update). In addition, after 2 years of Debian, I was still not convinced and I wanted to use my favorite distro : Archlinux.
I’m using the same “NAS softwares”, but a little bit more tuned…
At the time I dit the whole install (October, 12), following the official Archlinux Arm install doc was not working.
On a SDCard, I created a small FAT partition (128 Mb), and another ext4 partition with the remaining space. As always, I optimized the SDCard (see this article). and formated the ext4 partition with (SDCard as /dev/sdb):
mkfs.ext4 -O '^has_journal' -E stride=2,stripe-width=1024 -b 2048 /dev/sdb2
Then I deployed Archlinux base system on the SDCard (from my Laptop):
cd /tmp mkdir root mkdir root/boot mount /dev/sdb2 root mount /dev/sdb1 root/boot wget http://archlinuxarm.org/os/ArchLinuxARM-kirkwood-latest.tar.gz bsdtar -xpf /home/juju/Downloads/ArchLinuxARM-kirkwood-latest.tar.gz -C root sync umount root/boot umount root
After that, booting required to use the serial connection (usb cable) to set some uboot variables. I plugged the cable to my laptop and switched on the Sheevaplug. Then I used GNU Screen to connect to the Sheevaplug. I had to do this very quickly in order to interrupt the boot process by pressing <enter> as soon as the Sheevaplug booted:
screen 115200 8n1 setenv bootcmd 'setenv bootargs $(bootargs_console) root=/dev/mmcblk0p2 rootdelay=5; run bootcmd_mmc; bootm 0x00800000' setenv bootcmd_mmc 'mmc init; mmc init; ext2load mmc 0:1 0x00800000 /uImage;' saveenv boot
Setup the base system
First actions I did :
- change the hostname (/etc/hostname)
- create a new user
- delete the default “alarm” user (userdel alarm then rm -R /home/alarm)
- change root password
- change /etc/systemd/journald.conf to lesser SDCard write : SyncIntervalSec=10m
- custom mount options in /etc/fstab for root filesystem :
/dev/mmcblk0p2 / ext4 defaults,async,barrier=0,commit=100,noatime,nodiratime,errors=remount-ro 0 1
Install Yaourt and prepare for AUR
I set optimized CCFLAGS and CXXFLAGS in /etc/makepkg.conf and added the following parameters to the default ones (see https://gcc.gnu.org/onlinedocs/gcc/Optimize-Options.html and https://gcc.gnu.org/onlinedocs/gcc/ARM-Options.html) :
-mtune=xscale -fweb -frename-registers -fomit-frame-pointer
Then, I installed yaourt by
- installing base-devel : pacman -S base-devel
- adding my user to /etc/suders (yaourt refuse to run as root, for safety reasons)
- adding archlinuxfr repo to /etc/pacman.conf : Server = http://repo.archlinux.fr/arm
At this point, I had to manualy install package-query for arm as yaourt depends on it, but package-query does not exists as compiled package for arm devices
git clone https://aur.archlinux.org/package-query.git cd package-query makepkg -Acs sudo pacman -U package-query-1.7-1-arm.pkg.tar.xz sudo pacman -S yaourt
Cronie for cron (https://wiki.archlinux.org/index.php/Cron#Cronie) :
yaourt -S cronie timedatectl set-timezone Europe/Paris
I changed /etc/anacrontab to make cronjobs execution during the night :
Then I installed hdparm and a systemd rc.local implementation to easily put some customisations at boot.
yaourt -S hdparm rc.local sudo systemctl enable rc-local
/etc/rc.local with customization :
# deadlines cheduler could group small accesses to lesser sdcard latency echo deadline > /sys/block/mmcblk0/queue/scheduler echo deadline > /sys/block/sda/queue/scheduler echo deadline > /sys/block/sdb/queue/scheduler # NAS Disks setting # set hd standby # 50 sec for mirror disk hdparm -S 10 /dev/sdb & # 24 min for data disk hdparm -S 255 /dev/sda & # optimize disk readahead buffer / noise option /usr/bin/hdparm -a 1024 /dev/sda /usr/bin/hdparm -a 1024 /dev/sdb /usr/bin/hdparm -M 254 /dev/sda /usr/bin/hdparm -M 254 /dev/sdb # eth0 optimize /usr/bin/ifconfig eth0 txqueuelen 5000
- Allow the server to send mails : https://wiki.archlinux.org/index.php/SSMTP
- Monitor NAS drive by installing smartmontools and follow instructions at https://wiki.archlinux.org/index.php/S.M.A.R.T.
Samba and network optimizations
As always the Archlinux wiki was a very good start point. The important steps are :
- setting “hosts allow” to restrict accesses
- set “encrypt passwords = yes”
- set “security = user”
- create users according to systems accounts with “smbpasswd -a <login>”
I found 2 great articles about network optimizations:
https://linuxengineering.wordpress.com/2014/08/03/performance-tuning-with-pogoplug-v4 (similar to Sheevaplug, but a little less powerfull)
I ended up with the following customizations with which I currently have 18.6 Mb/s when reading and 17.8 Mb/s when writting.
net.core.rmem_max = 5603328 # 0,75 of wmem_max rounded to 4096 net.core.wmem_max = 4194304 # set tcp mem to 4M max (default = 16k) net.ipv4.tcp_rmem = 4096 87380 5603328 net.ipv4.tcp_wmem = 4096 16384 4194304 net.ipv4.tcp_timestamps = 0 # less CPU usage on small arm soc net.core.optmem_max = 65535 net.core.netdev_max_backlog = 5000
/etc/samba/smb.conf (network settings only) :
strict allocate = Yes read raw = yes write raw = yes strict locking = No socket options = TCP_NODELAY SO_KEEPALIVE IPTOS_LOWDELAY SO_RCVBUF=131072 SO_SNDBUF=131072 min receivefile size = 4096 use sendfile = true aio read size = 4096 aio write size = 4096 oplocks = yes max xmit = 65535 max connections = 16 deadtime = 15 getwd cache = yes
Regain : a search engine
Regain is a nice and simple search engine, initialy made for desktop search. It is composed of a crawler and a web interface for searching… So it can also be used as a server search engine.
It requires java (6 or 7) and tomcat6 to run. Installation guide can be found in Regain manual. I installed it in /opt/regain and made symlink for the webapp to be found by tomcat (in /var/lib/tomcat6/webapps)
I also installed pdfbox to better index pdf files. I had to change the source url in the PKGBUILD (yaourt -S pdfbox) because the default site was too slow (see http://www.apache.org/dyn/closer.cgi for mirrors)
I activated the use of PDFBox by uncommenting the dedicated “preparator” in /var/lib/tomcat6/conf/regain/SearchConfiguration.xml.
With tomcat6 started at this point, regain just displayed a message saying that there was no index. Building one needed to launch the indexer :
cd /opt/regain/runtime/crawler java -Xms128m -Xmx128m -jar regain-crawler.jar
It can take a lot of time depending on the number of files to index…
With an index, the web interface allowed to search and see some “google like” results. But clicking on a result did nothing !… Regain is a desktop search engine, so it generates links like “file:///” which simply can’t work from a workstation.
Chance is, regain allows to change links prefixes. With Windows workstations, links like “\\server\share” should work, but I only have Linux and Mac OS X clients. So the only “universal” prefix that could be recognized by any browser was… the (not so) good plain old ftp protocol (ftp://server/). This protocol is clearly not secure and FTPS or SFTP don’t work with all browsers. So I choosed to use a simple FTP connection for search result consultations. It’s not secure, but I only use the search from time to time, so un-encrypted passwords are only passing my network very few times.
Main options of my vsftpd :
- local users only
- limit to true users
- set home of all users to /home (for chroot)
- CHROOT users
Then in /var/lib/tomcat6/conf/regain/SearchConfiguration.xml, I added the following rewrite rules for all results :
<rewriteRules> <rule prefix="file:///myShares" replacement="ftp://myServer.address"/> </rewriteRules>
I could have used tomcat to securely access files in https by activating directory listing, but it would have meant
- to give tomcat user right to access all files – not that secure
- to duplicate system accounts for access restriction – not really nice
- to chroot tomcat user to where NAS sources are – not sure it could work correctly
- to set a signed certificate – hmm… that’s just a home network 😉
Pingback: Deploying Saltstack : master and minion (archlinux on ARM)