0.3 Raspberry Pi Startup

Okay, first of I am going to mention that this is my 4th attempt posting this entry! WordPress, for some unknown reason , continues to erase my posts…so please bare with me as this post doesn’t appear as nice as I would like it to.  Anyways…for my 0.3 I had planned on testing all of the tools and utilities that I was experimenting with in my 0.2 release except this time I wanted to test them on the actual Raspberry PI device itself.

First, I should mention, that before it came to testing, I wanted to create my own partition on the Raspberry Pi, that way if I messed something up it would just be my partition that crashes and the whole system.  It’s fairly simple to create a partition so I looked at Chris’s instructions here which guide you on how to create a partition for the Raspberry Pi.  Unfortunately, I was unsuccessful in doing this.  For some reason whenever I would try to reboot the device with cmdline.txt pointing to my partition (dev/sda6) the device wouldn’t boot.  The SD card wouldn’t get initialized. I tried to do this 4-5 times and was unsuccessful every time. After speaking with Chris, he suggested that I use the default partition on the device and hopefully I wouldn’t do anything drastic to hurt it, which I didn’t.

The first utility I wanted to try on the Raspberry Pi was the hdparm command. The Raspberry Pi located at Seneca CDOT has a USB external hard drive connected to it, this is where most of the information s stored allowing for the SD card to have less information on it.  I wanted to see if I could speed up the hard drive speed with hdparm. Below are the sets of commands that I used.

Get drive information: hdparm /dev/sda
Gather more information: hdparm –I /dev/sda
Test the Speed: hdparm –d1 –c3 –u1 /dev/sda

NOTE: The below command did not complete successfully.
Increase the Speed: hdparm –d1 –c3 –u1 /dev/sda

After trying the above commands I realized that this drive is fairly new and it’s already at the fastest speed at which it can operate.

After realizing that the hard drive could not be optimized I decided that I should enable the readahead service to see if it makes a difference.  On most Fedora machines readahead is installed by default.  However since the Raspberry Pi has a custom kernel I had to install it manually using YUM.  After installing there isn’t much initial configuration. The configuration file is located in /etc/readahead/readahead.conf. After inspecting the configuration file, add the following line to /boot/cmdline.txt or grub.conf:

init=/sbin/readahead-collector

The above command will enable the readaheadservice to start at boot.

 After entering the command I rebooted the machine and this is what I got:

The picture is a bit blurry so I’ll explain what it says below:

****** Readahead Collector Started ******
Readahead-collector: cannot insert audit rules
init: readahead-collector main process (536) terminated with status 1

Essentially what’s its saying is that it cannot find the audit daemon.  The reason why is because audit wasn’t built into the kernel resulting in readahead not being able to start or work at all.  After doing some troubleshooting with Chris he helped me determine that a new kernel needs to be built that supports audit that way readahead can start.  Unfortunately there wasn’t enough time left in the semester for me to try a different kernel that supports readahead.

With time running out and my 0.3 deadline fast approaching, I figured that there has to be something I can do on this device to make it boot a little bit quicker.  So I took another look at the current services running and determined that a few more services could be turned off.  Those services were: gpm, pcscd, avahi-daemon and udev-post.  I disabled these services for run level 3 and it is booted normally without any errors.  Below is a list of the services that are currently enabled on the Raspberry PI located at Seneca CDOT.

avahi-daemon        3:off
cups                3:off
gpm                 3:off
haldaemon           3:on
httpd               3:off
iptables            3:on
lvm2-monitor        3:off
mdmonitor           3:off
messagebus          3:on 
netconsole          3:off
netfs               3:off 
network             3:on
ntpd                3:on 
ntpdate             3:off
pcscd               3:off
portreserve         3:off
rdisc               3:off
rsyslog             3:off
saslauthd           3:off
sshd                3:on
svnserve            3:off
udev-post           3:off

As you can see there are only six enabled services on this device. It takes approximately 22.6 seconds for these services to boot; however this doesn’t include the initial boot that initialize the SD card and loads the modules.  For the SD card to initialize and load the modules it takes about 18.3 seconds.  Ive also seen it take as long as 25 seconds. The reason being that the Raspberry PI has a couple of modules that fail on the startup, during the boot it tries to load the modules which takes a lot of time this is why the device doesn’t boot that fast.

In order for the Raspberry PI to have a quick startup then it needs to be able to successfully load every module. This is what’s taking the longest to load, after the failed modules it takes 22.6 seconds to load the services.  Technically, if everything is loading properly it should take the device approximately 25-30 seconds to boot instead of the 36-40 seconds that it takes now. Keep in mind this isn’t incorporating readahead or any other utilities.  If the device had readahead enabled I would say that the device would boot in at least 25 seconds.  Below is a list of some of the deficiencies I’ve noticed when using the device. Some can be fixed while others will have to remain the same.

Deficiencies

As mentioned previously when device is first powered on it takes 18.3 seconds to initialize the SD card, load the modules and display errors.  However this time varies. I have seen it take 25 seconds to initialize the SD card. This should not take this long; no modules should be failing.

Services Deficiencies:

- To mount the USB external hard drive takes at least 5 seconds
- Takes 5 seconds to mount /swap files
- Takes 4 seconds for the haldaemon to start
To sum it all up I would say that with the current kernel that is configured on the Raspberry PI that this is the fastest speed it can get.  However if the errors are fixed at the beginning of the boot i.e. (failed modules etc…) I would definitely say that the boot could go from 36 seconds to at least 25 seconds.

With a new kernel that supports auditd & readahead, I would definitely think that the device should easily be able to boot in about 20 – 22 seconds.

You may ask where I am getting all of these boot estimations from. In my 0.1 post I mentioned that I wanted my goal to be getting the device to boot in 15 seconds.  While I couldn’t meet that goal for reasons beyond my control (failed modules, kernel issues) I figured that I should make a virtual machine with the same specs and the PI. I did this in my 0.2 I was able to make a virtual machine that just had the same services running as the PI in 11.6 seconds; you can watch that video here.

I have done more work to that virtual machine. I disabled more services that I mentioned earlier in my post; enabled readahead speed up the disk with hdparm mounted some file systems temporarily and before you knew it I had my virtual machine booting in 6.9 seconds! I definitely didn’t expect it to boot that quick.  Below is a video of the boot:

6.9 Second Boot video

These are the services running on the virtual machine, the exact same as the Raspberry PI located at Seneca CDOT.

These are the file systems I mounted as tmpfs.

I’m currently working on a bash script that will turn off all unneeded services and enable readahead.The reason why I haven’t uploaded it yet is because I know that everyone’s system is setup differently and I didn’t want to create something that’s going to harm or potentially crash your system. I want the script to be as efficient as possible without harming your system.  The script will be uploaded sometime next week.

Overall I actually loved experimenting and doing the work for Software Build and release.  This project was very interesting to me and lead me to learn a lot of new techniques and concepts. I definitely didn’t expect to find myself creating a virtual machined boot in 6.9 seconds, that’s for sure.  It also felt good actually working on a device that hasn’t even been fully released yet.  Now I can say that I know all about the Raspberry PI Foundation  and that I helped make it boot more efficiently and faster.

0.2 release

My 0.2 release consisted of a lot of research and playing around with a few virtual machines that I had setup.

I first wanted to see if it was possible to mount the log files as temporary files.  There are some advantages and disadvantages to this; one advantage is that if the log files are mounted as temporary files, they will be mounted a lot quicker and there is less data to process.  The disadvantage of this is that the logs will be erased on every start up.  I mounted the log files as temporary files and I found that there was about one second difference in the boot. Below is a screen shot of my /etc/fstab file.

As you can see, in the above screenshot I have set the /var/log, /tmp and /var/tmp files to be mounted as tmpfs.

Another tool that I have been experimenting and benchmarking with is the hdparm command. This command has been giving me some trouble in the virtual environment.  Once I get my partition up and running on the actual Raspberry Pi then I will play around with it more.  I would like to use hdparm to determine if i can see what speed the disk is operating at and If I can increase the speed.  The commands I will use will be similar to the following:

  • Test Speed: hdparm -tT /dev/hda
  • Increase Speed: hdparm -d1 -c3 -u1 /dev/hda

I should have more to say about this command/utility in my 0.3 release when I have more time to test it.

Another service that I have recently just started researching and working on is the readahead service.  For those of you who don’t know, readahead is a service normally installed by default on Fedora that prefetches files into system memory before being needed.  So when a file is accessed, the contents are read from the physical memory rather than the disk, which is much faster. If configured correctly, readahead can be used to speed up boot times substantially.  This is another utility that I would like to implement on the Raspberry Pi within the next few days.  I am currently doing research on how to configure readahead to start at boot.

I should also mention that I copied a Virtual Machine from my laptop to my desktop computer.  After opening it up in VMware 8 I quickly noticed how much faster the boot times are.  On my laptop which has 4GB of ram and an Intel i7 CPU, the VM boot is approximately 20.1 seconds, which isn’t bad; however, when I copied it to my desktop which also has 4GB of ram and an AMD Phenom II X6 CPU the VM boots in 11 seconds! I did not expect this. The link below is a short video that I took of the Virtual Machine booting on m desktop.

Raspberry Pi Virtual Machine Boot

This is why I want to move from the virtual machines to the actual Raspberry Pi device, it will allow me to try the things that I want in real time and my results won’t be different; unlike virtual machines where one could be booting a lot faster than another even when they have the exact same configuration.

0.1 Update

As I mentioned in my previous post my goal is to make the Raspberry Pi boot in approximately 15 seconds.  I am on my way to reach this goal, currently I have a few Virtual Machines with the same hardware specs as the Raspberry Pi booting in 20.1 seconds.  I did this by limiting the services that are running at boot.  Below is a list of the services that are currently running:

These services that are running are the same services that the Raspberry Pi is currently running.  By using a program called Bootchart I can visually see what components and modules are taking longer to boot.  In my next release I plan on incorporating a few more services that will speed up the boot process, I also plan on looking into init scripts and seeing if i can make any changes there. Stay tuned!


0.1 Overview

For my project I chose to work on optimizing and tweaking the boot process of the Raspberry Pi. For those of you not familiar with the Raspberry Pi, let me explain. The Raspberry Pi Foundation is a charity in the UK that’s promoting computer science and other related fields. They’re currently developing and manufacturing a ridiculously low cost computer that’s designed to teach computer programming to children;  however, this is not the only use for this device.  The Raspberry Pi can be used for many things such as programming, web development, robotics, administration and of course, personal use such as browsing the internet and emailing.  The best feature of this  device is that it sells for $25. This will allow for anyone to purchase the device once available. The cost also reduces the chances of it  stolen.   Below is a picture of the Raspberry Pi:

As you can see there isn’t much on this little  device. It’s equipped with HDMI  and Composite video output, 2 USB ports, general purpose I/0, 10/100 ethernet and it boots off an SD card which is located on the bottom.

The current boot process of the device that Seneca has is roughly 25 – 30 seconds.  My goal, for the end of the semester, is to have device booting in approximately 15 seconds.

For more information on other projects you can click the following link:Potential Projects

Creating a Repository

In this lab we were given the task of creating our own YUM repository.  There are three main important parts to this task:

  1.  Create a GPG key
  2. Sign the package
  3. Put the files in a web accessible location

Creating a GPG key is actually quite simple you just enter one command:

gpg --gen key

Simply follow the steps associated with the above command and your key will be created successfully.  The reason why this key is being generated is because we need a key in order for us to be able to sign packages.

After the key has been successfully created then you want to edit the ~/.rpmmacros file and insert your key I.D that was created.  This ensures that when we sign a package it will use our key.  Below is a screenshot of my ~/.rpmmacros file.

After I edited the ~/.rpmmacros file I was able to successfully sign a package using the following command:

rpm --addsign packagefile

Signing the package allows a user to verify the source of a package. It also doesn’t ensure the integrity of the package, it just verifies the package’s source.

After the package has been signed,I executed and ran the createrepo program:

 createrepo .

This program creates an xml-rpm-metadata repository.

After creating the repository, I uploaded it to my matrix account and put it in the public_html directory.  Below is a link to my repository:

My Repo

Mock and Koji

For this lab we were given the task of using Mock and Koji to test and resolve the dependencies from our RPM that we created in the previous lab.

Mock

Mock is a vital tool that’s used for resolving dependency requirements for packages.  Mock creates a sandbox environment where it isolates the RPM.  That way the RPM can be tested with no interference from any outside packages.  This tool is essential in the RPM build process.

Koji

Koji is similar to mock with a few more features.  Koji has full web interface where users can see their package’s current status and whether it passed or failed within the build system.

When installing Koji a certificate is generated making Koji extremely secure.  Only Fedora packagers are granted the certificate.  If you were to try to manually generate a certificate then you’d have some difficulty in doing so.  Below is a screen shot of what the Koji interface looks like.

When submitting a job to Koji just enter the following command:

koji build dist-f15 --scratch yourSRPM

After this command has been entered you will be notified if the build is either successful or unsuccessful.  Once this build is successful then you can loin to Koji to view the state of your build.  Below is a screenshot of my build in Koji.

 Overall Experience

At first using these tools was very difficult to understand the process of how everything worked.  It took a lot of troubleshooting and error checking to understand the concept of Mock.  I learned that the Spec file has to have all of the required dependencies order for the build to work.

Koji also took some time to understand as well.  At first I had problems importing my certificate but after generating a new one it imported successfully.  What I like about Koji is that its a practical tool that you can check in real time what the status of your package is.  I find Koji very interesting and I would like to learn more about it in the near future.

RPM Wiriting

After some extensive troubleshooting I was finally able to successfully build an RPM package from the source RPM.  I tried several packages when doing this lab but the only one that seemed successful was gzip, an open source package compressor and archiver.

When building the package I quickly learned that much of the program is written in the spec file and if some of the required files aren’t in the spec file then the RPM package wont build. Below is a screenshot of my gzip spec file:

Below is a link to my source RPMs:

http://matrix.senecac.on.ca/~jbhuestis/gzip-1.4-1.fc15.i686.rpm

http://matrix.senecac.on.ca/~jbhuestis/gzip-1.4-1.fc15.src.rpm