As you might or might not know, Eucalyptus 3.3.0 is bringing better support for the AWS JAVA SDK and the developers are working hard , that every aspect is covered in some way or the other.

If you want to read more on that, I would recommend the following:

https://github.com/eucalyptus/architecture/wiki/aws_java_sdk-3.3-design
https://github.com/eucalyptus/architecture/wiki/aws_java_sdk-3.3-spec

Those 2 links will give you a decent idea of what is coming in 3.3.0 with regards to AWS JAVA SDK support.

Now on to this blog post, well 3.3.0 is currently under development, the brave hearts can go and grab the source from here https://github.com/eucalyptus/eucalyptus , in the testing branch , lives the bits

If you are not so brave and you are running Eucalyptus 3.1.2 or 3.2.0/3.2.1 well this blog post is for you, because yes, you can use AWS JAVA SDK 1.3.14 against your Eucalyptus private cloud.

I spent considerable time back in July 2012 to figure out what is going on with this thing and I came up with 2 tickets in our bug tracking system which are available here

https://eucalyptus.atlassian.net/browse/EUCA-3001
https://eucalyptus.atlassian.net/browse/EUCA-3003

At that time I wrote some ugly patches , basically removing new stuff and adding back what was there earlier in the AWS JAVA SDK , to make it work with Eucalyptus.

Then recently, I saw 2 commits from Steve Jones one of the CLC developer at Eucalyptus , here to the AWS JAVA SDK

https://github.com/sjones4/aws-sdk-java/commit/88acbcc5d6a5db9a0855fa98de56341bc3772283
https://github.com/sjones4/aws-sdk-java/commit/211e220fede28d4c36d335910e8b86e035f9b931

I decided I will take these 2 commits and apply them to the AWS JAVA SDK version 1.3.14 , which I made to work in July 2012. So I went ahead and reverted the patches I had done and worked on applying the above 2 commits on the version 1.3.14

The code is available here, on my github local fork

https://github.com/jeevanullas/aws-sdk-for-java

And as usual the patch is here

https://github.com/jeevanullas/aws-sdk-for-java/commit/905311ae4d049696baf108aff6e87ddc1eab84ca

Thats about it. For people who would like to build from source , they can clone my fork and build from it, get the JAR file , start playing right away.

For those who do not have much time, I have build the JAR and have uploaded that here

http://jeevanullas.in/tmp/aws-java-sdk-1.3.14.jar

Feel free to grab and use it.

I did try to use the latest version of AWS JAVA SDK but could not make it work with the patch, on Eucalyptus 3.2.0/3.2.1/3.1.2 , so this is the latest version of AWS JAVA SDK, which I know off, that works.

Again don’t think it is end of the road, Eucalyptus 3.3.0 is bringing in better support for AWS JAVA SDK and, I have tried that personally, last known version of SDK I tried was 1.3.26 and it works like a charm, plus 3.3.0 Eucalyptus has got the other cool stuff like ELB/Autoscaling/Cloudwatch, so definitely something forward to look at.

In the blog post to follow up I will try to give few examples of how you can use the AWS JAVA SDK to write code which talks to Eucalyptus, because then we can see how many cool things are in store, so stay tuned and healthy!

The version of openbsd used is 5.2 amd64 and Eucalyptus 3.2.0 on CentOS 6.3. Note that there were problems with running an instance store backed EMI and hence I ended up running an instance from boot-from-EBS EMI.

For the sake of simplicity, this post is divided into 5 parts :

Part – 1 , Build the base image

First of all download the openbsd 5.2 install ISO from the following link:

ftp://ftp3.usa.openbsd.org/pub/OpenBSD/5.2/amd64/install52.iso

Next on your machine (having virt-manager and KVM), run virt-manager to create a new virtual machine, virt-manager provides an easy to use GUI based interface to create a virtual machine. The wizard based process is pretty slick, screenshots below shows the details that were given:

After this the VM would boot, and we need to just follow the installation process as we do normally for openbsd; I followed the following link:

http://www.openbsd.org/faq/faq4.html#More

Make sure you ensure the SSH and NTP services are installed and configured to start automatically at boot, this is really helpful, the only tricky part is the disk partitioning, I used a custom disk partitioning , using the disklabel ; following link would help do that

http://www.openbsd.org/faq/faq4.html#Moredisklabel

The need for a custom layout and a disk large as 10 GB (Check the wizard screenshot above) was because I planned on using the openbsd ports and install necessary software in my image.

Once the install is finished; just reboot the VM, next we will do some configuration

Part -2 Modify the base image, to include necessary tools and configuration

In my tests , I have found that you need VIRTIO enabled on the Node Controller (Eucalyptus) to make this openbsd EMI work; there are couple of things to do , to make sure the EMI works that way;

VIRTIO network driver if used creates a device file in the openbsd instance; vio0 , we need to make sure the network configuration is placed such that this device gets a DHCP provided IP address during boot, this is how we do it on openbsd

echo 'dhcp' > /etc/hostname.vio0

Next I followed this link here , to get ports installed and configured on this VM;

http://www.openbsd.org/faq/faq15.html#Ports

I basically put wget and curl inside the VM , because those help with getting meta-data from within the instance when it is running on Eucalyptus

Finally I put a custom version of rc.local in /etc , to get the SSH keys working, for the sake of completeness I have uploaded it on github at the following URL;

https://github.com/jeevanullas/cloud-scripts/blob/master/rc.local

NOTE: The above script is copied and hacked together (to make it work on openbsd) from the original script available here https://github.com/eucalyptus/Eucalyptus-Scripts/blob/master/rc.local

I also made sure that the SSH configuration within opebsd strictly only allows for SSH key base authentication and not password by modifying the necessary configuration in /etc/ssh/sshd_config

Now shutdown the VM and copy the virtual disk file in /var/lib/libvirt/images/ for this VM to the Eucalyptus Storage Controller (SC)

Part -3 Upload the base image to Eucalyptus and get an EMI

From Eucalyptus side ; the first requirement is to make sure we use VIRTIO for everything on the NC, so on the NC , please make sure inside /etc/eucalyptus/eucalyptus.conf USE_VIRTIO_* is all set to 1

The openbsd EMI would be a boot-from-EBS (bfEBS) EMI because that is what works, the instance store backed EMI seems to not work due to natural reasons of kernel/ramdisk issues;

Next create a 10G volume on the Eucalyptus cloud and attach it to any running instance (any running instance, from any image), basically need to dump the image file for openbsd we got in Part -2 into the volume

euca-create-volume -s 10 -z cloud3

euca-attach-volume -d /dev/vdb -i i-05CB3829 vol-9B4F3F76

Now go on the SC and dd the openbsd image to the corresponding LV device for the volume you attached above (you need to figure out the LV for your volume ; its pretty easy if you have only 1 volume attached to an instance in the whole cloud, otherwise some manual work is involved IIRC)

dd if=openbsd-blog.img of=dev/vg-itL1qZZjVsCuLg../lv-D0fGsw..

Once the above is finished, detach the volume from the running instance and snapshot it

euca-detach-volume vol-9B4F3F76

euca-create-snapshot vol-9B4F3F76

Once the volume is snapshotted we can just register the EMI out of the same using;

euca-register -n "openbsd" --root-device-name /dev/sda1 -b /dev/sda1=snap-60FA3B94

This should give you an EMI-ID which you can run instance out off;

Part -4 Run an instance

Now we would run an instance from this EMI and check if we can access it over SSH;

euca-run-instances -k sshlogin -t m1.large emi-2D8A446A

ssh -i sshlogin root@10.104.3.1

There you go! Your own openbsd instance on your own Eucalyptus private cloud, with root access, start playing!

Part -5 Run eutester, Instance check on the EMI

This last part was little tricky to crack and there are some loose ends, which I would like help on, from the openbsd users/developers on internet;

Basically we run a set of tests on the EMI via the eutester test suite to verify the EMI, you can check out more on the test here;

https://github.com/eucalyptus/eutester/blob/testing/testcases/cloud_user/instances/instancetest.py

The results for the test is stored here;

https://github.com/eucalyptus/image-verification-results

For openbsd there were 3 things that failed

• Instance meta-data does not return any ramdisk ID – This is natural as there is no ramdisk associated with the bfEBS EMI
• Ephemeral storage was not found – Apparently the check to find the ephemeral storage does a ; ls -l /dev/sda2 , this device does not exist on the openbsd instance, so it failed
• EBS Volume attach fails – This is important, because as soon as the volume gets attached to the running openbsd instance , the instance is not reachable over SSH or via console, it appears that the hotplug for disk is causing the kernel to panic, if we detach the volume and reboot (euca-reboot-instances) it comes to life again;

The result of the test are available on the following link:

https://github.com/jeevanullas/cloud-scripts/blob/master/openbsd-eutester-result

Hope this blog post is useful for the openbsd lovers and they would enjoy reading and running their favourite openbsd instances on top of Eucalyptus cloud without much hiccup;

My prime goal for day 2 was to have the lab ready for the Eucalyptus workshop at 2:30PM. I would really like to thank every one of the volunteers part of this lab setup in helping me get ready for the workshop. We took around 5 machines and setup cluster controller, cloud controller, storage controller and walrus on one machine. Rest all machines were our node controllers.

Volunteers were pretty excited about the preparation of the lab. They helped me with small small things like getting a extra network interface card for the server, a local switch and my macbook displayport to VGA converter so I can share my screen on a HDTV. They also prepared and recorded a complete video of the workshop. I am not sure if I would be able to get it but if I do it could be really helpful for me as well as the whole community I hope.

Special thanks for Chiraj Jain and Yashodhan for helping me the setup of eucalyptus packages on the machines and Yashodhan also wrote a simple bash script to automate certain things in eucalyptus installation and configuration process.

I had around 10 registrations for the workshop and plus almost same number of volunteers eagerly waiting for the workshop to get a feel of eucalyptus. We have planned for a live setup of eucalyptus 2.0.3 on centos 5.7 x86_64 manually. It was the hard way of doing things but that brought in lot of technical insight which I shared with the audience using a whiteboard.

I think the workshop was too technical again for many of the participants and the time duration allocated for the workshop was less considering we did the whole setup during the workshop. I was asked couple of good questions and was able to answer them pretty well I hope. In the end while booting up the VM in our cloud it didn’t run and kept in pending state. It seems to be the problem with the transfer of the base image to the node but I was glad to show a running setup to the participants I had with me.

I discussed with the participants about Eucalyptus 3, the new high availability feature in Euca 3 how it is useful many ways, we discussed about eustore to upload EMIs on eucalyptus cloud, I asked them to try out faststart to get a eucalyptus cloud up and running in no time and also check out eucalyptus community cloud service.

We also discussed about cloud security, keys etc. and networking modes how they work in eucalyptus. I was able to show how to setup a MANAGED-NOVLAN mode on a HDTV which looked pretty cool.

I realized that giving a technical workshop eats up all your time , which includes setting up of the lab also. I was not able to catch up with other speakers during day 2 but Atul did a great job at that. Atul had lot of good discussions with folks from RedHat around localization initiatives and he has promised me to write a blog about it soon.

Also I hope Atul and others would be sharing photographs from the event. Till next time , keep rocking!

This year the main theme for the conference was Cloud Computing as well as Mobile Computing and I was happy to see lot of good talks and workshops on the same.

Day 1

Day started with a south indian breakfast. Idly and sambhar after a month’s gap for me. Felt pretty good. I met Shakti Kannan now with RedHat India, Santhosh Thottingal from Wikimedia Foundation and Arky from Mozilla.

My talk on Eucalyptus was scheduled at 11:15am in the auditorium at SICSR. I got delayed because the first talk on building your own cloud computing infrastructure by Mr. Arun Khan took some more time, by some 15-20 minutes, but that was fine. I spoke with Mr. Arun briefly before my talk and he told me about his team’s recent work with openstack and Eucalyptus , we shook hands and decided to take the talk further during dinner/lunch which we did and I hope I was able to answer his questions and give my honest opinion.

I had around 40+ people in the audience for my talk and was introduced by one of the college volunteer to the audience. It was a very formal introduction digging into my past which refreshed my memories. I had prepare my deck hoping that I would be talking to an audience who may not be so familiar with Cloud Computing as most of them would be college students. It was a good guess and I was able to figure that based on the first few questions I asked the audience as part of the regular polling exercise I prefer doing.

My slide deck is now available for you to view here. I was talking about Eucalyptus during a time where we are so close to the new release (Eucalyptus 3.0) so couple of things in the deck might need a modification or new addition though I had a slide on Eucalyptus 3.0 and I discussed with the audience about that.

I realized that talking about Eucalyptus can be a serious technical discussion and one needs to balance out the technical and the non-technical stuff well sometimes because the audience might not be too technical. I had a slide on the new Starter EMI stuff we have started doing, a slide on community and the various projects we have been hosting on our projects website.

My talk went well above the time allocated (around one and a half hour long) to me but I believe the audience liked it because I saw everyone seriously listening to me and I received few questions during my presentation as well as some after it ended. I am sorry I don’t have photographs right now because I was all busy talking to people but hopefully Atul and the gnuNify volunteers will be uploading/sharing the same pretty soon.

After the talk we distributed around 40 Eucalyptus t-shirts to the audience. It was really great. Everyone just jumped right on top of the t-shirts and grabbed one for themselves before we were out of t-shirts in few minutes time. I had some Euca goodies which we gave at the reception so people registering for gnuNify can get one for them. Lot of stickers and pens. Thanks to Greg and Diana for sending those to us!

After my talk , Atul presented on Juju at the same place but I was not able to attend his talk because I was busy answering questions and talking to people outside the auditorium. Apologies to Atul for stealing some of his audience I hope he don’t mind.

I met a team of college students who developed something on live VM migration and I shared with them my experience on VM migration in VMWare world and asked them to join our mailing lists and tell a little about themselves and their project. I met one other team who was planning to use Eucalyptus as a private cloud for a parallel processing project and suggested them few things again my general request to everyone was in the end join the mailing list and introduce themselves. I know we have a very active and vibrant community and its growing so it is helpful to get associated. I had one college faculty who wanted to know what he could do to try out Eucalyptus in his college LAB. He had a bad experience with Ubuntu Enterprise Cloud it seems , I requested him to join my workshop on setting up your own private cloud based on Eucalyptus and use CentOS with Eucalyptus 2.0.3. He did attended the workshop and I hope he liked it!

I was able to meetup with some more folks during lunch. I would like to take this opportunity and congratulate the organizers on getting us awesome food, whether it was breakfast, lunch, dinner or snacks it was really delicious. I met Ramakrishna Reddy from RedHat, Sudhanwa Jogalekar, Karunakar Guntupalli from Symantec and Harshad Oak from Indicthreads. It was a nice time talking to all of them and also Mr. Arun Khan from Silver Arc Solutions Pvt Ltd.

I also happen to meet Mr. Manish Kumar from CDAC who is into Cloud computing and building up a integration using various tools and openstack the platform.

The key take aways from Day 1 were,

• People need to know about Eucalyptus. Go and speak at such conferences and meetups. Socialize face to face.
• There should be a easy way out for them to learn and use Eucalyptus. Ideas ? Faststart, Silvereye can be the answer to the ease of installation.
• We are based on JAVA some people don’t like it. The primary reason they put is that its slow and they can’t understand the code.
• Localization efforts in Indic languages for Eucalyptus. We actually took this one pretty seriously and Atul had a lot of useful discussions and good feedback on this one, day 2 updates follows.

I was not able to work on the lab setup for my workshop but had one dedicated volunteer, a BBA (Bachelor of Business Administration) student, Chirag Jain who figured out along with the help and guidance of Mr. Harshad Gune what we would do on day 2.

The day ended up with a awesome drinks + dinner where I was able to catch up with lot of folks and socialize a bit more. It was great meeting lot of people who are active for a long time in FOSS world and it was also nice seeing the active volunteers. I should say I saw more volunteers than attendee’s but everyone was so active and seems excited about gnuNify!

Would like to thank Harshad Gune, Sudhanwa, Atul, Greg in getting me this opportunity to speak and interact with some real good folks , follow up post on Day 2 is next.

Well for those who are new to Eutester, it is a framework written in python which helps you test your Eucalyptus private cloud setup. Remember, testing can sometime be a hard job so best to automate it as much as possible with all your test cases. Besides that there is a lot more to it. I guess using the existing code which is part of the test framework one can develop his/her own test cases for varied scenarios/bugs and contribute the same to the Eutester code on github.

For me personally it is a great tool to have in my ninja pockets, makes my job pretty easier. Imagine you have a bug on launchpad that you want to replicate in your environment? How long would it take to execute those Steps to follow to re-create the problem and now imagine one single program, a Eutester test case doing all that for you in one shot, that’s what I call smartness.

Before starting off with how the framework actually works and the various components of it (which I guess is pretty straight forward and easy to understand) I just thought of giving one of the test cases a try and hence I did a:

git clone https://github.com/eucalyptus/eutester.git

Note that in order to use Eutester you need to have certain pre-requisites installed which is understandable. Following is the list of pre-requisites:

a) boto>=2.1
b) paramiko (SSH2 module for python)
c) python
d) Linux?

Note that Eutester is written in python so it requires you to have python, I had python 2.6 which works well with boto 2.1 and you also need python-setuptools.

Guess where I figured all this? Well just open setup.py in the Eutester code base and you will figure that out.

There are certain other assumptions made before hand regarding the cloud environment to be tested

a) The eucalyptus cloud already setup

b) Access to CLC via SSH

c) At-least one EMI registered

d) One SSH key pair available and added to the private cloud. Not a hard stop because Eutester can create the keypair and use that same with the instances it fires up.

e) Enough resources to start an instance on the cloud

f) If Linux instance needs to be accessed on SSH appropriate rules for the access via security groups

Once all the assumptions and pre-requisites are cleared out, next was a simple command to set Eutester on my machine:

python26 setup.py install

Well before we jump into executing our first sample test case let me point out here that Eutester provides various ways to you to feed in details on how Eutester can understand your cloud environment. Following are some of those ways (please note that there might exists other ways to get around but this post is to keep matters simple):

a) Using a cloud.conf config file

It is basically a simple text files having rows (lines) and columns. If you see the above you would find 2 lines describing the two machines in my cloud environment.

The sample cloud.conf is shown below

172.22.0.15 CENTOS 5.7 64 REPO [CC00 CLC SC00 WS]
172.22.0.16 CENTOS 5.7 64 REPO [NC00]

The column information is described in README.md file that came alone with the eutester code. But for the sake of completion of the post:

1. IP or hostname of the machine

2. Distribution installed on the machine

3. Distribution version on the machine

4. Distribution base architecture

5. System built from packages (REPO) or source (BZR) (packages assumes path to eucalyptus is /, bzr assumes path to eucalyptus is /opt/eucalyptus)

6. List of components installed on the machine

CLC – cloud controller
WS – Walrus
now that as eucalyptus can have a multi-cluster setup hence the following (specific to cluster)
SC00 – storage controller for cluster 00
CC00 – cluster controller for cluster 00
NC00 – node controller for cluster 00

b) Passing on the eucalyptus credential path, to the constructor. In my case the credentials were on the same machine as eutester so I had to only provide the path to the credentials.

Note that based on option (b) I wrote a very simple test case which runs an instance and then prints in some instance specific information.

from eucaops import Eucaops
if __name__ == '__main__':
    tester = Eucaops(credpath="/home/jeevanullas/.euca/")
    tester.exit_on_fail = 1
    image = tester.get_emi()
    reservation = tester.run_instance(image,keypair="sshlogin",group="linuxvms",type="m1.small")
    instance = reservation.instances[0]
    print "Instance Public IP address: " + instance.ip_address
    print "Instance Private IP address: " + instance.private_ip_address
    instance.terminate()

So what this test does (at a very high level) is that it connects to the CLC and then get one of the available EMI’s which is available and boots that up. Once the instance is up and running we print some basic information about the instance like the instance private ip address as well as public ip address. Finally we terminate the instance.

There are other things which can be done before we terminate the instance like executing a remote command on the instance and using the output internally in your test case? Lots of ideas.

The output for the script above is shown in the screenshot below

You can probably ignore some of the things like the boto debug stuff which is due to extensive logging enabled.

So if we summarize this post with our simple scripts here are the basic things which are pretty straight forward and Eutester takes care of

a) Takes care of getting your cloud credentials so that it can talk to your cloud and do stuff.

b) Next in order to run an instance on the cloud you need to provide EMI which Eutester gets for you based on the first available EMI and expects you to pass parameters like key pair to use for ssh logins and security group. Please note that these parameters are optional and are only required probably for a successful SSH connection to be established, infact SSH keypairs can be generated by Eutester before the instance is fired and can same can be used to while starting the instance as well as for further SSH connections inside Eutester.

Eutester is growing pretty fast thanks to Vic and Matt. Also Graziano already came up with an Eutester test case which is now part of Eutester and recreates bug 737335

There are various possibilities and lot of cool stuff happening around Eutester project. So I would suggest everyone to stay in touch and follow the project on:

Code
IRC Meetings
Projects Page

This post basically summarizes 2011 for me and ends with the latest news which comes with the new year. For a better understanding of things I am categorizing it into sections

Travel
I was able to visit some pretty nice places during vacation, some of them which I would like to mention here are Mekedaatu, Pondicherry and some of the worst places like Hogenakkal. Compared to 2010 though 2011 was pretty dull, hope to improve that in 2012.

Work at Infosys
Lot of cool and exciting things happened last year at Infosys which I might not be able to tell here. I worked with some of “the” awesome people inside the organization on things I love to work on (Cloud Computing, Linux, Sysadmin etc).
The journey at Infosys for me finally came to an end on 12/26/2011. It was a joy ride full of good as well as bad experiences but overall speaking at the end of the day lot of learning. Planning on to be in touch with some of the real gems still there, all of you (you know who you are) guys you rock and I wish you all a very happy new year!

Family
No new additions to the family which is pretty cool. Keeps thing simple. Brother got placed at Salesforce and is now based out of Hyderabad. The house in Jaipur got renovated again this time it was turn of the ground floor. My visits to home (Jaipur) and Jodhpur (Birthplace) were really nice spent a healthy time with my cousins and relatives which I would consider real good.

Bangalore
Bangalore as usual was great in 2011 though technically speaking I lived outside Bangalore for most part of it. I was not able to catch up with the Linux-India folks most of the time (though was able to show up on few rare occasions and catch up), I really would like to change that in 2012 once I am back in town later this year.

Technology
Got real hands-on experience with exciting stuff both in open source (Cloudstack, KVM, Spacewalk) and non-open source (HP Cloudsystem matrix, IBM Service Delivery Manager, VMWare vsphere, Amazon web services) domain. Compared to 2010 it was much more action packed year and cloud really grew pretty fast.
Apple as well as OS X continues to dominate my personal computing needs and I predict the same would get stronger and deeper in 2012.
I was not of able to contribute to the community, I have been part of, specially Fedora Project , Linux User Group Jaipur and Eucalyptus (sort off) but was able to change couple of things around that too by the end of the year at-least on the Eucalyptus front lot of cool stuff started happening.

Future and 2012
The big news which I guess most of the people (in my circles) have been waiting to hear on, where do I move on to now? Now that I am out of Infosys and Bangalore (for a while). Well guys I am glad to share with you the news that I have joined Eucalyptus Inc. and the great army of fellow Eucalyptoids spread across the world. As part of my job I would be working within the Technical services team focused on Customer support and also be the community guy for Eucalyptus, which in my opinion is going to get strong this year and the years to come.
I have already started off with my journey at Eucalyptus and the 1st week itself has been so far full of exciting stuff focused around open source, Eucalyptus community and Eucalyptus 3.

In case you are interested to get associated with the Eucalyptus community and exciting things which we have been doing for a while now, feel free to get in touch and be part of the ecosystem. For IRC users we (jeevan_ullas and others) can be found at #eucalyptus on irc.freenode.net.

So on a positive note let me wish you all a very happy new year full of Clouds powered by Open Source and please do stay in touch!

I am trying to connect my Linux machine to Amazon VPC using end to end IPSec tunnel. I have set all the required VPC objects on Amazon side and now plan to set my Linux Laptop as a VPN gateway. But the only doubt I have is that my Laptop is behind NAT. Though I have opened and re-directed the necessary ports on my NAT device I am not sure if this thing is going to work.

Please let me know if this setup can work. I am trying to follow the following guide

http://openfoo.org/blog/amazon_vpc_with_linux.html

From what I understand so far in order to make this guide work for my setup I need to do some extra configuration. I have also found out that IPSec supports tunnels behind NAT devices but I am not sure if Amazon VPC will support such configuration.

Any help in this matter is highly appreciated.

First thing is that we need to run boxgrinder on CentOS if we would like to build a CentOS AMI and on Fedora if we would like to build a Fedora AMI. The good thing about boxgrinder is that it uses the latest pvgrub kernel images provided by Amazon which basically lets you boot into your own kernel. So gone are the days when we had to use Amazon EC2 kernel. Thanks to Marek Goldmann for making this possible in boxgrinder 0.5

Without wasting much time let’s get started. I am going to build a Fedora 13 AMI for EC2 in this article but you can do the same stuff using CentOS. First of all we will have to install some required packages which are basically the dependencies.

# yum -y install git parted wget rpmdevtools appliance-tools \
sudo libguestfs ruby  rubygems ruby-libguestfs guestfish \
 yum-utils e2fsprogs

Next we need to install the EC2 AMI tools

#  rpm -Uvh http://s3.amazonaws.com/ec2-downloads/ec2-ami-tools.noarch.rpm

Then we need to install couple of gems for boxgrinder which will let us build a AMI for EC2. The following are the gems

# gem install boxgrinder-build
# gem install boxgrinder-build-fedora-os-plugin
# gem install boxgrinder-build-ec2-platform-plugin
# gem install boxgrinder-build-s3-delivery-plugin

Now we need to create a appliance definition file which will be used to build our AMI. This file is basically written in YAML format. The following is the file which I used:

name: Fedora13EC2
summary: My Fedora on EC2
os:
 name: fedora
 version: 13
hardware:
 partitions:
   "/":
     size: 2
packages:
 includes:
   - bash
   - kernel-PAE
   - grub
   - e2fsprogs
   - passwd
   - policycoreutils
   - chkconfig
   - rootfiles
   - yum
   - vim-minimal
   - acpid
   - dhclient
   - iputils
   - openssh-server
   - openssh-clients
   - httpd
   - system-config-firewall-base

Save this file in a directory appliances/ with the name Fedora13EC2.appl. Now we need to create a file which will store our AWS credentials and other important paths to certificate and private key. These are required by boxgrinder to put the image in S3 and register it with EC2.

# vi $HOME/.boxgrinder/plugins/s3
access_key: yourawsaccesskey                        # required
secret_access_key: youawssecretkey             # required
bucket: myownfedora-box                               # required
account_number: youramazonaccountnumber  # required
path: /mnt/images                                 # default: /
cert_file: /root/.ec2/yourcertificate.pem   # required only for ami type
key_file: /root/.ec2/yourprivatekey.pem  # required only for ami type

We need to create the directory /mnt/images which will store the AMI.

# mkdir /mnt/images

Finally we can fire boxgrinder to build the AMI.

# boxgrinder-build appliances/Fedora13EC2.appl -p ec2 -d ami

It will run for few minutes and will end up with something like:

I, [2010-08-07T06:21:58.693095 #17381] INFO — : Image successfully registered under id: ami-

That’s it. So simple right? Now you run this AMI and enjoy !

I would like to encourage people to look into this project. This is really awesome. For more information you can go through the following links
http://community.jboss.org/wiki/BoxGrinderBuildPluginsDeliveryS3
http://community.jboss.org/wiki/BoxGrinderApplianceDefinitionFiles
http://community.jboss.org/docs/DOC-14384
http://cloudpress.org/2010/07/21/boxgrinder-build-0-5-0-release-with-fedora-13-on-ec2-support-and-stormfolio-update/
http://cloudpress.org/2010/06/24/judcon-2010-boston-slides/

In my next post I will cover my experience with passing user data scripts to EC2 AMI while starting the instance and much more to come. Happy hacking

I would like to thank Phil Chen for his excellent post here http://www.philchen.com/2009/02/14/how-to-create-an-amazon-elastic-compute-cloud-ec2-machine-image-ami which I followed and have mentioned below with some extra addition and some modification.

The steps I mention in this article are part of the manual process. The next part to this article will discuss a great tool call boxgrinder which can let you automate this process in a very easy way. I personally use boxgrinder on my machine to deploy brand new AMIs on Amazon EC2.

The first step of course to this tutorial is to have a account on Amazon EC2. It’s pretty easy to get one if you have a credit card . Luckily I was provided a already existing account by my Manager. After you have the account on Amazon EC2 you need to login to the EC2 web console and get few things for yourself like:

• S3 ACCESS KEY
• S3 SECRET KEY
• AWS ACCOUNT NUMBER
• EC2 PRIVATE KEY
• EC2 CERTIFICATE

In order to get these click on Account weblink inside the Amazon web console and then on Security Credentials. Once you have get the required data we need to install two packages provided by Amazon on our local system. The tools can work on Mac OSX and Linux, I am not sure about Windows.

I do have a local Linux machine but if I create the AMI on my local machine it will take lot of time to upload it to Amazon (due to slow internet connection). So instead of that I started a instance on EC2 for developing AMI. The major advantage is that it takes only few minutes to upload from their as the download/upload speed provided by Amazon is fantastic.

To start a instance you can use the web console provided by Amazon. I wanted to build a AMI for Centos 5 so I started the 32bit image for Centos 5.4 provided by RightScale free of cost available on EBS (Elastic Block Storage).

The instance will startup in few seconds and you will get a public DNS name for it. You can use one of your security keypair you associated when creating the instance to login as root to the instance.

Once you have logged in you need to set this instance as the development bed for creating AMIs. Download these 2 files:

• http://s3.amazonaws.com/ec2-downloads/ec2-ami-tools.zip
• http://s3.amazonaws.com/ec2-downloads/ec2-api-tools.zip

Extract these 2 zip files inside your home directory for e.g /root or wherever you would like to extract them. Next you need to copy your private key and certificate for the EC2 account inside this running instance. I created a directory ~/.ec2/ and copied them there.

Next we need to setup the environment for our configuration. The following is what I wrote inside my ~/.bashrc

export PATH=$PATH:/root/ec2amitools/bin/:/root/ec2apitools/bin/
export EC2_HOME=/root/ec2apitools/
export EC2_AMITOOL_HOME=/root/ec2amitools/
export EC2_PRIVATE_KEY=~/.ec2/pk-yourprivatekey.pem
export EC2_CERT=~/.ec2/cert-yourcertificate.pem

Note that I extracted the AMI tools inside /root/ec2amitools and API tools inside /root/ec2apitools/ hence those paths in EC2_HOME and PATH.

In order to reflect the changes you need to source the bash profile.

source ~/.bashrc

Now it’s time to create a image file inside the running instance which will basically hold our operating system for the new AMI.

dd if=/dev/zero of=/mnt/ami-centos5.5-32bit-base.img bs=1M count=10240

Next we need to create a filesystem on this image.

/sbin/mke2fs -F -j /mnt/ami-centos5.5-32bit-base.img

We need to now mount this image on some directory. I created a directory /mnt/centos and mounted this image as loopback.

Next we need to create some basic directories inside /mnt/centos/ which are required.

mkdir /mnt/centos/proc
mkdir /mnt/centos/etc
mkdir /mnt/centos/dev
mkdir /mnt/centos/var
mkdir /mnt/centos/var/cache
mkdir /mnt/centos/var/log
mkdir /mnt/centos/var/lock
mkdir /mnt/centos/var/lock/rpm

Also we need to create 3 basic devices inside /mnt/centos/dev

/sbin/MAKEDEV -d /mnt/centos/dev -x console
/sbin/MAKEDEV -d /mnt/centos/dev -x null
/sbin/MAKEDEV -d /mnt/centos/dev -x zero

We also need to create /mnt/etc/fstab file for our image and populate it with the following information which is required for a 32bit image

/dev/sda1	/		ext3		defaults	1	1
none		/dev/pts	devpts	        gid=5,mode=620	0	0
none		/dev/shm	tmpfs	        defaults	0	0
none		/proc		proc		defaults	0	0
none		/sys		sysfs		defaults	0	0
/dev/sda2	/mnt		ext3		defaults	0	0
/dev/sda3	swap		swap		defaults	0	0

Now we need to mount the proc filesystem for our new image.

mount -t proc none /mnt/centos/proc

We will be required to install the base operating system inside this image file. Yum configuration should be created and saved in a file. I saved it in /mnt/yum-ami.conf

[main]
cachedir=/var/cache/yum
keepcache=1
debuglevel=2
logfile=/var/log/yum.log
pkgpolicy=newest
distroverpkg=redhat-release
tolerant=1
exactarch=1
obsoletes=1
gpgcheck=1
plugins=1
metadata_expire=1800

[base]
name=CentOS-5 - Base
mirrorlist=http://mirrorlist.centos.org/?release=5&arch=i386&repo=os
#baseurl=http://mirror.centos.org/centos/5/os/i386/
gpgcheck=1
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-5

#released updates
[updates]
name=CentOS-5 - Updates
mirrorlist=http://mirrorlist.centos.org/?release=5&arch=i386&repo=updates
#baseurl=http://mirror.centos.org/centos/5/updates/i386/
gpgcheck=1
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-5

#packages used/produced in the build but not released
[addons]
name=CentOS-5 - Addons
mirrorlist=http://mirrorlist.centos.org/?release=5&arch=i386&repo=addons
#baseurl=http://mirror.centos.org/centos/5/addons/i386/
gpgcheck=1
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-5

#additional packages that may be useful
[extras]
name=CentOS-5 - Extras
mirrorlist=http://mirrorlist.centos.org/?release=5&arch=i386&repo=extras
#baseurl=http://mirror.centos.org/centos/5/extras/i386/
gpgcheck=1
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-5

#additional packages that extend functionality of existing packages
[centosplus]
name=CentOS-5 - Plus
mirrorlist=http://mirrorlist.centos.org/?release=5&arch=i386&repo=centosplus
#baseurl=http://mirror.centos.org/centos/5/centosplus/i386/
gpgcheck=1
enabled=0
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-5

#contrib - packages by Centos Users
[contrib]
name=CentOS-5 - Contrib
mirrorlist=http://mirrorlist.centos.org/?release=5&arch=i386&repo=contrib
#baseurl=http://mirror.centos.org/centos/5/contrib/i386/
gpgcheck=1
enabled=0
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-5

Now it’s time to install the base:

yum -c /mnt/yum-ami.conf --installroot=/mnt/centos -y groupinstall Base

This will install around 333 packages. After this is done we will install openssh packages inside the image because using openssh server only we will accessing the instance remotely

yum -c /mnt/yum-ami.conf --installroot=/mnt/centos/ -y install *openssh*

Make sure you put sshd on chkconfig so that it starts automatically when the instance runs:

/usr/sbin/chroot /mnt/centos/ /sbin/chkconfig sshd --add
/usr/sbin/chroot /mnt/centos/ /sbin/chkconfig --level 12345 sshd on

After this is done we need to configure our image for Amazon. First thing is to put kernel modules for Amazon inside the image. This is required in order to make this image run on EC2.

cp -r /lib/modules/2.6.21.7-2.fc8xen/ /mnt/centos/lib/modules/
/usr/sbin/chroot /mnt/centos/ /sbin/depmod -ae 2.6.21.7-2.fc8xen

One of the service which is set on in a default base install is kudzu (the hardware detecting service). We need not start it at all on EC2

/usr/sbin/chroot /mnt/centos /sbin/chkconfig --del kudzu

Next we need to configure openssh server configuration for EC2

vi /mnt/centos/etc/ssh/sshd_config

The two lines which we should make changes are:

UseDNS no
PermitRootLogin without-password

Next we need to setup networking inside this image. The following needs to be done for the same:

vi /mnt/centos/etc/sysconfig/network

NETWORKING=yes
HOSTNAME=localhost.localdomain

vi /mnt/centos/etc/sysconfig/network-scripts/ifcfg-eth0

ONBOOT=yes
DEVICE=eth0
BOOTPROTO=dhcp

When a instance boots in EC2 it needs to get your SSH keys in order to allow sshkey based authentication. We will create a init script for the same in /mnt/centos/etc/init.d/ with the name getssh

#!/bin/bash
# chkconfig: 2345 95 20
# description: getssh
# processname: getssh
#
export PATH=:/usr/local/bin:/usr/local/sbin:/usr/bin:/usr/sbin:/bin:/sbin
# Source function library.
. /etc/rc.d/init.d/functions

# Source networking configuration.
[ -r /etc/sysconfig/network ] && . /etc/sysconfig/network

# Check that networking is up.
[ "${NETWORKING}" = "no" ] && exit 1

start() {
  if [ ! -d /root/.ssh ] ; then
          mkdir -p /root/.ssh
          chmod 700 /root/.ssh
  fi
  # Fetch public key using HTTP
/usr/bin/curl -f \
 http://169.254.169.254/latest/meta-data/public-keys/0/openssh-key \
  > /tmp/my-key
  if [ $? -eq 0 ] ; then
          cat /tmp/my-key >> /root/.ssh/authorized_keys
          chmod 600 /root/.ssh/authorized_keys
          rm /tmp/my-key
  fi
  # or fetch public key using the file in the ephemeral store:
  if [ -e /mnt/openssh_id.pub ] ; then
          cat /mnt/openssh_id.pub >> /root/.ssh/authorized_keys
          chmod 600 /root/.ssh/authorized_keys
  fi
}

stop() {
  echo "Nothing to do here"
}

restart() {
  stop
  start
}

# See how we were called.
case "$1" in
  start)
    start
    ;;
  stop)
    stop
    ;;
  restart)
    restart
    ;;
  *)
    echo $"Usage: $0 {start|stop}"
    exit 1
esac

exit $?
###END OF SCRIPT

We will add this script to chkconfig and make sure it is executed everytime our instance starts

/bin/chmod +x /mnt/centos/etc/init.d/getssh
/usr/sbin/chroot /mnt/centos/ /sbin/chkconfig --level 34 getssh on

This makes our image ready for EC2. We need to do cleanup now so the image size is reduced:

yum -c /mnt/yum-ami.conf --installroot=/mnt/centos clean all
sync
umount /mnt/centos/proc
umount /mnt/centos

Now its time to bundle the image for EC2.

ec2-bundle-image --image /mnt/ami-centos5.5-32bit-base.img --prefix \
 ami-centos5.5-32bit-base --cert ~/.ec2/cert-yourcertificate.pem \
 --privatekey ~/.ec2/pk-yourprivatekey.pem \
 --user youraccountnumber --destination /mnt/centos/ --arch i386

After the bundle is over we need to upload it to S3 using the following command:

ec2-upload-bundle --manifest /mnt/centos/ami-centos5.5-32bit-base.manifest.xml \
 --bucket centos55-cpr-base  \
--access-key youraccesskey \
 --secret-key yoursecretkey \
 --location US

Finally we register this image with EC2 so we can use it to boot a instance

ec2-register -n centos55cpr  \
 centos55-cpr-base/ami-centos5.5-32bit-base.manifest.xml

Our image is ready to boot. We can now run the instance using this image via following command:

ec2-run-instances youramid \
-k yoursshkey  \
--kernel aki-a71cf9ce \
--ramdisk ari-a51cf9cc \
--instance-type m1.small \
--region us-east-1

It takes sometime for the instance to startup (as the image is in S3). To find what public DNS name this instance got we can issue the following command

ec2-describe-instances

After the instance has booted we will login into it using the sshkey we specified while starting the instance. Note that this instance we will be stored in S3 and will act as base for other images which we might plan to create in future. For e.g we have a requirement to have a Apache webserver image based on Centos 5.5 32bit we can utilize this base image and create a EBS volume for the new Apache image. Let’s see how to do that after the base image has booted.

We need to create a EBS volume of size at least 10GB.

ec2-create-volume -s 12 -z us-east-1d

Now we need to wait for few minutes till the volume is created. As soon as the volume creation is finished we attach it to our already running instance (image is in S3).

ec2-attach-volume vol-ea92ef83 -i i-b6dd60dc -d /dev/sdb

The volume id vol-ea92ef83 you can find out from ec2-describe-volumes and the instance id i-b6dd60dc you can find out from ec2-describe-instances.

Now we need to login to our instance and copy the whole root filesystem which is in /dev/sda1 to /dev/sdb using dd command.

dd bs=65536 if=/dev/sda1 of=/dev/sdb

Filesystem check on the EBS volume should be done now:

fsck /dev/sdb

Now we can mount the EBS volume and make any necessary changes we would like to make. For e.g we would like to make a HTTPD server EBS volume we simply follow these steps:

mkdir /mnt/ebs
mount /dev/sdb /mnt/ebs
mount -t proc none /mnt/ebs/proc
yum -c /mnt/yum-ami.conf --installroot=/mnt/ebs -y install httpd
/usr/sbin/chroot /mnt/ebs chkconfig httpd --add
/usr/sbin/chroot /mnt/ebs chkconfig --level 234 httpd on
umount /mnt/ebs/proc
umount /mnt/ebs

Finally we will have to create a snapshot of the EBS volume:

ec2-create-snapshot -d centos55-apache vol-ea92ef83

Now we can build a AMI out of this snapshot which will be stored on EBS volume using the following command

ec2-register -n centos5.5-32bit-httpd \
 --architecture i386 \
 --block-device-mapping /dev/sda1=snap-a06dbfcb:12:false

You can find out the snapshot id snap-a06dbfcb from ec2-describe-snapshots. The command above will finally generate the AMI id which we can use later to boot our instance out of EBS volume.

ec2-run-instances -k yoursshkeypair \
 ami-936289fa \
 --region us-east-1 \
--instance-type m1.small

I hope this article will help all the EC2 users out there who plans to roll their own base AMI and then custom software stack specific EBS volumes. In the next article I plan to cover boxgrinder a great tool which helps you do the whole process mentioned above in 1 single command. So stay tuned!

I personally never expected many folks to turn up because of two reasons. One been that many colleges in Bangalore were having end semester exams plus we don’t have much Fedora folks here in Bangalore. Initially we (Me, Ankur, Hiemanshu , Dipjyoti and Rangeen) had a hard time finding out the office. Point to remember GPS in India will not be accurate enough with all these narrow lanes everywhere.

Finally we found the place and till then FSMK folks also started turning up. I would like to specially mention Vignesh, Prabodh and Naveen from FSMK core team who turned up for the party and assisted with the logistics. We had few other people some friends of Rangeen , students of Prabodh’s previous college CMRIT and other FSMK members.

We started with the discussion about why people should use Linux and not Windows. I never expected to start with such discussion in a Fedora release party but I love to be part of such discussion. It was great. Many questions were thrown on us. Why Linux? Linux? Is it user friendly enough? Why would a normal user plan to switch to Linux? People there who were using Fedora for past 4 years had such questions. I respect all the points given by them.

We started answering one by one. Inputs were given by Naveen on how the current scenario is changing and government bodies across India are encouraging FOSS in schools and engineering colleges. The current projects by Knowledge Commons and MHRD were been discussed with all the participants.

Me and Rangeen told the participants about how things (hardware) works out of the box these days with Fedora. How easily if you plug a mobile broadband card you can be sure that it will work without any hassle (all thanks to the great NetworkManager). We discussed why latest hardware sometime doesn’t work on most of the Linux distributions and how things are now changing. Times have changed and we should look towards improving things instead of blaming about them (my personal opinion).

Naveen introduced us to the FSMK activities currently going on. I personally got to know lot from that and looking forward to contribute to the cause FSMK is supporting.

While the hot discussion on various topics related to Linux/Windows was going on we ordered pizza’s for all of us. I would like to mention here how happy was the pizza girl over phone that she told me to not pick her phone after few minutes. She loved my Kannada ring tone

Once the dust of doubts and questions settled down Ankur started giving the Fedora 13 DVD/Live CD media to all the members. We also copied ISO for others who brought USB stick. I personally got DVD 32bit ISO from Ankur. Thanks a lot Ankur because I am writing this blog post from a Fedora 13 loaded laptop 

After that we briefly discussed the new features in Fedora 13 among the participants and touched few special points from the release notes handouts given to them by Ankur and Rangeen

Pizzas had arrived by that time and we ate them all!! It was great having pizza and garlic bread-sticks with jalapeño. Finally we all exchanged our phone numbers and promised to stay in touch

Overall turn out was around 10 + we four Fedora folks (Me, Rangeen, Hiemanshu and Ankur). That was a good figure. We have now aligned ourself with the FSMK and we plan to conduct some good workshops as soon as the college re-opens in Bangalore. Fedora is going to spread everywhere in the air of Bangalore I am sure. We are looking for contributors and FSMK is also planning their strategy on the same grounds. I hope to see some good action coming in next few weeks.

Oh yes, I don’t want to miss this part. After the release party ended me and Hiemanshu planned to end up in pub (purple haze in Kormangla). Ankur and Rangeen didn’t supported us so we went there alone. Hiemanshu had I believe 2 different cocktails and me 1 pitcher of Foster’s

Drink beer use Fedora our motto!! Few pictures were shot by Me, Rangeen, Dipjyoti and Ankur. You can find them here:

http://picasaweb.google.com/dipjyoti.ghosh/Fedora13ReleaseParty#

http://picasaweb.google.com/sherry151/

http://picasaweb.google.com/sanjay.ankur/Fedora13ReleaseParty

I have been running my Eucalyptus private cloud on a single laptop till now and knew that I can’t use brother’s laptop as node because it has windows 7 and fedora 12 was running inside VM. So I started installing eucalyptus 1.6.2 on that VM via source and finished eucalyptus as well euca2ools installation. Created that VM as the head running Cloud controller, cluster controller, storage controller and walrus. Next I switched to my laptop from scratch installed eucalyptus and configured it to run as node. My laptop is super cool. It’s been 4 years I have had this machine but the good thing it has the special processor flags which gives VT support so that I can run fully virtualized VM in KVM (kvm_intel).

The next thing which started bothering me was I didn’t had much space on my laptop left to store the instance copies when it boots up and later runs. But I have a 500 GB hard drive which I can use for this purpose. Created a directory on one of the partitions in my external hard drive and inside my node’s eucalyptus.conf

INSTANCE_PATH="/media/f5a889e3-9300-4565-9cb3-9d14b79ad124/images/"

The other thing I changed in my eucalyptus.conf was the VNET_BRIDGE interface which by default is xenbr0. This works good for XEN but not for KVM. Changed it to following:

VNET_BRIDGE="virbr0"

Then I remember that console thing didn’t worked for me when I first tried eucalyptus on KVM+Fedora 12 sometime back and for that to work I was suggested a patch at the forums. Applied that patch on my /opt/eucalyptus/usr/share/eucalyptus/partition2disk and all was set.

The next big thing to work out was my networking. The two laptops could have been connected to each other via a ad-hoc network but I prefer a cable connection as it makes the image transfer faster. I have a cross cable with me so I used it to connect both the laptops. The cross cable connection was available inside my VMware VM via bridged connection. The only thing I did different was the gateway IP inside my head (brother’s laptop fedora 12 VM running inside VMware) I gave my laptop’s eth0 IP. This was done because my instance will get a IP on the subnet 192.168.122.0/255.255.255.0 and my network was on 192.168.1.0/255.255.255.0. Once I did that I started the instance and later when it booted was able to ssh to it with my private key (password less login).

The other problem I had was sharing internet connection at the same time. I use a reliance usb device for internet connectivity. I attached that to my laptop started the connection created a simple squid proxy configuration which allows 192.168.1.0/255.255.255.0 and configured the same proxy on the fedora 12 VM in my brother’s laptop. This made internet as well as yum (had to made a proxy entry in /etc/yum.conf) working on the VM which was required to install some RPMS from the repos.

Sounds cool, eh? Let’s see a picture of this setup.

In the picture above you can see my laptop on the left hand side connected to my external 500 GB maxtor hard drive, reliance USB device and my brother laptop on the right via cross cable wire.

Though the setup worked and I was able to boot instance and login to them I was not able to see my EBS volume inside the VM. I have updated my forum entry for the same and you can find more information about it here. Let me know if you have any suggestions for me.

Please note that this is a small private cloud facility running out of my home and its scalable so if you got a laptop or a machine which you would like to add to this cloud do let me know.

Signing off for now but the love for machines still not ends up going to office to play more with those.

I have seen lot of people blogging about eucalyptus specifically on Ubuntu Server edition. Well to be frank eucalyptus is a great software and it works with almost al latest linux distributions. Though I haven’t found time to test all the available linux distribution as I am stuck with work and Fedora but I have tested it on centos 5.4 and Fedora 12. Works great!

Few problems I have always faced but the IRC channel for eucalyptus on freenode as well as the online forums have been really helpful in solving my doubts.

I plan to write my experiences with eucalyptus on this blog in times to come. Besides eucalyptus creating virtual appliances in a automated way is also one of the areas I have worked on paste few weeks. This is all specific to fedora right now using boxgrinder. It is a alternative to vm-builder which ubuntu folks have got.

I am a strong support of the fedora project and love the way how the fedora community is structured and functions. Have been associated with it since the beginning (Fedora core 1).

In the end, for now, I would just like to say, cloud computing is the future and open source is the best medium we have all got to implement it.

1. If you are a hardcore non-vegetarian this is the place to be. This city has to offer variety of chicken and mutton and it is not costly either.

2. Briyani lovers, please go to paradise hotel. It’s like the makka madena of briyani. I have been there only once with my friends but would love to go again.

3. The place where I live is close to the famous hi-tech city. This hi-tech city is the area where you will see huge buildings some of them having nice arch. too. It is full of dynamic youth which are not at all driven by passion or enthusiasm but auto rickshaw drivers who run their auto’s in sharing mode. Even I am a regular customer of these sharing autos where you have to sit in front with the driver if there are ladies.

4. Description of this city will be in-complete without discussing my work place. The best place to be. It’s even better from my room because I don’t get power cuts there. It has a green landscape with small water falls, awesome food courts, big and nice buildings. We have in the employee care center everything. We got bank, gym, more store, indigo nation showroom, table tennis, carrum, snooker, dance class, tennis court, basketball ground, cricket ground, laundrament, apollo clinic and most important of all these if you would like to take rest or sleep we have got dormitory, separate for men and women. The dormitory is special cause it has got clean bed with air conditioned dark room where one can sleep for hours. Other things in the employee care center are strand book stall which has got some of the nice books for me to read at a discounted price.

What else? I am still exploring this city though not that much but as and when I get time. The places I have visited so far are near to my living place. I plan to go to the old city sometime and visit the original places where as I have heard there are still some nizams or at-least their property.

Right now I would like to advice you not to go to the old city just like that due to the telengana issue. It’s a big issue and sometimes results in curfew in the whole city. At that time it’s hard to find good food and easy way to commute. I have seen bike number plates having TG(telengana) instead of AP(Andhra pradesh)

I was able to write this post because of my E72 and pretty good airtel GPRS connection. Laptop battery died few minutes ago and still we don’t have lights here. What a life in the city they call as cyberabad.

I am glad to be part of you. Here I am, back, finally with my own space. I hope this journey is long and becomes learning experience for both of us. Looking forward to have a great time with you.

Thanks

~jeevanullas

Well at least I will try to be regular now on this blog and write few things which I have been engaged with these days.

So stay tune!

Cast your vote go to : https://admin.fedoraproject.org/voting

Cambridge was out yesterday night (IST).

And just when the download was about to finish for me I got my Directory Services exams result and I was told that I have passed the exam. w000t. That was the last hurdle in the way to become RHCSS.

Finally I can call myself Redhat Certified Security Specialist.

How awesome all this looks. Brand new fedora on my laptop and desktop machines and RHCSS!!!

For verficiation:-

http://www.redhat.com/training/certification/verify/?rhce_cert_display:certno=804006843818597&rhce_cert_display:verify_cb=Verify

Thanks for accepting me . Those who would like to know more about me just have a look here.

Hope to get some meaningful posts in the future.

Till then Enjoy!

The setup goes something like this:-

I have two RHEL5.2 machines one is station1 and the other is server1. The station1 machine sends the log for local6 facility of any type of priority to server1. But the log send over to server1 is going to be encrypted via stunnel package. Let’s see how:-

Setup at server1

This is going to be our central log server for local6 log facility. First of all we will install the rsyslog package which though comes with RHEL5.2 but is not the default:-

#yum install rsyslog
#service syslog stop
#yum remove sysklogd
#service rsyslog start
#chkconfig rsyslog on


Next we configure rsyslog such that it listens for connections on tcp/61514

#vi /etc/sysconfig/rsyslog


Edit it such that at line 6 it shows:-

SYSLOGD_OPTIONS="-m 0 -t 61514"


Now we need to add this port 61514/tcp to our semanage ports. This will be done via the following command:-

#semanage port -a -t syslogd_port_t -p tcp 61514


Later we can see if the above command have succesfully worked or not by issuing the following command:-

#semanage port -l | grep syslogd_port_t


The output of the above command will be something like this on a default installation of RHEL5.2

syslogd_port_t tcp 61514
syslogd_port_t udp 514


This tells that port 514/udp and port 61514/tcp are SELinux managed for the type syslogd_port_t. Okay that’s what we wanted. For securing the log server we want it to run on a tcp port and that’s why we did all this starting from editing /etc/sysconfig/rsyslog to semanage. Note that all our setups have SELinux in enforcing mode so it’s necessary that we take proper care of SELinux.

Next we restarted the rsyslog service.

#service rsyslog restart


Now we need to configure stunnel on server1 so that it accepts connections from the client on some fix port and forward them to port 61514/tcp running on server1. We will ensure via the iptables that the port 61514/tcp is not directly exposed to the network as well as port 514/udp.

#iptables -A MYCHAIN -p tcp --dport 60514 -j ACCEPT
#service iptables save


This rule opens up port 60514/tcp on server1. This will be the port where stunnel running on server1 will listen for client connections and later forward them to locally running rsyslog service at 61514/tcp.

The package for stunnel was installed default in a base installation of RHEL5.2 so that was not a big deal but if it’s not there in your setup ensure that you have stunnel installed.

After the installation is done we need to configure stunnel. The configuration directory for stunnel is empty but it’s package provide one sample conf file which can be used. To use the provided conf sample just follow the below commands:-

#cd /etc/stunnel
#cp /usr/share/doc/stunnel-4.15/stunnel.conf-sample stunnel.conf


Next we edited the stunnel.conf file according to our requirements and when we completely edited it that’s how it looked:-


; Certificate/key is needed in server mode and optional in client mode
cert = /etc/stunnel/stunnel.pem
key = /etc/stunnel/stunnel.key

; Service-level configuration
[ssyslog]
accept = 60514
connect = 61514


The section [ssyslog] specify which port stunnel will listen to and then which port it will forward the connection too. The destination port is of the local interface (127.0.0.1) as far as I know, haven’t digged much into it so I am not sure. Please feel free to comment on it.

There are number of other variables on top that configures alot of stuff. First is the filepath for the stunnel security certificate, then is the filepath for the stunnel security certificate key, next comes the directory under which stunnel will run (this makes stunnel run in a chroot jail, that’s good for security reason but it’s only available on windows host), after that the user and group with which the application will run and the pid file name and path for stunnel it actually is /stunnel.pid but that’s relative to /var/run/stunnel now, after that we had some performance tuning options which actually came enabled default in the sample conf file so I thought of keeping them up, after that verify=2 is used to verify the other end of the tunnel, the verification is done by checking the security certificate of the other end of the tunnel upto depth level 2 so that checks whether the security certificate of the other end (the client end, in our case station1) is actually signed by the same Certificate Authority (CA) as the one specified by the next option that is CAfile.

Now we need to create the directory in which stunnel will store it’s pid file and will also run in chrooted jail provided by that directory. The group/owner permission of that directory are also important (as specified in stunnel.conf):-

#mkdir /var/run/stunnel
#chown nobody:nobody /var/run/stunnel


Now we need to work on the security certificate stuff. Stunnel uses both self-signed or third party signed certificates. We went with the trusted third party signed certificate. For this we already had a private Certificate Authority running in our network which was used to sign/revoke security certificates of clients in the network.

So first of all we created the key to be used for the certificate and then we generated a certificate signing request for the stunnel certificate and later send that to the certificate authority to sign and return back to us. The certificate authority also sent us a copy of there own certificate which was also kept in /etc/stunnel for configuration purposes. The following command helped in the above task:-

#cd /etc/stunnel
#openssl genrsa -out stunnel.key 2048
#openssl req -new -key stunnel.key -out stunnel.csr
#scp stunnel.csr root@certificate.example.com:/etc/pki/CA


At certificate.example.com we issued the following commands:-

#cd /etc/pki/CA
#openssl ca -in stunnel.csr -out stunnel.pem
#scp stunnel.pem cacert.pem root@server1.example.com:/etc/stunnel/
#rm -f stunnel.*


Note that after we have recieved the signed certificate and CA certificate the first thing we did was secure those by strictly changing there file permissions as shown below:-

#chown root:root /etc/stunnel/*
#chmod 600 /etc/stunnel/*


That was sufficient. Well if you are not running a local CA I would suggest you do run it or have a commerical 3rd party trusted authority sign your certificate. For a small setup self sign certificate will do the job so no need for Certificate author
ity. Also note that the step I mentioned above are completely custom as I want them to be it might be that your setup is different then you have to use different commands and options.

That’s all about stunnel on the server side. Now was the time to start the tunnel, so that’s done just by running the command stunnel.

#stunnel
#ps aux | grep stunnel


The first command runs the tunnel and the next command is given to make sure if stunnel is running in the background successfully or not. The output should be something like this

nobody 4476 0.0 0.3 5060 984 ? Ss 16:46 0:00 stunnel


If you want to make sure that stunnel runs automatically on every boot up just put these lines in /etc/rc.d/rc.local of your system (at the bottom):-

/usr/sbin/stunnel


That’s it the job at server1 is done and now it’s time to proceed at the client side.

Setup at station1

First the same steps as performed on server1 installing rsyslog package and removing the stock sysklogd package via the following commands:-

#yum install rsyslog
#service syslog stop
#yum remove sysklogd
#service rsyslog start
#chkconfig rsyslog on


Now we need to configure the stunnel package on the client side too. As mentioned earlier stunnel comes with the default installation of RHEL5.2 but if it’s not installed just make sure you have it installed. Stunnel actually is part of official RHEL5.2 distribution. Next as done earlier copy the sample configuration file provided by the stunnel package to the stunnel configuration directory.

#cd /etc/stunnel
#cp /usr/share/doc/stunnel-4.15/stunnel.conf-sample stunnel.conf


Edit the file such that it looks as shown below:-

; Certificate/key is needed in server mode and optional in client mode
cert = /etc/stunnel/stunnel.pem
key = /etc/stunnel/stunnel.key

; Service-level configuration
[ssyslog]
accept = 127.0.0.1:61514
connect = 192.168.122.2:60514


The major difference between the stunnel.conf of station1 and server1 is that the stunnel.conf of station1 contains a variable client = yes that differentiates server end and client end in a stunnel.

First we will create the chroot directory in which stunnel will run. As done in the configuring the server section above:-

#mkdir /var/run/stunnel
#chown nobody:nobody /var/run/stunnel


Now it’s time to make the security certificate for this end of the tunnel. We will proceed in the same way as we did while setting up the server end. First we will generate a 2048 bit key. One particular thing which I forgot to mention about this key is that it’s not a password protected key. If it’s compromised that end of the tunnel is compromised. We could have protected the key with a password by specifying option like -des3 to the genrsa command but then we would have to give the password for the key when we ran stunnel that asks alot of overhead when we say our tunnel will automatically start on boot. In that case we have to manually feed in the password for the tunnel to get started.

#cd /etc/stunnel
#openssl genrsa -out stunnel.key 2048
#openssl req -in -key stunnel.key -out stunnel.csr
#scp stunnel.csr root@certificate.example.com:/etc/pki/CA


At certificate.example.com the following commands were issued:-

#cd /etc/pki/CA
#openssl ca -in stunnel.csr -out stunnel.pem
#scp stunnel.pem cacert.pem root@station1.example.com:/etc/stunnel
#rm -f stunnel.*


Now as we did during setting up the server end we secure the configuration file and the certificate files at the client end by modifying the file permissions accordingly:-

#chown root:root /etc/stunnel/*
#chmod 600 /etc/stunnel/*


Now we can start the stunnel at the client end too via the simple command stunnel. If we want to start the tunnel automatically on every boot up it’s simple just add the line /usr/sbin/stunnel in /etc/rc.d/rc.local at the end. To verify that stunnel is running properly or not just issue the old command ps aux | grep stunnel and see if there is any process owned by user nobody with the name stunnel.

Now we will configure the rsyslog service at the client so that it re-directs all the logs for local6 facility to 127.0.0.1:61514 where stunnel will read them up and send them to 192.168.122.2:60514. Note here that 192.168.122.2 is actually server1 but instead of specifying the name I preferred IP address as DNS can be un-available in my setup.

The below line I added in /etc/rsyslog.conf:-

local6.* @@127.0.0.1:61514


Save and exit and then restart rsyslog:-

#service rsyslog restart


Now to test the setup we issued the following command at station1 and while that was running we sniff the packet via wireshark available in RHEL5.2 to intercept what was getting transferred between the two tunnel ends:-

logger -i -p local6.info -t deependra "This is a test log message sent over stunnel"


The output was clearly seen at server1 in /var/log/messages as

Jul 28 18:24:30 station1 deependra[3460]: This is a test log message sent over stunnel


While the communication was happening between the two ends of the tunnel I sniffed the packets transferred between the two ends and it was all encrypted from what I saw.

That’s how I was able to secure my log server communication with clients. There are much better and inbuilt ways to provide security of log server which comes with rsyslog that you can check out at rsyslog website. ]]> http://jeevanullas.in/blog/2008/07/securing-log-server-in-rhel/feed/ 0 http://jeevanullas.in/blog/2008/07/ntp-server/?utm_source=rss&utm_medium=rss&utm_campaign=ntp-server http://jeevanullas.in/blog/2008/07/ntp-server/#comments Mon, 28 Jul 2008 06:47:00 +0000 jeevanullas http://jeevanullas.in/blog/2008/07/27/ntp-server/ It’s been long since I am using NTP server in my installations here. So I thought to document my setup a bit in order to explain myself what’s going on and to help others world wide so that they can also have a secure setup for Time server in Linux.

Time server is a important part of a network as everybody might be knowing. It is a must if we want to have a network setup which will later consists of kerberos or DNSSEC. It is also needed in windows environment but the configuration for that need not be done in the default case.

I have my test server running latest updated version of Fedora 9. First of all I make sure that my setup have the ntp package. Actually ntp comes default with Fedora distribution so I had no problem in getting the package.

Next step was to make sure I have the correct configuration file setup. So I took a backup of the original file that is /etc/ntp.conf first.

mv /etc/ntp.conf /etc/ntp.conf.bak

Next I wrote the following in a new /etc/ntp.conf

server 127.127.1.0
fudge 127.127.1.0 stratum 1
crypto pw redhat randfile /dev/urandom
keysdir /etc/ntp
restrict default ignore
restrict 127.0.0.1
restrict 192.168.122.0 mask 255.255.255.0 nomodify noquery
driftfile /var/lib/ntp/drift


I know the above options are not the best of the breed but I will explain. First of all if we used our local hardware clock as the time source and then declared it to be at stratum 1 via the fudge line. That may sound madness to everybody. This was done just for testing purpose. Don’t do this in your production servers. Please use reliable time source which can be found at http://support.ntp.org/bin/view/Servers/StratumTwoTimeServers

The next line is the crypto line which tells that my ntpkey files are protected with a password redhat and that the filesource which is used for generating random seed data is /dev/urandom. Note that the password attribute is a important one so this file which /etc/ntp.conf should have a strict permission.


#chmod 640 /etc/ntp.conf
#chown root.ntp /etc/ntp.conf


Now the next line tells the directory where all the ntpkey_* files are stored. In fedora 9 it defaults to /etc/ntp/crypto but I used /etc/ntp which is default in RHEL 5.

Next three lines controls the access to the NTP server. The first of them restricts everybody to use the time server or remotely configure the time server. Next restrict line opens up restrictions for the local interface that is 127.0.0.1 This address can do anything no restrictions apply on it. The last restrict line opens the network 192.168.122.0/24 to use the time server to get time service but it can’t modify or query (status query on time server) the time server itself. That means any client in the 192.168.122.0/24 can configure 192.168.122.1 as it’s reliable time source but it can’t use to connect to that server via ntpq or ntpdc utility.

The last line specify the file name which contains the latest estimate of clock frequency error. This file is owned by ntp user.

In the next step we switch to directory /etc/ntp and generate the host keys and IFF parameters as we are going to use IFF identity scheme in this setup.


#cd /etc/ntp
#ntp-keygen -T -I -p redhat


The above command generates the key files and IFF parameters file. The host key file is protected with a password redhat that we also mentioned in /etc/ntp.conf. The list of files which were generated in my case are listed below


ntpkey_cert_station1.example.com
ntpkey_IFFpar_station1.example.com.3426211635
ntpkey_RSA-MD5cert_station1.example.com.3426211635
ntpkey_host_station1.example.com
ntpkey_iff_station1.example.com
ntpkey_IFFkey_station1.example.com.3426211635
ntpkey_RSAkey_station1.example.com.3426211635


In the above list some are key files and some are symbolic links to them. Next we need to extract the IFFkey so that it can transferred to every NTP clients of this server. We can also protect this key with a password that only we and the NTP client knows.


#ntp-keygen -e -q redhat -p linux > ntpkey_IFFkey_station1.example.com.3426211635
#scp ntpkey_IFFkey_station1.example.com.3426211635 root@server1.example.com:/etc/ntp


The above command generate the IFFkey file but the IFF parameter file itself is protected by a password which we specified in the first ntp-keygen command so with -q we specified that password and with -p we specified the password with which the IFFkey file will be protected (the client needs to know this password). The -e option is used to export the IFFkey.

Now I started the ntpd service and configured it to start automatically at the next boot up. Also I had a custom chain in my iptable based firewall in which I opened the udp/123 port on which ntpd listens.

#service ntpd start
#chkconfig ntpd on
#iptables -A MYCHAIN -p udp --dport 123 -j ACCEPT
#service iptables save

Next was the setup at client side that was pretty easy. First of all I configured as usual the main configuration file /etc/ntp.conf

#chmod 640 /etc/ntp.conf
#chown root.ntp /etc/ntp.conf
#vi /etc/ntp.conf


The client side ntp.conf contained the following:

server station1.example.com iburst autokey
crypto pw linux randfile /dev/urandom
keysdir /etc/ntp


The above lines specify the preferred time server to use be station1.example.com aka 192.168.122.1. The option autokey enables the use of public key cryptography. The next line specify the crypto password with which the client ntpkey_* files will be protected and also specify the random seed source to be used. Next line specify where to find the key data.

Next we generated the client side parameters by the following commands

#cd /etc/ntp/
#ntp-keygen -H -p linux
#ln -s ntpkey_IFFkey_station1.example.com.3426211635 ntpkey_iff_station1.example.com
#ln -s ntpkey_host_server1.example.com ntpkey_iff_server1.example.com


The above generates the host parameters on the client side protected by the password linux and next create some symlinks which later configure the IFF keys at the client side. Note here that the file ntpkey_IFFkey_station1.example.com.3426211635 was sent by the time server which was protected by the password linux.

The list of file with the prefix ntpkey_ in there name at the client side /etc/ntp were finally:-


ntpkey_cert_server1.example.com
ntpkey_host_server1.example.com
ntpkey_IFFkey_station1.example.com.3426211635
ntpkey_iff_server1.example.com
ntpkey_iff_station1.example.com
ntpkey_RSAkey_server1.example.com.3426211933
ntpkey_RSA-MD5cert_server1.example.com.3426211933


Now we started the time service at the client and configured it to start automatically at boot and also open the udp/123 port.


#ntpdate -b station1.example.com
#service ntpd start
#chkconfig ntpd on
#iptables -A MYCHAIN -p udp --dport 123 -j ACCEPT
#service iptables save


The first command in the above code was issued to first synchronize the clock of the client with that of the server then start the time service to later keep that new time in synchronization with the server. It took approx. 5 minutes to get synchronized and after that when issued the fo
llowing command the output was:


#ntpq -cas

#ntpq -c"rv 28241 flags"
assID=28241 status=f624 reach, conf, auth, sel_sys.peer, 2 events, event_reach,
flags=0x83f21


The last command issued returned the flags as 0x83f21 that signifies that the communication with the time server was successful and that IFF identity scheme with cryptography enabled was used.

Client side utilities to check the time configuration are ntpq,ntptrace,ntpdate,nptdc,ntpstat etc.

Pretty cool.

That was it. Quick and dirty post


04:30 < R0CK> Hello guys
04:31 < R0CK> any news about the add/remove software ?
04:31 < opsec> R0CK: ask a real question or no one will respond to you
04:32 < R0CK> I'm having problem on pirut, It's hanging when i run it,.
04:32 * opsec puts on his mid reading hat
04:32 < opsec> R0CK: and?
04:32 < BULLE> R0CK: that seems to be a common problem lately
04:32 < opsec> run it from the command line ..
04:32 < R0CK> opsec, I know very well how to ask my questions.
04:32 < opsec> if you get an error --> dpaste.com
04:33 < opsec> R0CK: no, you don't
04:33 < R0CK> opsec, well, my question was any news about the issue?
04:33 < opsec> there is no issue that i know of
04:33 < R0CK> opsec, don't try to be intelligent baby, you`re here since yesterday.
04:33 < opsec> either run it from the command like and paste the error to dpaste.com or use yumex instead
04:34 < opsec> R0CK: i'm done with you moron.
04:34 < R0CK> opsec, then stop supporting me,


It happens alot of time at #fedora.

Talking about the other thing I did this week was to finish the synopsis document for my project. It was a great experiencing learning how to make Data Flow Diagrams, Use case model etc. Learned alot. Now the team started writing the nasty code thingie.

The Captcha code for the JAVA Project is finished and now I have to figure out how can i insert the captcha module into the project and use it whenever I need it. I had to also learn packages/interface’s in JAVA. The other thing on my agenda is XML and handling XML via JAVA because I can sense that it will be needed in the project during some point of the life cycle (and my personal attraction to XML).

The progress on my secret python project is going good. I have been doing the testing stuff for a long time now with a different nick on freenode and it seems to perform well for the moment. I had to see a better alternative to CGI and supported by my shell provider so I can get a better stat’s page.

Downloaded Oracle 11g for Linux and will give it a try soon, need to learn how to configure a Oracle database server and configure/install clients so that they can utilize the server database instance.

It has been along time and there are lot of articles piling up in my docs stack. I got to find sometime to finish them up and upload on to my server (most important of them is the samba plus ldap guide for the fedora-docs team).

I was happy to make the TATA indicom internet connection working on Fedora. Now I can hope that some more people get involved into the learning Linux stuff (more closely).

From above it seems like a busy yet exciting weekend ahead. Hmm..

Then I got busy in my 2nd semester examination which were a total hell (took the hell out of me).

Now that my exams are over and I got some time I decided to roll upon some new stuff. First is a secret project I have started working on (in python) and second is JAVA. Truely speaking I have never done JAVA sincerely though I got alot of time to do it. But now I decided to look around and started working on the Core stuff (as JAVA is in my current semester).

The other important thing these days is the IBM Challenge program going on in which I have enrolled as a participant having 3 more members in my team. So meanwhile I am learning the core JAVA stuff I am also looking into the Advance JAVA stuff the web thingie. We got our team blog now thanks to /me.

So I guess probably for some more time I will not be updating this blog (my first and only blog) because of some small works I am indulged in. Keep in touch with the team blog.

See you soon.

After installing all the stuff I did the following things (Note: This is for Intel video card users and GNOME no KDE support Yet ):

Step 1: vi /home/deepsa/compiz-fusion-run
LIBGL_ALWAYS_INDIRECT=1 INTEL_BATCH=1 compiz –replace –sm-disable ccp &

Save and exit

Step 2: chmod a+x /home/deepsa/compiz-fusion-run

Step 3: gnome-session-properties

Startup Programs -> New
Name : Compiz-Fusion
Command: /home/deepsa/compiz-fusion-run
OK

Step 4: System -> Preferences -> CompizConfig Settings Manager -> Window Decorations -> Command -> emerald –replace

Step 5: Logout and LogBack In.

Some screenshots of new features

Expo Plugin

Cube with reflection

Click on the Add button on the bottom and you will be shown something like this

I have filled almost all the values for my test but you need to change them according to your need.

Protocol: XMPP (earlier used to be Jabber)
Screen Name: Must be filled up with your gmail-id before @gmail.com.
Server: Should be gmail.com.
Resource: Can be anything the default "Home" will also work.
Password: You gmail password goes here.

This is it. Hope you get your g-talk account working with Pidgin 2.0.0 in GNU/Linux.

This article helped me learn how I can build my own rpm’s using the source rpm’s provided by Redhat. That was really nice. I got my whole virtualization stuff in Redhat5 upgraded to the latest version.

And yeah I am working on my website too. Don’t know when I will get satisfy with the design and stuff I am hosting.

For Physical Machines:

install
url --url=http://192.168.1.5/Dump/
key 2515dd4e215225dd
lang en_US.UTF-8
keyboard us
xconfig --startxonboot
network --device eth0 --bootproto static --ip 192.168.1.23 --netmask 255.255.255.0 --gateway 192.168.1.1 --hostname server1.example.com
rootpw --iscrypted $1$QGYhCela$pNOZoWf4XoONvUdND/nS01
firewall --disabled
authconfig --enableshadow --enablemd5
selinux --disabled
timezone Asia/Calcutta
bootloader --location=mbr --driveorder=hda --append="rhgb quiet"
# The following is the partition information you requested
# Note that any partitions you deleted are not expressed
# here so unless you clear all partitions first, this is
# not guaranteed to work
#clearpart --linux
#part / --fstype ext3 --onpart sda3
#part swap --onpart sda6
#part /home --fstype ext3 --onpart sda7

%packages
@base
@base-x
@core



manage_dhcp: 1
next_server: '192.168.1.5'
server: '192.168.1.5'



No setup problems found.
Manual review and editing of /var/lib/cobbler/settings is recommended to tailor cobbler to your particular configuration.
Good luck.


cobbler distro add --name=rhel5
-dvd --kernel=/rhel5/Dump/images/pxeboot/vmlinuz --initrd=/rhel5/Dump/images/pxeboot/initrd.img --arch=x86


cobbler profile add --name=redhat5y --distro=rhel5-dvd –kick-start=/rhel5/Dump/ks.cfg


For example I want to add a systemA in the profile redhat5y I can give the following command:

cobbler system add –name= --profile=redhat5y



cobbler distro add –name=rhel5-xen –kernel=/rhel5/Dump/images/xen/vmlinuz –initrd=/rhel5/Dump/images/xen/initrd.img --arch=x86



cobbler profile addd --name=redhat5x --distro=rhel5-xen --kick-start=/rhel5/Dump/ks1cfg --virt-file-size=2 –virt-ram=256



install
nfs --server=192.168.1.5 --dir=/rhel5/Dump
key 2515dd4e215225dd
lang en_US.UTF-8
keyboard us
network --bootproto=bootp --device=eth0 --onboot=on
rootpw --iscrypted $1$VwD9nalr$06K0bUawzanX72gNk0es91
firewall --disabled
authconfig --enableshadow --enablemd5
selinux --disabled
timezone --utc Asia/Calcutta
bootloader --location=mbr --driveorder=xvda --append="console=xvc0"
# The following is the partition information you requested
# Note that any partitions you deleted are not expressed
# here so unless you clear all partitions first, this is
# not guaranteed to work
clearpart --all --drives=xvda
part /boot --fstype ext3 --size=100 --ondisk=xvda
part pv.2 --size=0 --grow --ondisk=xvda
volgroup VolGroup00 --pesize=32768 pv.2
logvol / --fstype ext3 --name=LogVol00 --vgname=VolGroup00 --size=1024 --grow
logvol swap --fstype swap --name=LogVol01 --vgname=VolGroup00 --size=144 --grow --maxsize=288

%packages
@admin-tools
@base
@core



koan --virt –server=192.168.1.5 –profile=redhat5x



xm console 00_16_3E_6B_D5_39


libvir: Xen Daemon error : POST operation failed: (xend.err 'Error creating domain: I need 262144 KiB, but dom0_min_mem is 262144 and shrinking to 262144 KiB would leave only 235124 KiB free.')

libvir: Xen Daemon error : POST operation failed: (xend.err 'Device 0 (vif) could not be connected. Hotplug scripts not working.')


title Red Hat Enterprise Linux Client (2.6.18-1.2747.el5xen)
root (hd0,5)
kernel /boot/xen.gz-2.6.18-1.2747.el5
module /boot/vmlinuz-2.6.18-1.2747.el5xen ro root=LABEL=/ rhgb quiet
module /boot/initrd-2.6.18-1.2747.el5xen.img


virsh

Inside the virsh prompt

virsh # connect
virsh # setmem Domain-0 500000

cd /etc/xen/auto
ln -s ../rhel5b2-pv1 .

chkconfig xendomains on

service xendomains stop

So I went on to GNU/Linx community and put up this question. Okay I got some inputs some directions and finally I got what I wanted. First of all let me tell you the scenario once more so that you can better get what I want to say.

Suppose you have installed a RHEL5 system and now after the installation is complete you want to install a package (which is not installed). You put in the DVD and mount it. Go to the said directory and try to install the package via the well old “rpm” command. But to your surprise you found that it failed due to dependency problems. Okay no problem. We all know how to deal with it. We use the “–aid” switch with our “rpm” command that will automatically install the dependency rpm first then the said rpm. Well we try that but it again failed with the same error message. That means it’s not finding the dependency rpm. But wait. The dependency rpm and the rpm we want to install both are in the same directory then why is the “rpm” command failing.

Well that’s because in RHEL5 (as in Fedora Core 6) all the things are controlled by “yum”. I read somethings about “yum” and quickly found that it had problem with dvd sources. But I didn’t found any thing on how to disable “yum” completely and go through the well old command line way of installing packages. But I found a way out by which “yum” can access DVD sources and if that happens we can install/un-install packages easily either via graphical tool(system-config-packages) or the command line via “yum” command.

Okay so let’s start this.I inserted the RHEL5 Client DVD and mounted it on /media/dvd/

Then I created a ISO file for this DVD using the “mkisofs” command.

Installation procedure (anaconda) was exactly a copy of Fedora Core 6. I had tried FC6 earlier but only once and then I never used it. Same went for RHEL5. During installation it asked me to enter a key. That was the only difference I saw between FC6 and RHEL5 installation. I got the key from internet (I don’t remember from where) and now I don’t have the key. LOL.

The few changes I noted down between the previous release of RHEL and this release were that this release had 3D Desktop all because of AIGLX enabled Xorg 7.x. Compiz was providing the desktop effect. I also saw in this release Xen enabled 2.6.18 kernel. Oh wait. Let me show you the `uname -a` of RHEL5 beta2:

Linux deepsa.lenovo 2.6.18-1.2747.el5 #1 SMP Thu Nov 9 18:55:30 EST 2006 i686 i686 i386 GNU/Linux

Well the above one is not Xen enabled kernel but if you choose virtualiaztion as a installation option you surely get Xen only. Other changes were Eclipse, JAVA etc. development tools available in this release. I selected all of these and the Servers too. The installation took around 35 minutes. Wow. That was fast.
Okay. Now I booted the Kernel and as it should be, Xen started. Xend (Xen daemon) also started. Most notably I saw Avahi daemon. It was great to see it in RHEL(now surely will get to see some great desktop).

Okay so here came the Login screen. But hey what’s that. My resolution was not according to my laptop it was 1024×768 it should be 1280×800. No problem. I logged in(via root). So as I have guessed it was just like Fedora 6. No change. The following hardware which I need as soon as I install a Operating System didn’t worked in RHEL5.

a) Wireless Internet
b) Bluetooth
c) Proper Screen Resolution.


[roo
t@deepsa Client]# rpm -ivh xfig-3.2.4-21.1.i386.rpm --aid

I got the result

error: Failed dependencies:
transfig >= 1:3.2.4-12 is needed by xfig-3.2.4-21.1.i386


In the recent months I had tried alot of distros:

a) BackTrack Beta 2.0
b) OpenSUSE 10.2
c) Gentoo 2006.1
d) RHEL5 Beta2
e) Ubuntu Edgy
f) Ubuntu Fiesty Herd 2


mkdir /mnt/C
mount /dev/sda5 /mnt/C
cd /mnt/C
dd if=tiger-x86-flat.img of=/dev/sda1 bs=512 skip=63



mkdir /mnt/mm
mount /dev/sda6 /mnt/mm


Now I edit the menu.lst configuration file of grub so that I can boot OSX86.


cd /mnt/mm/boot/grub
vi menu.lst

Title Mac OSX86
rootnoverify (hd0,0)
makeactive
chainloader +1


I also edited my Windows XP and SuSE linux entry because the partition number have changed after adding the HFS partition at the beginning.


Title Windows XP
rootnoverify (hd0,1)
makeactive
chainloader +1


After that I saved the file and quit. Then I issued the command:


grub-install –root-directory='/mnt/mm' /dev/sda


This command installed my grub on the MBR.


mkdir /mnt/C
mount /dev/sda2 /mnt/C
cp /mnt/C/BOOT.INI /root
chmod a+w /root/BOOT.INI
vi /root/BOOT.INI


Edit the partition(1) to partition(2) on the two lines. Save and exit.


umount /mnt/C
chmod a-w /root/BOOT.INI
ntfscp /dev/sda2 /root/BOOT.INI /mnt/C/BOOT.INI


Hardware:
Lenovo Laptop 3000N100 (07684KA)
512 DDR2 SDRAM (667 Mhz)
Intel Centrino Dual Core 1.66 Ghz
80 GB SATA Hard Drive
RealTek Ethernet
Intel Pro 10/100 Wireless
Intel 945GM Mobile Graphics
Intel High Definition Audio (SoundMAX)
Integrated Camera
DVD RAM
PCMCIA
Card Reader
4 USB
TV OUT
External Monitor Port
Firewire Device
15.4” inch Wide Screen
Finger Print Scanner
Bluetooth Enabled

Step 1) Vmware needs that the kernel and it’s appropriate sources are installed properly on your system. I recommend you to install the latest kernel for Fedora core 5. You can do this by issuing the following simple commands (note that you need to be connected to internet as a root user in order for these commands to run successfully):
a) yum install kernel
b) yum install kernel-devel
This will install the latest kernel and it’s source code. The version I have is 2.6.16-1.2080_FC5. After you do this step you need to reboot and boot into the newly installed kernel.

Step 2) We need the VMware Software for Fedora Core 5. For this we can go here and follow the steps described there. Download the RPM file for VMware 5.5.1-19175 version.

Step 3) As the above software is a shareware version you get a key for the first 30 days trial on the e-mail provided by you during registration on the link provided in step (2), note down the key and save it somewhere in a text file we will need it after installation and configuration finishes up.

Step 4) Now as VMware 5.5.1 has reported some problem during the configuration in Fedora Core 5 we will need to download this specially created perl script from here.

Step 5) Install VMware from the rpm by issuing the following command:

rpm -ivh VMware-workstation-5.5.1-19175.i386.rpm

Step 6) After installing the above rpm we need to extract the script downloaded in step (4) and follow these steps:

a) gzip -d vmware-any-any-update101.tar.gz
b) tar -xf vmware-any-any-update101.tar
c) cd vmware-any-any-update101
d) ./runme.pl

Step 6) After you run the runme perl script it will ask you many things. These should be your answers:

Question 1) Before running VMware for the first time after update, you need to configure it for your running kernel by invoking the following command:
“/usr/bin/vmware-config.pl”. Do you want this script to invoke the command for
you now? [yes]

Answer 1) Press Enter

Then Read the License Agreement (press Space Bar 7-8 times) and in the end type yes and press Enter.

Question 2) In which directory do you want to install the mime type icons?
[/usr/share/icons]

Answer 2) Press Enter

Question 3) What directory contains your desktop menu entry files? These files have a
.desktop file extension. [/usr/share/applications]

Answer 3) Press Enter

Question 4) In which directory do you want to install the application’s icon?
[/usr/share/pixmaps]

Answer 4) Press Enter

Question 5) None of the pre-built vmmon modules for VMware Workstation is suitable for your running kernel. Do you want this program to try to build the vmmon module for your system (you need to have a C compiler installed on your system)? [yes]

Answer 5) Press Enter

Question 6) What is the location of the directory of C header files that match your running kernel? [/lib/modules/2.6.16-1.2080_FC5/build/include]

Answer 6) Yes (note your directory listing may differ with the given here !)

Question 7) Do you want networking for your virtual machines? (yes/no/help) [yes]

Answer 7) Press Enter (if you don’t want networking type no and press enter). If you need to run internet from your virtual machine in future and you happen to use a broadband connection please select yes and press enter.

Question 8) Do you want to be able to use NAT networking in your virtual machines? (yes/no) [yes]

Answer 8) This depends. For my setting I typed no and Pressed Enter.

Question 9) Do you want to be able to use host-only networking in your virtual machines?[no]

Answer 9) Answer it yes or no if you want/don’t want to communicate with your Fedora Core 5 host machine inside the virtual machine you install later. I choosed no.

Step 7) After these simple questions and configuration of VMware-5.5.1 finishes off and now we can run the vmware software by issuing the following command at command line:

vmware

Step 8) Once the software starts up goto Help->Enter Serial Number … and paste the serial number you got via email during step (3).

And voila! VMware is now installed and configured on your fedora core 5 machine.
In the next post I am going to explain you how did I configured the Windows XP Professional Virtual Machine inside VMware and that too through the step by step guide. So for the time being just play around with the above guide and try your hands with the different options giving in the software.

Happy Linux’ing !!

This document is a copy of a document which I saw in late 2004 when I was searching for a howto to configure internet in linux.

This document describes howto setup bsnl broadband service on your gnu/linux server/desktop. It assumes that you are neither linux beginer nor an expert. It assumes that you are aware of basic gnu/linux commands.

I have installed it on redhat 9. I dont find any reason why it would not work on following distributions:

• FC1
• FC2
• FC3
• FC4
• FC5

An unofficial FAQ on Dataone can be found here

Pre-Requisites

• Please ensure that you have connected a working ethernet card which is supported on gnu/linux. Test the ethernet card & ensure that it is working.
• Root access to a gnu/linux box

Before we start, please ensure that you have the following details on hand:

• BSNL Broadband connection -:)
• Username & Password

Using the builtin dialer

Using RP-PPPOE as dialer

Installation

• As BSNL broadband is using rp-pppoe (Roaring Penguin – Point to Point Over Ethernet), we would be required to download & install this package. This package is normally available on most of the distributions. But, if you are using redhat7.3, please download the following packages:

If you are downloading the tar.gz version, you have to build the rpm first. So, please walk through the following procedure to build the rpm:

Procedure to build RPM from tar.gz.

 1. Login as root
2. Download tar.gz version & execute the following commands
# tar -zxvf rp-pppoe-3.5.tar.gz (untar this package)
 * This command would untar the package in the current directory.
# cp rp-pppoe-3.5.tar.gz /usr/src/redhat/SOURCES/
# cd rp-pppoe-3.5
 * The rpm spec file, rp-pppoe.spec is available in this directory.
# cp rp-pppoe.spec /usr/src/redhat/SPECS/
# cd /usr/src/redhat/SPECS
# rpm -ba rp-pppoe.spec
*  The  above  command would untar the tar.gz package in ../BUILD
directory, compile the package and finally it would create neces
sary rpms in

If everything goes fine, you would see the following or similar message:

 Wrote: /usr/src/redhat/SRPMS/rp-pppoe-3.5-1.src.rpm
Wrote: /usr/src/redhat/RPMS/i386/rp-pppoe-3.5-1.i386.rpm
Wrote: /usr/src/redhat/RPMS/i386/rp-pppoe-gui-3.5-1.i386.rpm
Executing(%clean): /bin/sh -e /var/tmp/rpm-tmp.97349
+ umask 022
+ cd /usr/src/redhat/BUILD
+ cd rp-pppoe-3.5
+ rm -rf /tmp/pppoe-build
+ exit 0

If this is the case, the necessary rpms are successfully created and they are available in appropriate directory.

 Source RPMS - /usr/src/redhat/SRPMS/rp-pppoe-3.5-1.src.rpm
Binary RPMS - /usr/src/redhat/RPMS/i386/rp-pppoe-3.5-1.i386.rpm
GUI RPMS    - /usr/src/redhat/RPMS/i386/rp-pppoe-gui-3.5-1.i386.rpm

Now, at this stage you are having necessary rpms with you. Either you have downloaded them from the above website. Or you would have built them on your own:

Command Line Version

 # rpm -vih rp-pppoe-3.5-1.i386.rpm

It would install the command line version of rp-pppoe package.

GUI Version

 # rpm -vih rp-pppoe-gui-3.5-1.i386.rpm

It would install the gui version of rp-pppoe package. If you have successfully installed, then you can proceed to next section.

Configuration

Once you have installed rp-pppoe package, all necessary commands would have been installed.

 adsl-setup  - to configure the connection parameters
adsl-start  - to start the connection
adsl-status - to check status of the connection
adsl-stop   - to stop the connection

Now, we are ready to configure the BSNL broadband internet connection.

Here you have to configure the following parameters:

 * userid
* network interface (eth0 or ..)
* Demand value (normally you can leave it blank)
* DNS Information (primary & secondary)

• With regard to primary & secondary dns, You can leave it blank, as it would be taken care automatically. If you wish to know the dns details, just complete this wizard and when you run adsl-start, do:

 # tail -f /var/log/messages


Mar 15 15:29:48 Gateway pppd[3564]: Remote message: Authentication success,Welcome!
Mar 15 15:29:48 Gateway pppd[3564]: local  IP address 59.92.9.154
Mar 15 15:29:48 Gateway pppd[3564]: remote IP address 59.92.8.1
Mar 15 15:29:48 Gateway pppd[3564]: primary   DNS address 61.1.96.69
Mar 15 15:29:48 Gateway pppd[3564]: secondary DNS address 61.1.96.71

There it goes! The primary & secondary dns details are found and it would be automatically updated in /etc/resolv.conf.

• Then you should enter the following details:

 * Password
* Firewall (set is to 2, MASQUERADE)
* Confirm

• The following is the screen shot of adsl-setup command.

 # adsl-setup
Welcome to the Roaring Penguin ADSL client setup.  First, I will run
some checks on your system to make sure the PPPoE client is installed
properly...

Looks good!  Now, please enter some information:

USER NAME

>>> Enter your PPPoE user name (default test@dataone):

INTERFACE

>>> Enter the Ethernet interface connected to the ADSL modem
For Solaris, this is likely to be something like /dev/hme0.
For Linux, it will be ethn, where 'n' is a number.
(default eth0):

Do you want the link to come up on demand, or stay up continuously?
If you want it to come up on demand, enter the idle tim
e in seconds
after which the link should be dropped.  If you want the link to
stay up permanently, enter 'no' (two letters, lower-case.)
NOTE: Demand-activated links do not interact well with dynamic IP
addresses.  You may have some problems with demand-activated links.
>>> Enter the demand value (default no):

DNS

Please enter the IP address of your ISP's primary DNS server.
If your ISP claims that 'the server will provide DNS addresses',
enter 'server' (all lower-case) here.
If you just press enter, I will assume you know what you are
doing and not modify your DNS setup.
>>> Enter the DNS information here: 61.1.128.5
Please enter the IP address of your ISP's secondary DNS server.
If you just press enter, I will assume there is only one DNS server.
>>> Enter the secondary DNS server address here: 61.0.0.5

PASSWORD

>>> Please enter your PPPoE password: 
>>> Please re-enter your PPPoE password:

FIREWALLING

Please choose the firewall rules to use.  Note that these rules are
very basic.  You are strongly encouraged to use a more sophisticated
firewall setup; however, these will provide basic security.  If you
are running any servers on your machine, you must choose 'NONE' and
set up firewalling yourself.  Otherwise, the firewall rules will deny
access to all standard servers like Web, e-mail, ftp, etc.  If you
are using SSH, the rules will block outgoing SSH connections which
allocate a privileged source port.

The firewall choices are:
0 - NONE: This script will not set any firewall rules.  You are responsible
        for ensuring the security of your machine.  You are STRONGLY
        recommended to use some kind of firewall rules.
1 - STANDALONE: Appropriate for a basic stand-alone web-surfing workstation
2 - MASQUERADE: Appropriate for a machine acting as an Internet gateway
              for a LAN
>>> Choose a type of firewall (0-2): 2

** Summary of what you entered **

Ethernet Interface: eth0
User name:          test@dataone
Activate-on-demand: No
Primary DNS:        61.1.128.5
Secondary DNS:      61.0.0.5
Firewalling:        MASQUERADE

>>> Accept these settings and adjust configuration files (y/n)? y
Adjusting /etc/ppp/pppoe.conf
Adjusting /etc/resolv.conf
(But first backing it up to /etc/resolv.conf-bak)
Adjusting /etc/ppp/pap-secrets and /etc/ppp/chap-secrets
(But first backing it up to /etc/ppp/pap-secrets-bak)
(But first backing it up to /etc/ppp/chap-secrets-bak)



Congratulations, it should be all set up!

Type 'adsl-start' to bring up your ADSL link and 'adsl-stop' to bring
it down.  Type 'adsl-status' to see the link status.

Internet Connection

Now, if you see this Congrats! message, then you are all set & done. You can type,

# adsl-start ..Connected! #

Thats it. The bsnl broadband setup has been installed & configured.

At this point, you can go through the man pages of appropriate adsl-* commands for details.

Hope this article helps you out in configuring internet on linux specially for the redhat persons. I will soon post a article regarding configuring network interfaces on linux and other stuff related to it.