Monthly Archives: July 2008

Help Organize PHPCamp Pune

PHPImage via Wikipedia

PHP enthusiasts in Pune are organizing a barcamp focused towards PHP based web application development, in September 2008. (If you don’t know what a barcamp camp is, now is a good time to find out.) What started out as a small initiative at getting together the PHP community in Pune for an informal gathering is, judging from the initial response, fast snowballing into a major all-India conference.

It is still very early, and the details are all still being worked out. Since this is an unconference, it is all volunteer driven, and anyone can participate, and more importantly, anyone can join the organizing committee. Which means that if you are interested in PHP, you should seriously consider helping out with the effort. They are still looking for speakers (and due to the unconference nature of the event, this will continue until the day of the event). More importantly they are looking for sponsors, to smoothen the process of hosting the event. This means that if you are a company (or know a company) that wants to get visibility with the most enthusiastic PHP developers in the country, you should sponsor this event.

In any case, join the mailing list to keep tabs on what the organizers are planning, and register yourself for attending the event, so they have an idea of how many people to expect.

Check out other technology user-groups and organizations in Pune at the PuneTech wiki (for example, Pune Java, Pune Ruby, Pune Linux User Group, PHPCamp, Pune Project Management Institute etc.)

Zemanta Pixie

Introduction to Server Virtualization

Virtualization is fast emerging as a game-changing technology in the enterprise computing space. What was once viewed as a technology useful for testing and development is going mainstream and is affecting the entire data-center ecosystem. Over the course of the next few weeks, PuneTech is going to run a series of articles on virtualization from experts in the industry. This article, the first in the series, gives an introduction to server virtualization, and has been written by Anurag Agarwal and Anand Mitra, founders of KQ Infotech.

What is virtualization

Virtualization is essentially some kind of abstraction of computing resources. There are various kinds of abstractions. Files provide an abstraction of disk blocks into linear space. Storage virtualization products, like logical volume manager, virtualize multiple storage devices into single storage and vice versa.

Processes are also a form of virtualization. A process provides an illusion to the programmer that she has the entire address space at her disposal and has exclusive control of hardware resources. Multiplexing of these resources between all the processes on the system is done by the OS, transparent to the process. This concept has been universally adopted.

All multi-programming operating systems are characterized by executing instructions in at least two privilege levels i.e. unprivileged for user programs, and privileged for the operating system. The user programs use “system calls” to request the operating system to perform privileged operations on its behalf. The interface which consists of the unprivileged instruction set and the set of system calls define an “extended machine” which is easier to program than the bare machine and makes user programs more portable.

The benefits of having the kernel wrapping completely around the hardware and not exposing it to upper layer has its advantages. But in this model, only one operating system can be run at a given time. One cannot perform any activity that would disrupt the running system (for example, upgrade, migration, system debugging, etc.)

A virtual machine provides an abstraction of complete physical machine. This is the also known as server virtualization. The basic idea is to run more than one operating system on the single server at the same time.

The History of Server Virtualization

In 1964, IBM had developed a Virtual Machine Monitor (CP) to run their various OSes on their mainframes. Hardware was too expensive to leave underutilized. They had addressed many of the performance challenges inherent in virtualization by designing hardware amenable to virtualization. However with the advent of cheap computing resources and proliferation of commodity hardware, virtualization was no longer popular and was viewed as a artifact of a an era where computing resources were scarce. This was reflected in design of x86 architectures which no longer provided enough support to implement virtualization efficiently.

With the cost of hardware going down and complexities of software increasing, a large number of administrators started putting one application per server. This provides them isolation, where one application does not interfere with other application. However, over some time it started resulting into a problem called server sprawl. There are too many underutilized servers in data centers. Most windows servers have average utilization between 5% and 15%. This utilization rate will further go down with dual core and quad core processors becoming very common. In addition to the cost of the hardware, there are also power and cooling requirements for all these servers. The earlier problem of utilization of hardware resources has started surfacing again.

Ironically the very reason which resulted in the demise of virtualization in the mainstream, was the cause of its resurrection. The features which made the OSes attractive, also made them more fragile. And this renewed interest in virtualization resulted into VMWare providing a server virtualization solution for x86 machines in 1999. Server consolidation has increased the server utilization to the 60% to 80% level. This has resulted in 5 to 15 times reduction in the servers.

Virtual machines have introduced a whole new paradigm of looking at operating systems. Traditionally they were coupled with physical machines, and they needed to know all the peculiarities of hardware. Once hardware becomes obsolete, your operating system becomes obsolete too. But virtual machines have changed that notion. They have decoupled the operating systems from hardware by introducing a virtualization layer called virtual machine monitor (VMM).

Types of Virtualization architectures

There are many VMM architectures.

Full emulation: It is the oldest virtualization technique in use. An emulator is a software layer which tracks the memory and CPU state of the machine being emulated and interprets each instruction applying the effect it would have on the virtual state of the machine it has constructed. In a regular server, machine instructions are directly executed by the CPU and the memory is directly manipulated. In full emulation, the instructions are handed over to the emulator and it then converts these instructions into a (possibly different) set of instructions to be executed on the actual underlying physical machine. Full emulation is routinely used during the development of software for new hardware which might not be available yet. Virtualization can be considered as a special case of emulation where both the machine being emulated and host are similar. This allows execution of unprivileged instructions natively. Qemu falls in this category.

Hosted: In this approach, a traditional operating system (Windows or Linux) runs directly on the hardware. This is called the host OS. VMM is installed as a service in the host OS. This application creates and manages multiple virtual machines as processes. Each virtual machine process has a full operating system inside it. Each of these is called a guest OS. This approach greatly simplifies the design of the VMM as it can directly use the services provided by the host operating system. VMWare server, VMWare workstation, Virtual box, and KVM fall in this category.

Hypervisor based: Hosted VMM solutions have a high overhead, as the VMM does not directly control the hardware. In the hypervisor approach the VMM is directly installed on the hardware. The VMM provides virtual hardware abstractions to create and manage multiple virtual machines. Performance overhead in this approach is very small.

Another way to classify virtual machines is on the basis of how privileged instructions are handled. Modern processors have a privileged mode of execution that the OS kernel executes in, and a non-privileged mode that the user programs execute in. This can cause a problem for virtual machines because although the host OS (or the hypervisor) runs in privileged mode the entire guest OS runs in non-privileged mode. Most of today’s OSs are specifically designed to run in privileged mode, and hence their binaries end up having some instructions that must be run in privileged mode. (For example, there are 17 such instructions in the Intel IA-32 architecture.) This causes a problem for the virtual machine, and there are two major approaches to handling this problem.

Para virtualization: In this approach, the binary of the OS needs to be rewritten statically to replace the use of the privileged instructions by appropriate calls into the hypervisor. In other words, the operating system needs to be ported to the virtual hardware abstraction provided by VMM. This requires changes in the operating system code. This approach has least performance penalty. This is the approach taken by Xen.

Full virtualization: In this approach, no change is made in the operating system code. There are two ways of supporting this.

  • Using run time emulation of the privileged instructions. The VMM monitors program execution during runtime, and takes over control of execution whenever a privileged instruction arises in the guest OS. This approach is called binary translation. VMWare uses this technology.
  • Hardware assisted virtualization: Both intel and AMD have come up virtualization extensions of their hardware to support virtualization. Intel calls this VT technology and AMD calls it SVM technology. These extensions provide an extra privilege level for VMM to run. These extensions have provided a number of additional features like nested page tables and IOMMU, to make virtualization more efficient.

Virtualization Vendors

VMWare: VMWare has a suite of products in this area. There are two hosted products, called VMWare workstation and VMWare server. Their hypervisor product is called VMWare ESX. They have one version of ESX that comes burned in the bios. It is called VMWare ESXi. They have virtual center as management product to manage complete virtual machine infrastructure in the data center. There all the products are based on the dynamic binary translation technology. They support various flavors for Windows and Linux.

Xen: It is an open source project. It is based on para-virtualization and hypervisor technologies. Linux is modified to support para-virtualization. Xen now supports Windows with hardware assisted virtualization. There are number of products based on Xen. Citrix, which bought XenSource has couple of Xen based products, Sun has xVM, Oracle has Oracle VM. Redhat and Suse have been shipping Xen as part of their Linux distributions for some time.

Hyper-V: This is Microsoft’s entry in this space. It is similar to the Xen architecture. It also requires hardware assistance. It comes bundled with Windows server 2008, and supports running Windows and Linux guest operating systems in the virtual machines.

Advantages of Virtualization

Virtualization has also provided some new capabilities. Server provisioning becomes very easy. It is just creating and managing a virtual machine. This has transformed the way testing and development are done. There is another interesting feature called Vmotion or live migration, where a running virtual machine can be moved from one physical machine to other physical machine. Executing of the virtual machine is briefly suspended, and the entire image of the virtual machine is moved to a different machine. Now the execution can be re-started and the guest OS will continue from exactly the same point where it was suspended. This eliminates the need for downtime, even for things like hardware maintenance. This also enables the dynamic resource management or utility computing.

Adoption of server virtualization has been phenomenal. There are already hundreds of thousands servers running virtual machines. Initial adoption of virtual machine was restricted only to test and development, but now it has matured enough to become quite popular in production too.

About the Authors

Anurag Agarwal

Anurag Agarwal has more than 11 years of industry experience both in India and US. Prior to founding the KQInfotech, he was a technical director at Symantec India. Anurag has designed, developed various products at Symantec (earlier Veritas). During 2006-2007, Anurag has conceived the idea of Software Fault Tolerance for Xen at Symantec. He was awarded highest technical award of outstanding innovator in 2006 for this invention. Anurag has build and lead a team of ten people in India to take it from idea stage to product.

During the same time Anurag has started working with College of Engineering, Pune. There he and his friends offered a full semester course in Linux kernel. Anurag was also involved in mentoring a large number of students from various engineering colleges. This involvement in teaching and mentoring students resulted in formation of KQInfotech with training and mentoring focus.Prior to this, Anurag has architected scalable transaction system for Cluster file system at Symantec in USA. This architecture improved scalability of cluster file system from three nodes to sixteen nodes and beyond. He was awarded star award for this work. He has filed half a dozen patents at Symantec. Anurag has extensive knowledge in Solaris, Linux kernel, file system, storage technologies and virtualization.He has ME from Indian Institute of Science, Bangalore and BE from MBM Engineering College, Jodhpur.

Anand Mitra
After completing his post-graduation (iitb) in 2001, Anand had been working with Symantec India (Formerly Veritas). Prior to founding KQInfotech, he was a Principal Software Engineer at Symantec, chartered with the task of scoping and designing a support for windows on Xen based Fault Tolerance. He has worked for 6.5 years on the clustered filesystem product VxFS & CFS. He had architected the online upgrade for Veritas File system and designed the write fastpath which improved performance of the file system. He has also designed the integration of Power6 (powerPC) CPU feature of storage keys for the Veritas storage stack. He co-maintained technical relations with IBM for special proprietary kernel interfaces within AIX and designed a filesystem pre-allocation API for IBM DB2 database.

POCC meeting (Footprint Ventures and Google) report

I’m liveblogging the Pune OpenCoffee Club meeting organized by nFactorial. There are over 50 people and SEED Infotech‘s classroom is overflowing. They are having to bring in more chairs for people who are still standing.

Neill Brownstein of Footprint Ventures is giving a pitch about his company

  • Footprint has created $30 billion in value over 35 years. 20% IRR. 23 IPOs
  • Among other companies, Footprint has invested in our own Veritas Software.
  • Average run-rate of companies they fund is currently in the $1 million

Neeraj Arora, works in Mergers and Acquisitions at Google, focusing on APJ – mainly Australia and India

  • They don’t invest in startups. They acquire. But they work closely with some seed funds.
  • These are the guys that did the acquisition of a company in Australia which ultimately became Google maps
  • They are not looking for anything in particular. Consumer, enterprise, early stage, late stage. Is all OK. Pre-revenue is good. But must be a product company.
  • Question: do they only acquire companies with technology that is compatible with theirs? Answer: if it needs to integrate with Google products, then yes, the technology does need to be compatible.
  • The company needs to relocate. Either to Mountain View, US, or Bangalore, in India. (in most cases)
  • They sometimes buy companies just because they think it is cool, even if they can’t figure out where it will really fit in
Zemanta Pixie

Java Meet, PLUG meeting, and startup chat – 5 July

Java

Image via Wikipedia

What: Pune Linux Users Group (PLUG) meeting followed by a Talk on “Contrasting Java and Dynamic Languages” by Dhananjay Nene (as part of the Java meet by IndicThreads.) Also, check out the proto.in online startup chat event at 3pm.

When: Saturday, 5th July. PLUG meeting 4pm to 6pm. Dhananjay’s talk from 6pm to 7:30pm

Where: Symbiosis Institute of Computer Studies and Research (SICSR), 7th floor, Atur Center, Model Colony, Pune, India (Map)

Registration and Fees: The event is free for all. Register here.

Details: PLUG meeting
The PLUG meeting is open to all, there are no charges or pre-requisites to attend the meeting. If you are intrested in FOSS (Free/Open Source Software) you are welcome to the meeting.

Details: Contrasting Java and Dynamic Languages
Given the increasing interest in dynamic languages such as Python, Ruby, PHP along with the increasing threat perception to Java, Dhananjay Nene talks about his experience with using these languages and how they distinguish themselves from Java. The session “Contrasting Java and Dynamic Languages” will also discuss the role of Java based scripting languages such as Groovy, JRuby and Jython.

About the Presenter:
Dhananjay Nene has been programming for 17 years and was associated with Citicorp Overseas Software Limited and AT&T and his last assignment was being the CTO and Head of Product Development for CashTech Solutions. He has worked on a number of domains including banking, telecom, network management, wireless networking and educational software. He is an MBA from IIM Ahmedabad, has worked in senior management positions and has managed large sized development teams.

In recent years he had focused on building very high performance java based frameworks and solutions. Currently he is developing software using Python. He blogs at http://blog.dhananjaynene.com

Proto.in’s online startup chat

If you don’t feel like leaving your house, you can check out the proto.in online start-up chat event that starts at 3pm.

Zemanta Pixie

POCC meetup – Footprint Ventures and Google M&A – 3 July

What: Pune OpenCoffee Club get-together. Informal meeting with Footprint Ventures, and Google Corporate Development and M&A.

When: Thursday, 3rd July, 6pm.

Where: Seed Infotech’s office in Erandawana

Registration and Fees: This event is free for everyone. Register at upcoming.

Details:

Hemant Joshi, of nFactorial would like to invite POCC members to an informal get-together with special invitees: Footprint Ventures, and Google Corporate Development on 3rd July, 5pm.

Footprint ventures will be there to meet startups and talk about what they look for when considering Series A investments.

Neeraj Arora will be representing Google’s Corporate Development and M&A division and will be there to interact with Startups.

The venue will be Seed Infotech’s office in Erandawana.

The complete address is:
Seed Infotech
Nalanda building, Opposite Gandhi Lawns,
Erandwane, Pune

Many thanks to Naredra Barhate, CEO of Seed Infotech for arranging place for the meeting.

Footprint Ventures are an early stage venture capital fund, based out of India. (http://footprintventures.com/index.htm).

Zemanta Pixie