However, three of Pune’s top tech bloggers have weighed in with positive opinions about Google+

Arun Prabhudesai of trak.in was first to say that Google+ will be adopted by Businesses and Brands:

After Google’s half baked and feeble attempts at Social Networking earlier, Google Plus is a refreshingly fresh & honest attempt at making people’s lives Social.

and

The biggest factor that Google Plus takes care of – Individual Privacy. It probably has simplest of privacy policies and user can control everything as to what is supposed to be public and what is not. Yes, initially users do have to spend time in creating circles (aka groups of people), but once you are done with that, it becomes far more easier.

and

Google Plus adoption for Brands & Businesses might be slow initially, but over a period of time, it will surely catch on. It will be a place where Brands can put up their profiles, their “+ses” and it will be accessible to anyone and everyone without having to actually “follow” the brand.

After that, Mahendra Palsule, the Skeptic Geek, and Editor at TechMeme, wrote to say that Facebook and Quora should be worried. His main points are this:

• The future belongs to the “Interest Graph” of users complementing the “Social Graph”. Facebook does a bad job of capturing users interests. Google+ is taking steps in the right direction with Sparks.
• Quora should be worried because:
  > It was reported earlier [...] that code for Questions has been found in Google Plus. If this comes as a surprise, you haven’t understood Google’s ambitions with Emerald Sea.

Finally, Dhananjay Nene was initially lukewarm about Google+ (“good, but people will not shift from facebook for this”, and “circles are too complicated for average users”) but after spending some time with Google+, he has decided that Google+ is the social network of the future:

[Google+] is really building public / private, asymmetric networking built using social graphs based on friendships, work relationships, online discoveries and probably soon enough interest graphs as well. It is building the network that will be. While google wants to own the experience, it is liberal enough to publicly commit that the data is owned by the user. Combined with the awesome google portfolio and its evergrowing warchest built out of search advertising revenues – This is the network to beat.

You really should read the full articles that I have linked above. In fact, you should follow these guys on twitter (@dnene, @ScepticGeek, and @trakin) and follow them religiously.

Any Apple announcement from new products/services to the Worldwide Wide Developer Conference (WWDC) is often hotly anticipated by the media and the Apple faithful alike. The WWDC 2011 held on June 6^th this year was no exception. Steve Jobs (Apple CEO) took the stage to make a whole slew of announcements; most notable among them was iCloud, Apple’s vision of consumer centric cloud services.

Before the ink was even dry on the announcement, iCloud began to be touted as a Windows Killer , as a copy of Android Services, as the next big thing, attacked as not even having to do anything with cloud computing and even got Apple sued. By time all is said and done, gallons more ink will have flowed (or hundreds more blog posts will have been created) regarding iCloud. This post is an effort to separate the Facts from the Hype and provide some overall context on the implications of iCloud in different areas.

What is iCloud?

iCloud is Apple’s vision of a omnipresent cloud connection in all Apple devices that will seamlessly act as a sort of a “super synch” for different Apple applications. However it has a lot more features than just a remote storage mechanism such as DropBox. Let us look at this in a bit more detail:

(Note that as of today, iCloud is in private beta. The full public release has rather amorphously been defined as “in the Fall”. So everything that is known about iCloud is in the form of press releases from Apple/Developers given early access to it.)

There are 9 default services or (Apps) in the free version of iCloud:

Contacts - Your contacts will be synced to the cloud and shared between all of your devices.

Calendars - Calendars in the cloud supports calendars in the cloud, shared calendars and calendars pushed to all of your devices.

Mail - The new Mail service will include an @me.com account.

iBooks - your book purchases and places are stored across your devices.

Backup - Daily backups of your apps, music, camera roll, app data and more over WiFi.

Documents in the Cloud – You can download your documents, and edit it on multiple devices.

App Store – Your apps can be downloaded right to your new devices.

Photo Steam - A new built in feature will move your photos to the cloud so that you can easily share them with others on any Apple Device.

iTunes in the Cloud – Shows you all your songs, albums and artists you have purchased and download to your device. These are limited to only items purchased from iTunes to begin with.

Each iCloud consumer will be given a free 5GB of storage capacity for their mail, documents, and back-ups. A really interesting feature of the service is that for music, apps and books purchased from Apple, and the storage required by Photo Stream doesn’t count towards this 5GB total.

For the PhotoStream service, Apple will store the latest 1000 photos long-term while every new photo taken from any device will be stored for 30 days.

Apple really seems to be shooting for two things with iCloud:

1. Ubiquity: All iPods, iPhones, iPads that can be upgraded to iOS 5 and all Macs (MacBooks, and Desktops) with OS X Lion will be able to avail of iCloud. This will be at least tens of millions of users if not more. There will also be a Windows client (Windows 7 and up no XP support) that will support iCloud on non Apple desktops.
2. Simplicity: As presented, the iCloud service looks like it falls into the “Just works” category with minimal user meddling. If Apple can really pull-off this vision the simplicity would be the real killer feature of the service.

Is it cloud computing?

In a rather grumpy post Carl Brooks wrote: “Apple iCloud is not cloud computing.” He went to deride as “Nothing but Streaming Media”. (He has since updated his post to clarify that it has more capabilities).

Let us address this issue “Is Apple iCloud cloud computing?”

YES it most certainly is cloud computing.
Take a look at the figure that I created recently that shows a simplified cloud computing stack.

iCloud clearly fits in the top two layers – SaaS and the Client layer.

However there are those that define cloud computing more narrowly as “On-demand Infrastructure (IaaS) or Platform as a Service” in which case, No, iCloud is not strictly cloud computing from this angle. Keep in mind that by now the term “Cloud Computing” or “Cloud” has become so diluted as to be essentially meaningless, so the question raised is in-fact a very relevant one.

What are the challenges Apple faces?

The first and biggest challenge that Apple faces to iCloud is history. This is the fourth time Apple has tried its hand at internet services after failing in its three previous attempts. It first launched iTools way back in 2000 followed by .Mac and its most recent attempt was MobileMe. All the previous attempts were duds and Steve Jobs Apple CEO even admitted it on stage when he was announcing iCloud, calling MobileMe “not our finest hour”. The problem is rather simple – if used correctly the service should fade into the background and be seamless – but Apple is a master at splashy well-designed hardware and “just works”, well thought-thru software, neither of which directly align with iCloud. So the trick of getting it right will all be in the execution.

The second and somewhat lesser problem might be that Apple has underestimated the actual amount of data that its consumers will want to push thru iCloud. Steve Jobs took some pains to address this issue by showing slides with pictures of huge data centers at WWDC (Screen grabs):

And sleek next-gen hardware:

Apple is also aggressively investing in building datacenters, so, time alone will tell on this front.

Who is the competition?

Apple is essentially in a three horse race at this point with Consumer Cloud Services. The first and most obvious competitor is Google.

Google’s Android OS has provided much of the functionality of iCloud, namely

GMail and the related contact manager; Google Calendar, Google Docs, where you can view, edit and collaborate on Office-style documents, Picasa for images, Google Books and Google Music, and the Android AppStore.

In a way, iCloud is complete validation of Google’s strategy of Cloud hosted data and consumers with multiple endpoints such as Android based cell phones and Chrome Books. The one difference is that Apple touts “Apps” as the consumption medium of choice Google focuses on the browser as the ultimate medium of consumption. Google and Apple are now locked in bitter fight for consumer’s data and both are using the Cloud as the weapon of choice.

The Second challenger is the dark horse Amazon. Amazon has become the de-facto leader in the “traditional” Cloud computing space. It’s EC2 and other Amazon Web Services (AWS) offerings are the leaders in the IaaS space. What is not as well known is that it is also quietly ramping up its consumer cloud services strategy. The recently announced Cloud drive is just the start with rumored plans for Amazon branded Tablets, Amazon will be in a position to challenge Apple all across the cloud stack for dominance.

The biggest consumer name missing from the list? Microsoft. It was late to the Tablet space after Apple revitalized it with the launch of the iPad. It was unsuccessful in the mobile phone space until its recent moves towards Windows 7 based phones. This is the challenge it must now confront to be relevant again in the Consumer cloud services space.

What are the likely implications?

At the launch of the iPad 2, Steve Jobs had famously declared that we are in the Post-PC era, implying that consumers had moved on from PCs and were ready to embrace more portable devices as their main computers. The iCloud vision would seem to make that a reality.

Earlier, whenever you purchased an iPhone/iPad, the very first thing the device would prompt you to do was sync with iTunes on your PC/Mac. With iCloud this will no longer happen, just type-in your credentials and you are synched with all your data and apps – truly a Post-PC experience.

Another obvious result of this announcement is a phenomenon I like to term “Consumerization of the Cloud”. This announcement is likely to associate the words “cloud computing” with Apple in a very sticky way in the minds of regular (non-tech) consumers. The next time one of us says we work in cloud computing, one sure question is “Is that like the Apple iCloud thing?” As if the cloud hype was not high enough already, this announcement has undoubtedly pushed it to stratospheric (cloudy) levels. However the positive side of this is that Cloud Computing will now become much more main stream than ever before.

About the Author – Amit Naik

Amit Naik works as an Architect with BMC Software. He builds performant cloud solutions with a focus on heterogeneity and monitoring across different virtualization and provisioning vendors in the cloud computing space. His main focus is the Architecture and Design of BMC solutions with emphasis on building highly-scalable systems with REST and other SOA interfaces.

Amit has a Bachelor’s degree from College of Engineering Pune and a Master’s degree from Purdue Univ., West Lafayette. He has more than 15 years of experience in the IT industry, much of it in the USA, across a variety of Technical and Techno-Managerial roles.

Suhas Kelkar, who is the Director of the Innovation Team at BMC Software India has done this port-mortem of the incident.

Note: if you have trouble viewing the video embedded above click here. Suhas has created this video+slides presentation using kPoint another Pune-based cloud software product.

A couple of years back, Suhas had written an article for PuneTech titled Musings on Why Cloud Computing will Prevail which is also interesting reading in this context.

How to prevent such outages from affecting your own infrastructure? A few days after the outage, Dhananjay Nene, Chief Architect at Vayana, and also a consulting software architect, wrote an article arguing that the cloud just got stronger as a result of the AWS outage.

Here are his recommendations:

AWS has multiple availability zones. An application should ideally leverage at least two. If you read the Netflix presentation I referred to, Netflix apparently uses three. Do not assume the servers will not go down. Assume it is possible that at least one availability zone could go down. Make sure you have the systems to quickly activate, systems in the alternative availability zone. For that you will need to find ways to keep data current across availability zones. Also find ways to ensure you have the ability to quickly switch to and fro between availability zones. More advanced options could include concurrently active systems across availability zones or those spread across AWS regions or even between AWS and other vendors.

Read the whole article, and also check out Dhananjay’s blog.

This is NOT a post about differences between Rails 2.x and Rails 3 but these are some musings about what I love in Rails. A lot of goodies come ‘in the box’ (I hate saying out-of-the-box) with Rails3 and some of them have been there since early version of Rails but somehow less frequently used or talked about. I spoke about this at Techweekend #8 and the presentation is here.

Bundler

Ever had a production system crash one day – without any code deployment or even anyone logging in. After the initial ‘its not me’ excuses, one system administrator says ‘Hey, I had updated the system libraries’. Having been burnt already before (hopefully), you check on the system and find that some system library has been upgraded and some gems have been upgraded and that is causing incompatibility between other gems etc. We had the case where rack (1.0.1) was upgraded to rack (1.1) causing incompatibility with the Rails gem we were running! The fix is simple — upgrade or downgrade your gems or libraries and you’re on your way. A few days later, another developer needs to deploy a simple sinatra application. He takes the latest version which requires rack > 1.1 and it automatically upgrades the gem. Boom! Your Rails app crashed again.

Did I hear you freeze the gems? Nah – not a good approach, as it causes your application deployment bundle to be huge and ‘frozen’. Every application you use would require to freeze gems and this does not really solve your problem.

Bundler (by Yahuda and Carl) built this awesome gem which is now the de-facto standard for any Rails application. In fact, it was so cool, its not Rails 2.x compatible and very highly recommended. You can now specify your dependencies in a Gemfile and prevent any clashes with any other gem versions and their dependencies. Since the gems are installed in the system default location (not frozen in your app), it means it us re-usable and version friendly!


source "http://rubygems.org"

group :test do # only in test environment
gem "rspec", :require => "spec"
end


UJS

Unobtrusive Java Script has been around for ages now but Rails lingered with prototype.js. Now, with the awesome features of JQuery, we can easily use UJS to solve some common and nagging problems:

• Avoid submit if button clicked twice!
• Make non-get requests from Hyperlinks!
• Submit form data via Ajax

Add :remote => true to hyperlink, forms and other elements. This adds the data-remote=true in your html properties. The ‘live’ JQuery function binding kicks in and sets up the events for various elements. Simple and awesome – this is available here.

XSS

Cross site scripting has been a pain to handle for a long time. Rails does this under covers – you dont event need to know too many details:

protect_from_forgery is automatically added during basic rails project creation. This ensures that every form created by the application has an authenticity_token as a hidden data field. During a post request, this is verified and thus ensures that the source of the form creation is the same server – this avoid session stealing where a malicious form is posted to your server using an authenticated user’s session!

While using UJS, you need to add csrf_meta_tag in your layout to avoid silent Ajax errors.

SQL injection is cleanly avoided with new where syntax:

# Wrong
where("user_name = '#{user_name}' AND "password = '#{password}'").first

# Correct and clean
where(:user_name => user_name, :password => password).first


In Rails3, all html spewed out is HTML SAFE! So, you cannot leave gaps for non-HTML safe code, even by mistake! If indeed you do trust the source, you can use the ‘raw’ method to spew out the HTML as is.

Rails Eager Loading

The N+1 query problem is fairly common and largely ignored until you hit serious performance issues.  Straight out of the Rails guide, consider the case

clients = Client.all.limit(10)

clients.each do |client|
puts client.address.postcode
end

There are 11 queries fired here. Using the :includes construct, Rails does eager loading like this:

clients = Client.includes(:address).limit(10)

clients.each do |client|
puts client.address.postcode
end


Here only 2 queries are fired as Rails includes the address relationship too while fetching the client objects.

Transliteration / Babosa

What happens to your permalinks if a user enters the information in Arabic? We faced exactly this issue and were asked by our client to prevent input which is not English. Woh! ActiveSuppprt in Rails3 addresses a lot of these transliteration issues:

"Jürgen Müller".to_slug.transliterate.to_s #=> "Jurgen Muller"

Performance using Identity Map

The awesomeness of Rails progression – As of this inclusion the Identity Map pattern is now part of Rails 3 caching mechanism. An identity map is a design pattern used to improve performance by providing a in-memory cache to prevent duplicate retrieval of the same object data from the database, in context of the same request or thread.

Optimistic Locking

A really old concept which has been there since REALLY early versions of Rails. This is commonly overlooked but is critically important when it comes to concurrent request processing. By adding a ‘lock_version’ field in the table, Rails automatically kicks into optimistic locking mode and prevents concurrent writes when the data is stale. The StaleObjectError is raised incase the lock_version is not the same as when it was read.

Named Scopes

This is almost cliched now Mames scopes were added since Rails 2.1. Its one of the things I love about Rails. The scopes are chained together and the query is fired only when the data is really needed. This is excellent for report filters! Adding new filters is a breeze as its only one of the scopes to be chained. Remember that scopes do not return an Array but an association object like has_many. That is how they can be chained to other scopes.

I’m pretty sure I have missed some things here. Do comment on what features you like best about Rails3!

Designing a SaaS software stack poses challenges that are very different from the considerations for host-based software design. The design aspects for performance, scalability, reliability of SaaS with lots of servers and lots of data is very different and interesting from designing a software that is installed on a host and is used by that host.

Here I list the top 5 design elements for Cloud Based SaaS.

High availability

SaaS software stack is built on top of several disparate elements. Most of the times these elements are hosted by different software vendors, such as Rackspace, Amazon, Akamai, etc. The software stack consists of several layers, such as – application server, database server, data-mining server, DNS, CDN, ISP, load-balancer, firewall, router, etc. Highly availability of SaaS actually means thinking about the high availability of all or most of these components. Designing high availability of each of these components is a non-trivial exercise and the cost shoots up as you keep on adding layers of HA. Such design requires thinking deeply about the software architecture and each component of the architecture. Two years back I wrote an article on Cloud High Availability, where I described some of these issues, you can read it here.

Centralized Manageability

As you keep on adding more and more servers to your application cluster the manageability gets hugely complex. This means:

• you have to employ more people to do the management,
• human errors would increase, and
• the rate at which you can deploy more servers goes down.

And, don’t just think of managing the OS on these servers, or these virtual machines. You have to manage the entire application and all the services that the application depends on. The only way to get around this problem is to have centralized management of your cluster. Centralized management is not an easy thing to do, since every application is different, making a generalized management software is oversimplifying the problem and is not a full solution.

Online Upgradability

This is probably the most complex problem after high availability. When you have a cluster of thousands of hosts, live upgradability is a key requirements. When you release a new software revision, you need to be able to upgrade is across the servers in a controlled way, with the ability of rolling it back whenever you want – at the instant that you want, across the exact number of servers that you want. You would also need to control database and cache coherency and invalidation across the cluster is a controlled way. Again, this cannot be solved in a very generic way; every software stack has its own specificity, which needs to be solved in its own specific ways.

Live testability

Testing your application in a controlled way with real traffic and data is another key aspect of SaaS design. You should be able to sample real traffic and use it for testing your application without compromising on user experience or data integrity. Lab testing has severe limitations, especially when you are testing performance and scalability of your application. Real traffic patterns and seasonality of data can only be tested with real traffic. Don’t start your beta until you have tested on real traffic.

Monitor-ability

The more servers and applications that you add to your cluster the more things can fail and in very different ways. For example – network (NIC), memory, disk and many other things. It is extremely important to monitor each of these, and many more, constantly, with alarms using different communication formats (email, SMS, etc.). There are many online services that can be used for monitoring services, and they provide a host of difference services and have widely varying pricing. Amazon too recently introduced CloudWatch, which can monitor various aspects of a host such as CPU Utilization, Disk I/O, Network I/O etc.

As you grown your cluster of server you will need to think of these design aspects and keep on tuning your system. And, like the guys at YouTube said:

Recipe for handling rapid growth

    while (true)
    {
        identify_and_fix_bottlenecks();
        drink();
        sleep();
        notice_new_bottleneck();
     }

About the Author – Mukul Kumar

Mukul Kumar is the Co-Founder & Senior Vice President Engineering at PubMatic. PubMatic, an online advertising company that helps premium publishers maximize their revenue and protect their brands online, has its Research & Development center in Pune.

Mukul is responsible for PubMatic’s Engineering team and resides in Pune, India. Mukul was previously the Director of Engineering at PANTA Systems, a high-performance computing startup. Before that he was at VERITAS India, where he joined as the 13th employee and helped it grow to over 2,000 individuals. Mukul has filed for 14 patents in systems software, storage software, and application software. Mukul is a graduate of IIT Kharagpur with a degree in Electrical Engineering.

Mukul is very passionate about technology, and building world-class teams. His interests include architecting scalable and high-performance web-applications, handling and mining massive amounts of data and system & storage architecture.

Mukul’s email address is mukul at pubmatic.com.

The smartphones market is very fragmented.

In 3Q2010, Symbian had 37% of the smartphone market, Android was second with 25% (it was at 2% 18 months ago), and iOS in third place with 16%. RIM (Blackberry) was next. Windows was losing.

So, what should a developer do? Which to target?

I talked to Romin Irani of Xoriant about this problem, and whether HTML5 is the answer to these issues. My key takeaway’s from this conversation were:

• HTML5 is here already. I was under the impression that HTML5 is something that will arrive sometime in the near future. Romin pointed out that HTML5 support is pretty good even today, especially if you’re thinking of mobile phone browsers.
• But HTML5 not the answer to all your problems. If you need access to device sensors, you’re probably better off with a native app. If you want access to the appstore/marketplace, then you need a native app. HTML5 doesn’t qualify!
• If you’re a new startup, and you want to build a mobile app, what should you do? These are the guidelines:
  • If you don’t need device sensors, and don’t need to be in the appstore/marketplace, strongly consider a HTML5+CSS+JavaScript app
  • If you want to go after the US market, you must have an iPhone native app. (Maybe followed by Android)
  • If you want to go after Europe market, then you will need to have a Nokia based native app, just for the sheer numbers they have

Rohit Nayak of Talentica had talked about the use of cross-platform app development frameworks like Titanium and PhoneGap. Both allow you to write apps in JavaScript. Titanium cross-compiles them to native apps on each platform. PhoneGap uses a modified version of the browser so that your app is HTML+CSS+JavaScript, but there are modifications that allow you to access native phone features (like sensors).

There are some limitations, and such apps aren’t as good as native apps.

So, would he really recommend the use of PhoneGap/Titanium for developing apps? Rohit had this to say:

• Titanium and PhoneGap are rapidly getting better and better. More and more apps built using them are showing up on the android marketplace.
• If you already know JavaScript, and need to get to the market quickly, you should definitely consider using one of these tools
• If you don’t really need advanced native features of any specific platform, then it makes a lot of sense to go this route
• If you are a software outsourcing company that’s building apps for third parties, you should seriously considering building a team that uses Titanium. For most of your customers, you’ll be able to quickly complete an app that satisfies them. Otherwise, you’re faced with a nightmare – you’ll need to build teams with expertise in each of the major platforms, and this is almost impossible to do with today’s attrition.

The last few points seem very similar to the advantages of HTML5, so I asked Rohit whether PhoneGap/Titanium had any advantages over HTML5. Answer:

• PhoneGap/Titanium generally support more native features than HTML is planning on supporting
• An app built Titanium/PhoneGap can go on the appstore/marketplace.
• An HTML5 app necessarily requires you to have a “cloud” presence – a web server and an API, and supporting all the online connections. PhoneGap/Titanium application does not require any of that.

But first some background, taken from previous PuneTech articles about KQInfoTech.

What is KQInfotech?

KQInfoTech is a Pune company that's trying to combine mentorship programmes for technology students, along with technology services to the industry and open source projects. Click on the logo to see other PuneTech articles on KQInfoTech's various initiatives.

Pune-based KQInfoTech is an organization started by Anurag Agarwal and Anand Mitra, both of whom chucked high-paying jobs in the industry because they felt that there was a desperate need to work on the quality of students that is being churned out by our colleges. For the 2 years or so, they have been trying various experiements in education, at the engineering college level. All their experiments are based on one basic premise: students’ ability to pay should not be a deterrent – in other words, the offerings should be free for the students; KQInfoTech focuses on finding alternative ways to pay for the costs of running the course. As a part of this initiative, they provide services to industry, and take on open source projects, and the students in their mentorship program actually do the work under their guidance.

What is ZFS?

ZFS – the Zettabyte File System – is an enormous advance in capability on existing file systems. It provides greater space for files, hugely improved administration and greatly improved data security. Wikipedia has this to say:

ZFS is a combined file system and logical volume manager designed by Sun Microsystems. The features of ZFS include support for high storage capacities, integration of the concepts of filesystem and volume management, snapshots and copy-on-write clones, continuous integrity checking and automatic repair, RAID-Z and native NFSv4 ACLs.

Why ZFS on Linux by KQInfotech?

ZFS is arguably one of the best file-systems available today, and Linux is one of the most widely used operating systems for servers by new startups. So, having ZFS available on Linux would be great. And, With many years of experience in Veritas building VxFS, another one of best file-systems in the world, the founders of KQInfoTech do have the technical background to be able to do a good job of this.

At this point, ZFS is not available on Linux. See the Linux section of ZFS entry on Wikipedia for more details.

So what does this port contain?

This port of ZFS is an extension to the port of DMU layer by Brian Behlendorf. We have added the missing ZPL layer to Brian’s port. With this addition it becomes possible to mount the zfs filesystem on linux and leverage ZFS’s features on linux.

What next?

If you’re interested in participating in the beta and helping out, or you’re one of the people whose business would really be helped by having ZFS available on Linux, apply for the beta, or get in touch with KQInfoTech: zfs-query@kqinfotech.com.

Also, check out the FAQ.

Kalpak Shah is the founder and CEO of Clogeny, a company that does consulting & services in cloud computing. His talk is about the various choices available in cloud computing today, and how to go about picking the one that’s right for you.

These are the slides that were used by Kalpak for this talk. Click here if you can’t see the slideshow above.

Kalpak’s definition of a cloud:

• If you cannot add a new machine yourself (i.e. without making a phone call or email), then it’s just hosting, not cloud computing
• If you cannot pay as you go (i.e. pay per use) it is not cloud computing
• If you don’t have APIs which allow integration with the rest of your infrastructure/environment, then it is not a cloud

Kalpak separates out cloud infrastructure into three parts, and gives suggestions on how to choose each:

Infrastructure as a service

Basically allows you to move your local server stuff into the cloud. Examples: Amazon EC2, Terremark vCloud, GoGrid Cloud, Rackspace Cloud

You should check:

• Support and Helpdesk. Is it 24×7? Email? Phone?
• Hardware and Performance. Not all of them are the same. Amazon EC2 not as good as Terremark.
• OS support. Which OS and distributions are supported. Is imaging of server allowed? Is distribution and re-selling of images allowed? Not everybody allows you to save the current state of the server, and restart it later on a different instance.
• Software availability and partner network. Example, Symantec has put up their anti-virus software for Windows on EC2. How many such partners are available with the provider you’re interested in? (EC2 is far ahead of everybody else in this area.)
• APIs and Ecosystem. What APIs are available and in what languages. Some providers don’t do a good job of providing backward compatibility. Other might not be available in language of your choice. EC2 and Rackspace are the best in this area.
• Licensing is a big pain. Open source software is not a problem, but if you want to put licensed applications on the cloud, that is a problem. e.g. IBM Websphere clustering is not available on EC2. Or Windows licenses cannot be migrated from local data center to the cloud.
• Other services – How much database storage are you allocated? What backup software/services are available? What monitoring tools? Auto-scaling, load-balancing, messaging.

Kalpak has put up a nice comparison of Amazon AWS, Rackspace, GoGrid and Terremark on the above parameters. You can look at it when the PPT is put up on the IndicThreads conference website in a few days.

Platform as a Service

This gives you a full platform, not just the hardware. You get the development environment, and a server to upload the applications to. Scalability, availability managed by the vendor. But much less flexibility than infrastracture-as-a-service. You are stuck with the programming language that the PaaS supports, and the tools.

For example, Google AppEngine. Which is available only for Python and Java. Or Heroku for Ruby + Rails.

PaaS is targeted towards developers.

Software as a Service

This gives you a consumer facing software that sits in the cloud. You can start using the software directly, or you can extend it a bit. A business layer is provided, so you can customize the processes to suit your business. Good if what is provided fits what you already want. Not good if your needs are rather different from what they have envisoned.

Examples: Sales Force, Google Apps, Box.net, Zoho

Storage as a Service

Instead of storing data on your local disks, store it in the cloud. Lots of consumer adopton, and now enterprise usage is also growing. No management overhead, backups, or disaster recovery to worry about. And pay either flat fees per month, or by the gigabyte.

Examples: Mozy from EMC. Amazon S3. Rackspace CloudFiles. Carbonite. DropBox.

Comparing PaaS and SaaS

Some choices automatically made for you based on development language and available skill sets. Python + Java? Use Google AppEngine. Ruby on Rails? Use Heroku. Microsoft shop? Use Azure.

Other ways to compare are the standard ones: size of vendor and ecosystem maturity. Tools, monitoring, connectors, etc. e.g. AppEngine has a Eclipse plugin, so if your developers are used to Eclipse (and they should be!) then this is very good. Another question to ask is this – will the vendor allow integration with your private cloud? Can you sync your online hosted database with your local database? If yes, that’s great. If not that can be very painful and complicated for you.

Interesting Private Cloud Platforms

These are some interesting private cloud platforms

• Eucalyptus: open source IaaS cloud computing platform.
• VMWare Cloud: Partnered with Terremark. Expensive but worth it.
• Appistry: Allows installing of the platform on Amazon EC2, or in your private data center. Allows application deployment and mgmt, various services across the stack IaaS, PaaS, SaaS. Integration with SQL Azure, SharePoint, Dynamics CRM. Visual Studio development and testing. Supports multiple development languages.

Database in the cloud

You can either do regular relational databases (easy to use, everybody knows them, scaling and performance needs to be managed by you). Or do NoSQL – non-relational databases like SimpleDB (Amazon), Hadoop (Yahoo), BigTable (Google). They’re supported and managed by cloud vendor in some cases. Inherent flexibility and scale. But querying is more difficult and less flexible.

Business Considerations

Licensing is a pain, and can make the cloud unattractive if you’re not careful. So figure this one out before you start. SLAs are around 99.9% for most vendors, but lots of fine print. Still evolving and might not meet your standards, especially if you’re an enterprise. Also, if SLA is not being met, vendor will not tell you. You have to complain and only then they might fix it. Overall, this is a grey area.

Pricing is a problem – it keeps changing (e.g. in case of Amazon). So you can have problems estimating it. Or the pricing is at a level that you might not understand. e.g. pricing of 10 cents per million I/O requests. Do you know how many I/Os your app makes? Maybe not.

Compliance might be a problem – your government might not allow your app to be in a different country. Or, for banking industry, there might be security certification required (for the vendor) before the cloud can be reached.

Consider all of these before deciding whether to go to a cloud or not.

Summary

IaaS gives you the infrastructure in the cloud. PaaS adds the application framework. SaaS adds a business layer on the top.

Each of these are available as public clouds (that would be somewhere out there on the world wide web), or private clouds that are installed in your data-center. Private is more expensive, more difficult to deploy, but your data is in your premises, you have better (local) connectivity, and have more flexibility. You could also have a hybrid cloud, where some stuff is in-house and some stuff in the public cloud. And if your cloud infrastructure is good enough, you can easily move computation or data back and forth.

Kalpak Shah, CEO of Clogeny, gave a broad overview of the various options available in cloud computing infrastructure, platforms and software, and the questions you need to ask before you choose the one for you.

About the Speaker – Kalpak Shah

Kalpak Shah is Founder & CEO of Clogeny Technologies Pvt. Ltd. and guides the overall strategic direction of the company. Clogeny is focused on providing services and consultancy in the cloud computing and storage domains. He is passionate about the ground-breaking economics and technology afforded by the cloud computing platforms. He has been working on various cloud platforms including IaaS, PaaS and SaaS vendors.

You can also follow @clogeny and @kalpakshah on twitter.

Live-blog of a talk by Dr. Vipin Chaudhary, CEO of CRL, on High Performance Computing at Institute of Engineers, Pune. CRL are the makers of Eka, one of the world's fastest privately funded supercomputers. For more information about HPC and CRL, click on the photo above.

• Commonly associated with scientific computing
• Only used for large problems
• Expensive
• Applicable to niche areas
• Understood by only a few people
• Lots of servers and storage
• Difficult to use
• Not scalable and reliable

This is not the reality. HPC is:

• Backbone for national development
• Will enable economic growth. Everything from toilets to potato chips are designed using HPC
• Lots of supercomputing is throughput computing – i.e. used to solve lots of small problems
• “Mainstream” businesses like Walmart, and entertainment companies like Dreamworks Studioes use HPC.
• _(and a bunch of other reasons that I did not catch)

China is really catching up in the area of HPC. And Vipin correlates China’s GDP with the development of supercomputers in China. Point: technology is a driver for economic growth.  We need to also invest in this.

Problems solved using HPC:

• Movie making (like avatar)
• Real time data analysis
  • weather forecasting
  • oil spill impact analysis
  • forest fire tracking and monitoring
  • biological contamination prediction
• Drug discover
  • reduce experimental costs through simulations
• Terrain modeling for wind-farms
  • e.g. optimized site selection, maintenance scheduling
  • and other alternate energy sources
• Geophysical imaging
  • oil industry
  • earthquake analysis
• Designing airplanes (Virtual wind tunnel)

Trends in HPC.

The Manycore trend.

Putting many CPUs inside a single chip. Multi-core is when you have a few cores, manycore is when you have many, many cores. This has challenges. Programming manycore processors is very cumbersome. Debugging is much harder. e.g. if you need to get good performance out of these chips then you need to do parallel, assembly programming. Parallel programming is hard. Assembly programming is hard. Both together will kill you.

This will be one of the biggest challenges in computer science in the near future. A typical laptop might have 8 to 10 processses running concurrently. So there is automatic parallelism, as long as number of cores is less than 10. But as chips get 30, 40 cores or more, individual processes will need to be parallel. This will be very challenging.

Oceans of Data but the Pipes are Skinny

Data is growing fast. In sciences, humanities, commerce, medicine, entertainment. The amount of information being created in the world is huge. Emails, photos, audio, documents etc. Genomic data (bio-informatics) data is also huge.

Note: data is growing way, way faster than Moore’s law!

Storing things is not a problem – we have lots of disk space. Fetching and finding stuff is a pain.

Challenges in data-intensive systems:

• Amount of data to be accessed by the application is huge
• This requires huge amounts of disk, and very fat interconnects
• And fast processors to process that data

Conventional supercomputing was CPU bound. Now, we are in the age of data-intensive supercomputing. Difference: old supercomputing had storage elsewhere (away from the processor farm). Now the disks have to be much closer.

Conventional supercomputing was batch processed. Now, we want everything in real-time. Need interactive access. To be able to run analytic and ad hoc queries. This is a new, and difficult challenge.

While Vipin was faculty in SUNY Buffalo, they started an initiative for data-intensive discovery initiative (Di2). Now, CRL is participating. Large, ever-changing data sets. Collecting and maintaining data is of course major problem, but primary focus of Di2 is to search in this data. e.g. security (find patterns in huge logs user actions). This requires a new, different architecture from traditional supercomputing, and the resulting Di2 system significantly outperforms the traditional system.

This also has applications in marketing analysis, financial services, web analytics, genetics, aerospace, and healthcare.

High Performance Cloud Services at CRL

Cloud computing makes sense. It is here to stay. But energy consumption of clouds is a problem.

Hence, CRL is focusing on a green cloud. What does that mean?

Data center optimization:

• Power consumption optimization on hardware
• Optimization of the power system itself
• Optimized cooling subsystem
• CFD modeling of the power consumption
• Power dashboards

Workflow optimization (reduce computing resource consumption via efficiencies):

• Cloud offerings
• Virtualizations
• Workload based power management
• Temperature aware distribution
• Compute cycle optimization

Green applications being run in CRL

• Terrain modeling
• Wind farm design and simulation
• Geophysical imaging
• Virtual wind tunnel

Summary of talk

• Manycore processors are here to stay
  • Programmability have to improve
  • Must match application requirements to processor architecture (one size does not fit all)
• Computation has to move to where the data is, and not vice versa
• Data scale is the biggest issue
  • must co-locate data with computing
• Cloud computing will continue to grow rapidly
  • Bandwidth is an issue
  • Security is an issue
  • These issues need to be solved
  ]]> http://punetech.com/live-blog-overview-of-high-performance-computing-by-dr-vipin-chaudhary/feed/ 0 http://punetech.com/the-rise-and-fall-of-google-wave/ http://punetech.com/the-rise-and-fall-of-google-wave/#comments Tue, 10 Aug 2010 03:25:24 +0000 Navin Kabra http://punetech.com/?p=2399 (In this guest post, Markus Hegi, partially-Pune-based CEO of partially-Pune-based company Colayer, laments the death of Google Wave, and points out that the concept behind the Wave is right. Google should have re-launched a new, improved Wave, he feels, because the world does need a paradigm shift in business communications. This article is a shortened & modified version of a post published on ex.colayer.com)
  

  Google's revolutionary new communication and collaboration platform Wave is dead. Did it deserve to die? Markus Hegi thinks not. He believes that sooner or later, the world needs a Wave like system. Image via Wikipedia

  3 days ago, Google announced that it would stop the development of Wave and would stop supporting it by the end of the year. Even though the buzz about Wave and the (visible) progress of Wave was low for the last few months, the shut down is surprising: I would have expected a re-launch, a change of the architecture, integration with gmail – anything, but not a complete halt – The concept behind Wave is right and ahead of its time – and Google could have been a leading player in this space!

  When I looked at Wave for the first time right after the announcement one year ago, it struck me, how similar the concepts were to what we were working for years with Colayer. I started Colayer in 99 – suffering myself the mess of email communication. As a travelling business consultant I was convinced, that this can not be the way we will communicate in future! This is fundamentally wrong! – I mean: the basic idea of SENDING information on the web is wrong! (You GO TO and ARE ON Facebook, twitter, yahoo – you don’t ‘download’ it.) Google Wave addresses exactly these same issues.

  We were excited to see, what approach Google would take to implement the new paradigm of online communication – But also realized quickly, that this product in this stage would not be usable for 3 main reasons:

  1. The Technical Architecture was too heavy and complex
  2. The Operability – The way to operate the tool was limiting
  3. The Notification – the way the users would be notified about updates in their many waves.

  If you would use this product in a real world scenario with heavy communication, it would not work! – But Wave was at its very start. We thought Google would quickly realize the problems and implement solutions for it – and with their market power, Google would be able to initiate the paradigm shift in online communication.

  But after the Wave launch, it seemed that innovation stopped. Yes, there was development, improvements & many extensions were released. But the above 3 problems were not addressed. They couldn’t be solved through improvements or extensions, but needed fundamental shifts in the product design – which never happened. And as many users seemed to loose patience too, Google pulled the plug for poor user adoption after only one year.

  What went wrong? – Gartner has a valid point: “Startup innovation” has simply no place in a large enterprise software company. Well, this is not exactly what Gartner writes, but this is essentially the meaning: Either you are in the business of breaking & paradigm shifting innovation (Startups), or you are serving a large base of enterprise customers – Both together is almost impossible, because there is no breaking innovation, without messing up with your customers. After Wave was launched, even though it was still tagged as ‘beta’, the team could not just say to its 100’000 users: “you know, we just realized that the architecture has a fundamental problem – lets start it all over again …!” – which we, in a small company did several times …

  Maybe another problem of Wave was, that Google choose the wrong market: Wave was intended for the broad consumer market, as well as for enterprises – But the paradigm shift happens elsewhere first: If you observe today’s kids and young nerds, you can imagine, how the next generation of businesses will use online communication: Email for them is ‘lame’ and just used for communication with outsiders, older people and the ‘conservative’ business world. Why would you need email anyway in a world of Facebook & Foursquare?

  After 10 years, we are still in the beginning of the massive paradigm shift of online communication. I am eager to see, who will join the journey next!

  About Google Wave

  Wave is a web application for real-time communication and collaboration.

  (See one of the most popular videos explaining the basic concepts of Wave)

  Announced in May 2009, Wave attracted a lot of attention for a couple of months. The project was stopped by Google after just a little more than one year for poor user adoption.

  About the author – Markus Hegi

  Markus Hegi founded Metalayer (now renamed to Colayer) 10 years ago. The Colayer platform is a software technology to create collaborative web sites.

  Colayer is a Swiss-Indian company with headquarters in Zurich, Switzerland and development center in Pune, India. Markus ‘commutes’ since 10 years between Zurich and Pune and spends almost half of his time here in Pune. See his linked-in profile, or follow him on twitter.

  About Colayer vs Google Wave:

  See an overview of articles about Colayer vs Google Wave on colayer.com.

  
  ]]> http://punetech.com/the-rise-and-fall-of-google-wave/feed/ 14