Category Archives: Technology

TechWeekend: Three tech talks on Google Android, 4:30pm, 1st Aug

Events, Technology, , , ,
Image representing Android as depicted in Crun...
Image via CrunchBase

What: TechWeekend featuring “What makes Google Android different from other systems, and from regular Java” by Navin Kabra, “Maps, GPS and sensors in Android” by Rohit Ghatol, with a demo on an Android G1 phone, and more
When: Saturday, 1st August, 4:30pm
Where: Symbiosis Institute of Computer Studies and Research, Atur Centre, Model Colony. Map.
Registration and Fees: This event is free for all to attend. Please register here.

What makes Google Android different from other systems – Navin Kabra.

Google’s Android is a brand new platform for mobile phones, and has been created from scratch specifically for this purpose. This means that it is a “modern” system that does not suffer from any legacy issues, and has taken the best ideas from various other projects to build a system that is arguably better than any of the other, competing, systems. Thus, for example, it uses the Java language as the development language, but has rejected the rest of the Java ecosystem. Specifically it uses a compeletely new virtual machine (Dalvik) which is redesigned with mobiles in mind – and has a number of very interesting design decisions that we will discuss. Similarly, the Android application framework represents a departure from the traditional way of doing things, and has a learning curve, but once you get used to it, it is great, especially for allowing different apps to share data, code, and in general co-operate.

We will explore and discuss this and various other design decisions in Android.

Maps, GPS and Sensor Capability – Rohit Ghatol

Rohit Ghatol is a co-founder of the Pune Google Technologies User Group (PuneGTUG), and one of the early adopters of the Google Android platform. He has already built a number of applications on Android, and is working on some interesting ideas in this area. In this talk, he will be discussing the mapping in the Android framework, and how the GPS and sensor capabilities can be combined with it to give powerful and rich experiences to users. He will be using his Google Android G1 phone to demonstrate these capabilities.

There will be one more speaker, but unfortunately, the details of that talk were not ready in time for this announcement. Please check back in a day for that update.

Also check out Dhananjay Nene‘s slides from the first TechWeekend: “REST Explained.”

Reblog this post [with Zemanta]

Too Soon to call for a Semiconductor Recovery?

punechips, Technology, ,

(This post by Abhijit Athavale, the driving force behind PuneTech’s PuneChips forum, is cross-posted from his blog.)

Are we really out of the most dreaded recession after the great depression and on our path to recovery? The answer might be a resounding yes if we look at the recent crop of earnings reports from technology companies, and other data emerging from market analysis firms. In fact, the NASDAQ composite is already into deeply positive territory on the expectations of an imminent recovery.

Revenue and Inventory Declines from 2008 to 2009
Revenue and Inventory Declines from 2008 to 2009

However, a quick dissection of recent earnings announcements from a few prominent semiconductor companies shows that we are not out of the woods yet. The chart above shows that sales numbers are still horribly bad compared to last year, and inventories have declined sharply as well. This means that companies have managed to satisfy the underlying demand by using up their inventories, and will or already have started ordering replacement units from fabs. This explains why the fabs have reported better numbers than the IC vendors. In a normal growth phase, every company wants to keep its channel well stocked to feed demand. This is not the case now as companies are drawing down on finished stock to reduce costs and improve margins. So much for the expected recovery in demand …

Where does this demand come from ultimately? The consumer, of course. The US, Europe (as a whole) and Japan are the three largest economies in the world.

People living in these countries constantly keep (or, at least used to) buying new things such as laptops and PCs, cell phones, gaming consoles, automobiles, audio and video equipment, personal media players, TVs, toys, and other gadgets. Companies that manufacture consumer electronics ICs directly feed this demand. Consumers also buy services such as wireless and wireline voice and data, broadband, cable and satellite TV feeds. This in turn generates demand for the entire telecom, datacom and wireless infrastucture and service providers. Infrastructure IC manufacturers are responsible for suppplying this demand. In the US alone, the consumer is 70% of the entire economy. The consumer is also a major portion of the EU and Japanese economies. The consumption in all three major economies in the world has cratered, as evidenced by the continuous 6 to 9 month declines in the exports from China, Taiwan and Japan, major export reliant countries.

The real demand will not be back till the consumer starts spending again. And, that is almost impossible in this economy. Take the US for example – official unemployment is at ~11% (some say, it is near 16% unofficially), one in ten homeowners is underwater, and credit lines have been severely downgraded by banks. Three major sources of funds used by consumers have been seriously compromised. To use a banking term, the consumer is now de-leveraging and getting his house in order. There will be no more spending sprees till the US savings rates are back to historical levels. The EU and Japan are in even worse situations as their banks are still over leveraged and that still needs to be played out a la Bear Sterns and Lehman Brothers.

My belief is that the semiconductor industry and the companies that serve the semiconductor industry are not coming back till personal finances get better. That is not likely to happen for a while. Some analysts are calling Q2, 2009 as the bottom, which very well may be so, but I don’t expect to see a quick recovery from this point on. In fact, the worsening unemployment and foreclosure numbers may push us back into a double dip recession. If not, the recovery will be tepid and totally unspectacular.

Reblog this post [with Zemanta]

Changing Landscape of Data Centers

In Depth, Technology, , , ,

Today’s post is a guest post by Suhas Kelkar, the Head of Innovation & Incubation Lab at BMC Software India. Prior to BMC he was the Vice President of Product Management at Digite, an enterprise software company in the field of Project Portfolio Management. See his linked-in profile for details.

I had an opportunity to speak at the very first BMC India Technical Event held in Bangaluru on June 11th, 2009. At this event I talked about the changing landscape of data centers. This article is an excerpt of the talk intended to facilitate understanding of the presentation. The entire presentation is available here.

There are many factors causing the landscape of data centers to change. There are some disruptive technologies at play namely Virtualization and Cloud Computing. Virtualization has been around for a while but only recently it has risen to the level of making significant impact to data centers. Virtualization has come a long way since VMware first introduced VMware Workstation in 90s. The product was initially designed to ease software development and testing by partitioning a workstation into multiple virtual machines.

The virtual machine software market space has seen a substantial amount of evolution, The Xen® hypervisor, the powerful open source industry standard for virtualization. To vSphere, the first cloud operating system, transforming IT infrastructures into a private cloud-a collection of internal clouds federated on-demand to external clouds. Hardware vendors are also not too behind. Intel/AMD and other hardware vendors are pumping in lot of R&D dollars to make their chipsets and hardware optimized for hypervisor layer.

According to IDC more than 75% companies with more than 500 employees are deploying virtual servers. As per a survey by Goldman Sach’s 34 per cent of servers will be virtualized within the next 12 months among Fortune 1000 companies, double the current level of 15 per cent.

Cloud computing similarly existed as a concept for many years now. However various factors finally coming together that are now making it ripe for it to have the most impact. Bandwidth has been increasing significantly across the world that enables faster access to applications in the cloud. Thanks to success of SaaS companies, comfort level of having sensitive data out of their direct physical control is increasing.

There is increasing need for remote work force. Applications that used to reside on individual machines now need to be centralized.

Economy is pushing costs to go down. Last but not least, there is an increasing awareness about going green.

All these factors are causing the data center landscape to change. Now let’s look at some of the ways that the data centers are changing.

Data centers today are becoming much more agile. They are quick, light, easy to move and nimble. One of the reasons for this is that in today’s data center, virtual machines can be added quickly as compared to procuring and provisioning a physical server.

Self service provisioning allows end-users to quickly and securely reserve resources and automates the configuration and provisioning of those physical and virtual servers without administrator intervention. Creating a self-service application and pooling resources to share across teams not only optimizes utilization and reduces needless hardware spending but it also improves time to market and increases IT productivity by eliminating mundane and time consuming tasks.

Public clouds have set new benchmarks. E.g. Amazon EC2 SLA for availability is 99.95% which raised the bar from traditional data center availability SLA significantly. Most recently another vendor, 3Tera came out with five nines, 99.999% availability. Just to compare Amazon and 3Tera, 99.999% availability translates into 5.3 minutes of downtime each year, the different in cost between five 9’s and four 9’s (99.99 percent, or 52.6 minutes of downtime per year) can be substantial.

Data centers are also becoming more scalable. With virtualization, a data center may have 100 physical servers that are servicing 1000 virtual servers for your IT. Once again due to Virtualization, data centers are no longer constrained due to physical space or power/cooling requirements.

The scalability requirements for data centers are also changing. Applications are becoming more computation and storage hungry. Example of computation sensitive nature of apps, enabling a sub-half-second response to an ordinary Google search query involves 700 to 1,000 servers! Google has more than 200,000 servers, and I’d guess it’s far beyond that and growing every day.

Or another example is Facebook, where more than 200 million photos are uploaded every week. Or Amazon, where post holiday season their data center utilization used to be <10%! Google Search, Facebook and Amazon are not one off examples of applications. More and more applications will be built with similar architectures and hence the data center that hosts/supports those applications would need to evolve.

Data center are becoming more fungible. What that means is that resources used within the data centers are becoming easily replaceable. Earlier when you procured a server, chances were high that it will be there for number of years. Now with virtual servers, they will get created, removed, reserved and parked in your data center!

Data centers are becoming more Utility Centric and service oriented. As an example look at Cisco‘s definition of Data Center 3.0 where it calls it infrastructure services. Data center users are increasingly going to demand pay as you go and pay for what you use type of pricing. Due to various factors, users are going to cut back on large upfront capital expenses and instead going to prefer smaller/recurring operating expenses.

Most organizations have either seasonal peaks or daily peaks (or both) with a less dramatic cost differential; but the cost differential is still quite dramatic and quite impactful to the bottom line. In addition, the ability to pay for what you use makes it easy to engage in “proofs of concept” and other R&D that requires dedicated hardware.

  • As the discrepancy between peak usage and standard usage grows, the cost difference between the cloud and other options becomes overwhelming.

Technology is changing; the business needs are changing, with changing times organization’s social responsibilities are changing. More and more companies are thinking about the impact they have on the environment. Data centers become major source of environment impact especially as they grow in size.

A major contributor to excessive power consumption in the data center is over provisioning. Organizations have created dedicated, silo-ed environments for individual application loads, resulting in extremely low utilization rates. The result is that data centers are spending a lot of money powering and cooling many machines that individually aren’t doing much useful work.

Cost is not the only problem. Energy consumption has become a severe constraint on growth. In London, for example, there is now a moratorium on building new data centers because the city does not have the electrical capacity to support them!

Powering one server contributes to on an average 6 Tons of carbon emissions (depending upon the location of the server and how power is generated in that region) It is not too farfetched to claim that every data center has some servers that are always kept running because no one knows what business services depend on them but in reality no one seems to be using them. Even with the servers that are being used, there is an opportunity to increase their utilization and consolidate them.

Now that we have seen some of the ways that the data centers are changing, I am going to shift gears and talk about evolution of data centers. I am going to use the analogy of evolution of web to changing landscape of data centers. Just like web evolved from Web 1.0 where everyone could access, to Web 2.0 where people started contributing to Web 3.0 where the mantra is everyone can innovate.
Image showing Web-3.0 and DC-3.0
Applying this analogy to Data Centers we can see how it has evolved from its early days of existence to where we are today,
Evolution of a DC
Using the analogy of Web world, we can see how data centers have evolved from their early days till now.

  • In the beginning, Data centers were nothing but generic machines stored together. From there it evolved to blade servers that removed some duplicate components and optimized. Now in DC3.0, they are becoming even more virtual and cloud based.
  • So from mostly physical servers we have moved to Physical and Virtual servers to now where we would even treat underlying resources as virtual.
  • Provision time has gone down significantly
  • User participation has changed
  • Management tools that used to be nice to have are playing a much important role and are becoming mandatory. Good example once again is UCS where Bladelogic Mgmt tool will be pre-installed!
  • The role of a data center admin itself has changed from mostly menial work into a much more sophisticated one!

Slideshow for “Changing Landscape of Data Centers”

Changing Landscape of Data Centers

View more documents from Suhas Kelkar.

If you cannot see the slideshow above, click here.

Reblog this post [with Zemanta]

How green will be my valley?

Featured, punechips, Technology, , ,

(This is a guest blog by Chaitanya Rajguru, Associate Technical Fellow at KPIT Cummins, and a member of the PuneChips group.)

The integrated circuit from an Intel 8742, a 8...
Image via Wikipedia

The “greening” of all things commercial and industrial is all around us. Every industry from transportation to technology to power to finance is in a rush to be perceived as “green”. So should the EDA industry stay behind? I think not. And here are my thoughts on some possible scenarios on what may happen.

So where does one begin? One good starting point may be with a popular indicator used to gauge the “goodness” of EDA tool’s output: “Quality of Results”, or QoR. QoR is used as a higher-level indicator of process quality, much like a Customer Satisfaction Index that up-levels feedback on specific aspects such as timely delivery and responsiveness. IC design EDA tools have used to showcase what they can do. So is it possible to expand its scope to include “greenness” as well? Or is it just an attempt to paint a turkey blue and pass it off as a peacock?

QoR is one of the long-lived and often-used keywords in Silicon Valley – surely on par with “information superhighway” in sheer citation count. Yet the latter phrase isn’t heard much anymore. It just reminds us of the 90’s internet boom, and doesn’t convey anything that is new today. After all, this superhighway is now as much part of our lives as electric power distribution is, and it has been a while since either created much excitement. And so is “QoR” similarly frozen in time as well, not staying up-to-date with today’s design challenges?

Let us take a quick look at how QoR has evolved over time. In the early days of IC design, the biggest challenge was to pack as many transistors onto a single die as possible. The self-fulfilling prophecy of Moore’s Law had setup expectations that somehow had to be met! And while the accompanying frequency spiral required lots of efforts to maintain, it was achievable. Thus the QoR directly reflected “transistor count” and “frequency” as the most important indicators of EDA tool capability. Other variations appeared, such as the packing density of logic and analog circuitry.

“Power” then appeared on the QoR scene, as limits of battery power and even socket power were approached by systems. Now EDA vendors could speak the language of the system architects with their “power-performance-area” optimization triangle. Higher-level performance metrics such as MIPS and FLOPS entered. Then came combinations such as “MIPS per megahertz per watt.” Thus the QoR definition expanded from the “micro” qualities to encompass the “macro”: from frequency and packing density to power and performance.

Looking at current trends in the economy, “Going Green” has taken on big importance everywhere. It is the socio-politically correct thing to do, regardless of your product or service. Companies with physical products joined the bandwagon early: building architects, automobile manufacturers, consumer electronics OEMs, and IC manufacturers. One software company that has made a start is Google, with its goal to “minimize its carbon footprint.” Other companies have been slower to adapt – maybe due to having “soft products,” or maybe because they find it hard to make the right connection into this trend. But the semiconductor industry and the EDA industry are inevitably subject to the same greening trend, and can not convincingly “opt out.”

But “Being Green” is as high-level a quality metric for an EDA product as any – so much so, that whether it even applies to EDA tools is sure to be hotly debated. Yet suppose, for a moment, that it were to be made a part of QoR, how do you think it can be done?

Initial thoughts that come to my mind suggest getting a “Green Process” certification for the EDA tool development cycle, analogous to the ISO9001 or CMMI certifications. In the future, such certifications could surely be applicable to any business or organization (maybe even an individual!), and the EDA industry would be no exception. Another possibility is to publish a “carbon footprint” or “carbon neutrality indicator.”

But the above “green indicators” apply only to the development of the EDA tools, and give no satisfactory indication of whether their use will lead to “green products”. My best suggestion so far to gauge that quality is to measure the tool performance (the fewer compute cycles it burns, the better) and its reuse (the more, the better). Reuse can be in terms of reusing the building blocks (within a project), the output (across projects) and even the hardware utilization (e.g. exploiting multicore architectures). I believe these quality measures will anyway be applied to the evaluation of EDA tools, because they also affect development cost and schedule. So one might as well explicitly go after these indicators and kill two birds in one stone!

On the downside of a green QoR, we could be chasing a red herring. Isn’t it be better to focus on the core job of the EDA tool, which is to make the design task easier? To what extent do we go in order to conform to this latest fad? And how about degrees of greenness, and who measures those? If two tool vendors claim to be green, how do I verify their claims and compare them against each-other?

So, what do you think about the “Greening of QoR?” Is it meaningful? If not, why not? And if yes, how can we go about it? It’s always fun to make predictions, so please do share yours …

About the Author – Chaitanya Rajguru

Chaitanya is an Associate Technical Fellow at KPIT Cummins Infosystems Ltd. He has extensive experience in end-to-end development of semiconductor products, from definition to production, with specialization in PC chipset, graphics and Flash memory IC products. He has played various roles such as product development lead, technical expert, people manager and organizational development facilitator.

Related articles by Zemanta
Reblog this post [with Zemanta]

Rise of the Virtual Machines – Some thoughts on the impact of virtual machines

In Depth, Technology,
Virtual Machine Monitor Type I
Schematic diagram of a Virtual Machine setup. The physical hardware is at the bottom, the virtual machine monitor (VMM) sits in the middle, and multiple actual virtual machines sit on top of the VMM. Image via Wikipedia

(This post by Dilip Ranade on his blog, takes a look at how Virtual Machines are going to change the way we do computing, and also how we will start using virtual machines in new and interesting ways as they mature. It is republished from there here with permission.)

Synopsis: some thoughts on the impact of virtual machines

Virtual Machines were invented in IBM in the early seventies , but it appears that it was only VMWare started much later in 1998 that figured out how to make money purely out of virtualization. However, with Xen and Microsoft Virtual Server also entering the fray, things are getting interesting.  The green pastures of Virtual Machines, often misnamed virtualization (which is actually
a broader term) now appear poised to support a large herd of bulls.

Although it is hard to predict all the ways in which a new technology will change the world– think of telephones and sex hotlines for example — here are some thoughts on how VM’s can have an unforeseen impact, arranged roughly in order of increasing ambitiousness:

  • VM’s can break the HW/SW Red Queen Effect
  • VM’s can break vendor “lock-in”
  • Processors can become commoditized
  • Operating systems can become commoditized
  • Rise of virtual appliances
  • Rise of virtual machine swarms

VM’s can Break the HW/SW Red Queen Effect.

Software vendors and hardware vendors are in a mutually beneficial race, leading to an exponential spiral: customers are forced to buy ever more powerful computers to run ever more resource-hogging versions of software. But with a Virtual Machine this collusion can be broken. First of all, customers will balk at buying bloated software, as happened with Microsoft Vista. Secondly, marginally bloated software can be tolerated without having to replace the virtual servers with more powerful machines. For example, a VM can
be virtually upgraded to larger memory or more CPUs without making new purchases.
Thus, the existence of virtualized servers brings genuine economic pressure for software developers to be more frugal with CPU and memory consumption in their products. This works in conjunction with the next point.

VM’s can Break Vendor “lock-in”

When a software product is on a virtual machine, it is easy and non disruptive to try
out a competing product on another virtual machine, even if it
requires different type of hardware. However, this effect is not as powerful
as it can potentially be, because todays virtualization is too focused to x86
architecture.

Processors can Become Commoditized

The time is ripe for the evolution of a standard virtual processor,
just like TCP/IP is for network protocols. Consider the advantages: Considerably reduced development and testing costs (write once run anywhere); potentially longer software product life (delinked from hardware obsolescence); clean room environment for “dusty decks” (very old software can continue to run in a virtual environment). I am thinking of a more abstract kind of virtual processor that is also extensible or mutable in ways that hardware processors cannot be. It may not need to make hard choices between various hardware tradeoffs.
The Java virtual machine is an example.

Operating Systems can Become Commoditized

As the virtual processor evolves towards higher levels of
abstraction, so should virtual devices that it connects to. This should reduce the complexity of the virtual operating system; then it should not need a team consisting of thousands of progammers to maintain a virtual operating system.
For example, a virtual OS does not need bootstrapping code – it can boot of a virtual network boot service. Similarly, there is no need for every virtual Operating system to implement its own file system and to interact only with (virtual) hard disks. All it needs is a simple file system client to discover and connect to the correct virtual Network Attached Storage (NAS) devices.

Rise of Virtual Appliances

General-purpose operating systems can be replaced with lean-and-mean
tailor-made variants designed for specific applications. For example
an OS built specifically for a web server, or different one for a
database.

Rise of Virtual Machine Swarms

The trend towards multi-core, multi-thread programming can be fitted
better to a virtual machine designed to work in swarms. The Transputer of late 1980’s comes to
mind: multiprocessor meshes could be built from multiple Transputers just by physically connecting built-in serial links between pairs of Transputers. The
standard virtual processor’s simple network interconnect could support
easy interfacing within a swarm. I think PVM and grid computing concepts can be considered precursors of VM swarms.

About the Author – Dilip Ranade

For more information, see his linked-in profile.

Related articles by Zemanta

Comments are closed on this post. If you have any comments, please leave them at the original article.

Reblog this post [with Zemanta]

“wh[0x01] WildHack Contest” for articles/video/code related to cyber security by null.co.in

Featured, Technology, , , ,

null-logoNull.co.in, Pune’s network security community for hackers, security professionals, security enthusiasts, and in fact anyone related to IT for whom security matters (ahem: if you are in IT, and security does not matter to you, you should really not be in IT, should you?) is holding a month long contest for the best security related content. Content means anything that you can produce: article, blog post, whitepaper, advisory, disclosure, tutorial, video/audio, source code, tool, proof-of-concept. Pretty much anything that you created on your own, and relates to cyber security, and would be educational for other people to see/read/consume.

The contest runs from 15th June to 15th July, and winners get cool “????” T-shirts (there will be at least 10 winners). It’s OK to submit content that you’ve previously published elsewhere (e.g. bugtraq, or your blog), but it must be your original content. So get cracking (or is it “hacking”) and email your submission to submit _at_ null.co.in

Contest Details

1. The contest starts on 15th June 2009 and ends on 15th July 2009. Winners will be announced on the null mailing list on 20th July 2009.
2. The submissions can be anything related to security/hacking.

3. Submission Categories expected(but not limited to):

  • – L2-L4 security/hacking.
  • – Web 2.0 vulnerabilites and countermeasures.
  • – .NET Malware/security.
  • – Code injection (Binary/XSS/SQL/Command etc).
  • – Spam mitigation and antispam evasion techniques.
  • – Malware detection and antimalware evasion techniques.
  • – Protocol vulnerabilities.
  • – Voip.
  • – Mobile networks GSM/CDMA/3G.
  • – Wireless.
  • – Cryptography.
  • – OS/Kernel and Virtualization security/hacking.
  • – Bluetooth.
  • – Hardware based security/hacking.
  • – Cyber Forensics.
  • – Cyber Warfare.
  • – Social Engineering.

4. Research work in Progress will also be accepted.
5. The submissions can be in the form of:

  • – White papers.
  • – Advisories/Disclosures.
  • – Best Practices.
  • – Video/Audio Demos.
  • – Tutorials.
  • – Hacks, tricks & tweaks.
  • – PoCs.
  • – Source code/Tools.

6. The submission should be original work of the author/submitter.
7. Your submissions* should be emailed to (submit _at_ null.co.in).
8. It is ok to submit your work already published on the net like advisories already posted on FD/Bugtraq, paper presented at a conference etc.
9. Submissions will be judged by core group members of null. Criteria for judgement:

  • – More technical the submission, more chances it has for winning.
  • – Innovation/Some thing new and never heard of before.
  • – Age of the work. Newer work will get more preference.

10. NULL will be giving away atleast 10 T-shirts.
11. For any further queries/details please write to (dhiraj _at_ null.co.in).

*Disclaimer: By submitting your work to null you are giving null the right to publish and redistrubute it in any form.

See the contest website for more details.

Related articles by Zemanta
Reblog this post [with Zemanta]

optimization: a technical overview

In Depth, Overviews, Technology, , , ,

(This is the fourth in the PuneTech series of articles on optimization by Dr. Narayan Venkatasubramanyan, an Optimization Guru and one of the original pioneers in applying Optimization to Supply Chain Management. The first one was an ‘overview’ case study of optimization. The second was architecture of a decision support system. The third was optimization and organizational readiness for change.

For Dr. Narayan Venkatasubramanyan’s detailed bio, please click here. For the full series of articles, click here.)

this is a follow-up to optimization: a case study. frequent references in this article to details in that article would make this one difficult to read for someone who hasn’t at least skimmed through that.

the problem of choice

the wikipedia article on optimization provides a great overview of the field. it does a thorough job by providing a brief history of the field of mathematical optimization, breaking down the field into its various sub-fields, and even making a passing reference to commercially available packages that help in the rapid development of optimization-based solutions. the rich set of links in this page lead to detailed discussions of each of the topics touched on in the overview.

i’m tempted to stop here and say that my job is done but there is one slight problem: there is a complete absence of any reference to helicopter scheduling in an offshore oil-field. not a trace!

this brings me to the biggest problem facing a young practitioner in the field: what to do when faced with a practical problem?

of course, the first instinct is to run with the technique one is most familiar with. being among the few in our mba program that had chosen the elective titled “selected topics in operations research” (a title that i’m now convinced was designed to bore and/or scare off prospective students who weren’t self-selected card-carrying nerds), we came to the problem of helicopter scheduling armed with a wealth of text-book knowledge.

an overview of linear programming

A series of linear constraints on two variable...
the lines represent the constraints. the blue region is the set of all “permissible values”. the objective function is used to choose one (“the most optimal”) out of the blue points. image via wikipedia

having recently studied linear and integer programming, we first tried to write down a mathematical formulation of the problem. we knew we could describe each sortie in terms of variables (known as decision variables). we then had to write down constraints that ensured the following:

  • any set of values of those decision variables that satisfied all the constrains would correspond to a sortie
  • any sortie could be described by a set of permissible set of values of those decision variables

this approach is one of the cornerstones of mathematical programming: given a practical situation to optimize, first write down a set of equations whose solutions have a one-to-one correspondence to the set of possible decisions. typically, these equations have many solutions.

click here for an animated presentation that shows how the solutions to a system of inequalities can be viewed graphically.

the other cornerstone is what is called an objective function, i.e., a mathematical function in those same variables that were used to describe the set of all feasible solutions. the solver is directed to pick the “best” solution, i.e., one that maximizes (or minimizes) the objective function.

the set of constraints and the objective function together constitute a mathematical programming problem. the solution that maximizes (or minimizes) the objective function is called an optimal solution.

linear programming – an example

googling for “linear programming examples” leads to millions of hits, so let me borrow an example at random from here: “A farmer has 10 acres to plant in wheat and rye. He has to plant at least 7 acres. However, he has only $1200 to spend and each acre of wheat costs $200 to plant and each acre of rye costs $100 to plant. Moreover, the farmer has to get the planting done in 12 hours and it takes an hour to plant an acre of wheat and 2 hours to plant an acre of rye. If the profit is $500 per acre of wheat and $300 per acre of rye how many acres of each should be planted to maximize profits?”

the decisions the farmer needs to make are: how many acres of wheat to plant? how many acres of rye to plant? let us call these x and y respectively.

so what values can x and y take?

  • since we know that he has only 10 acres, it is clear that x+y must be less than 10.
  • the problem says that he has to plant at least 7 acres. we have two choices: we can be good students and write down the constraint “x+y >= 7” or we can be good practitioners and demand to know more about the origins of this constraint (i’m sure every OR professional of long standing has scars to show from the times when they failed to ask that question.)
  • the budget constraint implies that 200x + 100y <= 1200. again, should we not be asking why this farmer cannot borrow money if doing so will increase his returns?
  • finally, the time constraint translates into x + 2y <= 12. can he not employ farm-hands to increase his options?
  • the non-negativity constraints (x, y >= 0) are often forgotten. in the absence of these constraints, the farmer could plant a negative amount of rye because doing so would seem to get him more land, more money, and more time. clearly, this is practically impossible.

as you will see if you were to scroll down that page, these inequalities define a triangular region in the x,y plane. all points on that triangle and its interior represents feasible solutions: i.e., if you were to pick a point, say (5,2), it means that the the farmer plants 5 acres each of wheat and 2 acres of rye. it is easy to confirm that this represents no more than 10 acres, no less than 7 acres, no more than $1200 and no more than 12 hours. but is this the best solution? or is there another point within that triangle?

this is where the objective function helps. the objective is to maximize the profit earner, i.e., maximize 500x + 300y. from among all the points (x,y) in that triangle, which one has the highest value for 500x + 300y?

this is the essence of linear programming. LPs are a subset of problems that are called mathematical programs.

real life isn’t always lp

in practice, not all mathematical programs are equally hard. as we saw above, if all the constraints and the objective function are linear in the decision variables and if the decision variables can take on any real value, we have a linear program. this is the easiest class of mathematical programs. linear programming models can be used to describe, sometimes approximately,a large number of commercially interesting problems like supply chain planning. commercial packages like OPL, GAMS, AMPL, etc can be used to model such problems without having to know much programming. packages like CPLEX can solve problems with millions of decision variables and constraints and produce an optimal solution in reasonable time. lately, there have been many open source solvers (e.g., GLPK) that have been growing in their capability and competing with commercial packages.

Illustrates a cutting plane algorithm to solve...
integer programming problems constrain the solution to specific discrete values. while the blue lines represent the “feasible region”, the solution is only allowed to take on values represented by the red dots. this makes the problem significantly more difficult. image via wikipedia

in many interesting commercial problems, the decision variables is required to take on discrete values. for example, a sortie that carries 1/3 of a passenger from point a to point b and transports the other 2/3 on a second flight from point a to point b would not work in practice. a helicopter that lands 0.3 in point c and 0.7 in point d is equally impractical. these variables have to be restricted to integer values. such problems are called integer programming problems. (there is a special class of problems in which the decision variables are required to be 0 or 1; such problems are called 0-1 programming problems.) integer programming problems are surprisingly hard to solve. such problems occur routinely in scheduling problems as well as in any problem that involves discrete decisions. commercial packages like CPLEX include a variety of sophisticated techniques to find good (although not always optimal) solutions to such problems. what makes these problems hard is the reality that the solution time for such problems grows exponentially with the growth in the size of the problem.

another class of interesting commercial problems involves non-linear constraints and/or objective functions. such problems occur routinely in situations such refinery planning where the dynamics of the process cannot be described (even approximately) with linear functions. some non-linear problems are relatively easy because they are guaranteed to have unique minima (or maxima). such well-behaved problems are easy to solve because one can always move along an improving path and find the optimal solution. when the functions involved are non-convex, you could have local minima (or maxima) that are worse than the global minima (or maxima). such problems are relatively hard because short-sighted algorithms could find a local minimum and get stuck in it.

fortunately for us, the helicopter scheduling problem had no non-linear effects (at least none that we accounted for in our model). unfortunately for us, the discrete constraints were themselves extremely hard to deal with. as we wrote down the formulation on paper, it became quickly apparent that the sheer size and complexity of the problem was beyond the capabilities of the IBM PC-XT that we had at our disposal. after kicking this idea around for a bit, we abandoned this approach.

resorting to heuristics

we decided to resort to a heuristic approach, i.e., an approach that used a set of rules to find good solutions to the problem. the approach we took involved the enumeration of all possible paths on a search tree and then an evaluation of those paths to find the most efficient one. for example, if the sortie was required to start at point A and drop off m1 men at point B and m2 men at point C, the helicopter could

  • leave point A with the m1 men and proceed to point B, or
  • leave point A with the m2 men and proceed to point C, or
  • leave point A with the m1 men and some of the m2 men and proceed to point B, or
  • leave point A with the m1 men and some of the m2 men and proceed to point C, or
  • . . .

if we were to select the first possibility, it would drop off the m1 men and then consider all the options available to it (return to A for the m2 men? fly to point D to refuel?)

we would then traverse this tree enumerating all the paths and evaluating them for their total cost. finally, we would pick the “best” path and publish it to the radio operator.

at first, this may seem ridiculous. the explosion of possibilities meant that this tree was daunting.

there were several ways around this problem. firstly, we never really explicitly enumerated all possible paths. we built out the possibilities as we went, keeping the best solution until we found one that was better. although the number of possible paths that a helicopter could fly in the course of a sortie was huge, there were simple rules that directed the search in promising directions so that the algorithm could quickly find a “good” sortie. once a complete sortie had been found, the algorithm could then use it to prune searches down branches that seemed to hold no promise for a better solution. the trick was to tune the search direction and prune the tree without eliminating any feasible possibilities. of course, aggressive pruning would speed up the search but could end up eliminating good solutions. similarly, good rules to direct the search could help find good solutions quickly but could defer searches in non-obvious directions. since we were limited in time, so the search tree was never completely searched, so if the rules were poor, good solutions could be pushed out so late in the search that they were never found, at least not in time to be implemented.

one of the nice benefits of this approach was that it allowed the radio operator to lock down the first few steps in the sortie and leave the computer to continue to search for a good solution for the remainder of the sortie. this allowed the optimizer to continue to run even after the sortie had begun. this bought the algorithm precious time. allowing the radio operator the ability to override also had the added benefit of putting the user in control in case what the system recommended was infeasible or undesirable.

notice that this approach is quite far from mathematical programming. there is no guarantee of an optimal solution (unless one can guarantee that pruning was never too aggressive and that we exhaustively searched the tree, neither of which could be guaranteed in practical cases). nevertheless, this turned out to be quite an effective strategy because it found a good solution quickly and then tried to improve on the solution within the time it was allowed.

traditional operations research vs. artificial intelligence

this may be a good juncture for an aside: the field of optimization has traditionally been the domain of operations researchers (i.e., applied mathematicians and industrial engineers). even though the field of artificial intelligence in computer science has been the source of many techniques that effectively solve many of the same problems as operations research techniques do, OR-traditionalists have always tended to look askance at their lowly competitors due to the perceived lack of rigour in the AI techniques. this attitude is apparent in the wikipedia article too: after listing all the approaches that are born from mathematical optimization, it introduces “non-traditional” methods with a somewhat off-handed “Here are a few other popular methods:” i find this both amusing and a little disappointing. there have been a few honest attempts at bringing these two fields together but a lot more can be done (i believe). it would be interesting to see how someone steeped in the AI tradition would have approached this problem. perhaps many of the techniques for directing the search and pruning the tree are specific instances of general approaches studied in that discipline.

if there is a moral to this angle of our off-shore adventures, it is this: when approaching an optimization problem, it is tempting to shoot for the stars by going down a rigorous path. often, reality intrudes. even when making technical choices, we need to account for the context in which the software will be used, how much time there is to solve the problem, what are the computing resources available, and how it will fit into the normal routine of work.

other articles in this series

this article is the fourth in the series of short explorations related to the application of optimization. i’d like to share what i’ve learned over a career spent largely in the business of applying optimization to real-world problems. interestingly, there is a lot more to practical optimization than models and algorithms. each of the the links leads to a piece that dwells on one particular aspect.

optimization: a case study
architecture of a decision-support system
optimization and organizational readiness for change
optimization: a technical overview (this article)

About the author – Dr. Narayan Venkatasubramanyan

Dr. Narayan Venkatasubramanyan has spent over two decades applying a rare combination of quantitative skills, business knowledge, and the ability to think from first principles to real world business problems. He currently consults in several areas including supply chain and health care management. As a Fellow at i2 Technologies, he tackled supply chains problems in areas as diverse as computer assembly, semiconductor manufacturer, consumer goods, steel, and automotive. Prior to that, he worked with several airlines on their aircraft and crew scheduling problems. He topped off his days at IIT-Bombay and IIM-Ahmedabad with a Ph.D. in Operations Research from the University of Wisconsin-Madison.

He is presently based in Dallas, USA and travels extensively all over the world during the course of his consulting assignments. You can also find Narayan on Linkedin at: http://www.linkedin.com/in/narayan3rdeye

Reblog this post [with Zemanta]

Improve your web based software development and maintenance ROI with dynamic programming languages

In Depth, Technology, , , ,

After we carried a few quick articles on why you should learn more about Ruby and Ruby on Rails (take 1, take 2) last month, we decided that we wanted to give people a much deeper article on why these new languages (Ruby, Python, PHP) and frameworks (Rails, Django) are setting the web world on fire. We invited Dhananjay Nene to write an article with an in depth discussion of the technical reasons how these new languages differ from the older ones and when to choose one over the other. He responded with this article which, as an added bonus, also includes the business reasons for your decisions. At the request of the community, Dhananjay is also giving a talk on the relative strengths and weaknesses of different programming languages on Saturday, 28th March, 4pm, at SICSR. All those who found this article interesting should definitely attend.

Introduction

Programing language selection is often a topic that elicits a lot of excitement, debate and often a bit of acrimony as well. There is no universally superior programming language that one can recommend, so I tend to generally disregard most language opinions which say ‘X language is the best’, without specifying the context under which it is superior. Finally most language debates often deal with the technical issues and not the ROI issues. Hopefully I shall be able to address this topic without being guilty of any of these problems.

So what languages are we referring to here ?

Official Ruby logo
Image via Wikipedia

The range of languages that fall under Dynamic Programming Languages category is rather extensive. My experience is primarily limited to Python and to a lesser extent PHP, Ruby, Javascript, and Groovy. For the rest of this article, I shall be primarily referring to Python or Ruby when I use the word dynamic languages, though many of the references may continue to be applicable and relevant for a number of other dynamic programming languages.

As I describe the technical characteristics, I shall also continue to attempt to address the business aspects as well, so you might find this article at a little techno-business level. Assuming I am able to excite their interest, the tech guys would not find sufficient technical details and would be hungry to hunt for more, and while the business guys would get a little teased with the possibilities, they will not quite get the ROI served in the traditionally formatted excel spreadsheets. Being aware of that, I continue down this path with a feeling that this perhaps will be the most appropriate level for me to abstract this article to.

Characteristics of Dynamic Programming Languages.

Let us quickly review some of the characteristics :

CPython
Image via Wikipedia

Object Oriented : Many dynamic languages support full object orientation. There are many who don’t necessarily buy the benefits of Object Orientation, but it is my strong belief, that once a piece of software grows beyond a certain threshold of complexity and / or size, Object Orientation starts delivering very strong dividends. There are a few areas such as highly complex, algorithmic processing which might be better suited for functional programming. However a majority of the medium-to-large sized web applications are better served by OO. The empirical evidence at least bears out the fact that most of the most popular languages today (except C) are Object Oriented. However this still is a very very large class of languages which in them include C++, Java, PHP, Python, Ruby etc. The one area where some dynamic languages separate themselves from the others is in the notion of “everything is an object”, ie. primitives such as numbers, functions are all objects by themselves.

Business implications: OO code well designed and implemented allows for a substantial reduction in maintenance costs. When working with a team which is up the curve on OO, it is likely to lead to lower costs and time on inital coding as well. On the other hand, both training costs and skill requirements are higher for fully OO languages. If you are already using partialy OO / hybrid languages such as PHP, C++ or Java, and are convinced about OO, using fully OO languages such as Python or Ruby will help you leverage the OO capabilities even further.

Duck Typing : In very loose terms, duck typed languages do not require you to declare an explicit interface. You send an object a message (ie. invoke a function or access an attribute) and if it can respond to it, it will, and if it can’t it will result in an error. Duck typing is a specific typing system which is a subset of a broader system called Dynamic Typing, which often makes for an interesting debate with its counterpart – Static typing : Static and Dynamic Type checking in practice. For people well grounded in static typing alone, this can sometimes seem to be sacrilegious. I am convinced that duck typing makes writing code much much faster for two reasons – a) You now require to write fewer lines of code and b) You often don’t have to keep on regularly waiting for the compiler to do its work. There is also a substantial capability enhancement that dynamic typing makes to the language type system, which allow the frameworks to build dynamic types on the fly. This in turn offers the framework users many more capabilities than frameworks written in other languages. That is why it is nearly impossible to write frameworks like Rails or Django in Java (You can modify the class loaders and use byte code generation to generate the new types, but the compiler can’t see them so you cant use them). That is also why there is a lot of anticipation of using JRuby, Jython and Grails on the JVM since the languages underlying them (Ruby, Python and Groovy respectively) bring the dynamic typing capabilities to the JVM platform.

Business Implications :Writing code is much much faster. Maintenance depending upon the situation can sometimes be more or less difficult in case of dynamic typed languages. Refactoring is usually a lot more difficult in case of dynamically typed languages since the underlying type system is not able to infer sufficiently about the code to help the refactoring tools, as is possible in case of statically typed languages. It is my opinion that a skilled and trained development team using dynamic languages can generally substantially outperform another equally capable team using static languages. Insufficiently or poorly skilled development teams however can lead to very very different kind of pitfalls in these class of languages. In both cases the code becomes difficult to change or maintain due to a) cryptic code in case of dynamically typed languages and b) extremely large code bases in case of statically typed languages. Both are undesirable situations to be in but if I had to choose between one of the two, I would go for being in the cryptic mess since it is at least manageable by bringing in external skilled help.

Metaprogramming : Metaprogramming is in loose terms the ability of programs to write programs. A large proportion of developers may not use this capability too frequently. Specifically in web application development it gets used as a mechanism to transform one set of datastructures which a programmer specifies into code at runtime. As I point out later in this article, it in fact is a very important element in designing common frameworks and libraries which in turn offer substantial capabilities including small code and easier maintenance. A quick note to state that metaprogramming is not code generation. In case of code generation, one uses the generator to generate code which is then compiled. A big limitation with this is the fact that often people modify the generated code leading to really tough maintenance nightmares and the fact that it is a two stage process which is prone to more errors. Metaprogramming results in new code “coming to life” so to speak while your program is running.

Business Implications : Read on, they will get covered in the final roundup. They are large and they are positive.

Function blocks/objects, iterators, closures, continuations, generators: I will not go into any substantial details of this issue except to say that small pieces of code logic can be handled in a much much more concise way than if these weren’t supported. While many situations may not need closures support, you will be glad to have them on your side when needed.

Business Implications : Helps having shorter, cleaner code leading to lesser development and maintenance costs. Another significant positive is that your developers are just likely to be so much happier since they get some truly nice building blocks for concise and elegant expression of their logic. Can’t think of any significant negatives.

There are a full range of other capabilities, but none come to mind immediately as something that have strong business implications as well.

The role of frameworks

Ruby on Rails
Image via Wikipedia

When did these languages say Ruby and Python originate ? Most people are likely to be a little surprised if the answer is in the last millenium. Yet Guido von Rossum started working on Python in 1986 and Ruby was released in 1992. Python has been rather well known within the scientific community and perhaps a bit within the systems / OS utility programming communities for quite some time. However both languages grabbed a large mindshare only post 2005. A big reason for their popularity (especially in case of Ruby’s case) came from the popularity the frameworks which used them. Ruby on Rails for ruby and Django (to the best of my knowledge) for python. These frameworks combined the language capabilities with the learnings of good design practices for internet applications (eg MVC, declarative validations, simple ORM etc) into a simple usable package, which developers could take and build web applications quickly. There are examples of people having built simple web apps within a day and medium complexity apps in 1-3 weeks using these frameworks. The languages are the ingredients, the frameworks are the cooks – a great combination for serving great meals. Now you will find many such frameworks in these languages, including some which have better capabilities for building more sophisticated / complex applications eg. Merb and Pylons.

I am not too sure of how many people are exactly aware of the role of metaprogramming in the frameworks’ successes. I am willing to believe that but for metaprogramming, these frameworks simply would not have achieved anywhere close to the success they achieved. It is metaprogramming which takes the datastructures as defined by a developer and converts it into runtime code implicitly, saving the developer lots of time and effort. So even if most developers don’t actively write metaprograms, their lives are so much easier. Metaprogramming capabilities are also the reason why it is virtually impossible to write similar frameworks in Java. However if you are on the .NET or JVM environments, things are definitely looking encouraging with the possibilities to use IronPython or IronRuby on .NET or JRuby or Jython or Groovy+Grails on the JVM.

Business implications : If you are focused on scientific or desktop or highly algorithmic applications, where python especially is used extensively, you are likely to get benefits from these languages on their own merit alone. For web applications you will see the maximum benefits by using the web MVC frameworks along with the languages. I submit that on the whole you are likely to see very substantial reduction in development, enhancement and maintenance times – sweet music for any end user, investor or project manager.

Increased Business Agility

There is one more reason why I believe these languages are especially helpful. They help by increasing development agility to an extent where it now allows for the business to be more agile. You can get a first prototype version up in weeks, take it around to potential users, and gather feedback on the same. Incorporate elements of this feedback into the next release of working code quickly. The business benefits of such a scenario are tremendous. You might wonder that this is a process issue, so what does it have to do with a language selection. I would submit, that languages which allow changes to be made faster, help support this process in a far superior way. Another equally important facet is the superior risk management. Since you are able to build features with lower investments, you are able to get a series of customer feedbacks into your decision making process much faster. This helps being able to come up with a product that really meets the customer expectations much earlier. This happens by allowing the better features to come in earlier and also by allowing the lesser important or lesser relevant features to be decided to be deferred earlier. That’s precisely the reason why the dynamic languages have found a strong acceptance in the startup world. I believe the increasing agility which is often required in the startup world, is and will continue to be increasingly required of established enterprises. Precisely the reason why I believe these languages will continue to do better in the enterprise space as well. Finally, these languages make it relatively easier to tell your business sponsor – We will work with you on imprecise requirements rather than spending months on nailing down requirements which anyways are likely to change later. This has both a pro and a con especially for outsourcing situations. It is likely to allow for tremendous customer delight in terms of a vendor that works with him in such a flexible manner, yet it does introduce challenges in terms of how the commercials and management of the project are handled.

The reason I would like to especially point out increased business agility is because programmers don’t often visualise or evangelise it much, but when I wear a manager’s hat, it is perhaps the most compelling benefit of these languages.

Concluding

As I said earlier, there is no single universal language which is the best for all scenarios. There are some scenarios where using dynamic languages will not be helpful

Programming language book sales 4Q2008

A Treemap view of sales of programming language books by O’Reilly Media in 4Q2008. The size of a box represents the total sales of a book. The color represents the increase or decrease in sales compared to same quarter in 2007. Green = increase, bright green = big increase, red = decrease, bright red = large decrease. See full article at O’Reilly Radar for lots of interesting details.

When not to use these languages

  • You are building a simple / small application and don’t have the available skill sets. One exception to this is where you decide to use it in a simple application to allow yourself a non risky mechanism of building these skillsets.
  • Extremely High performance requirements. However please make sure that you really need the high performance capabilities of say a C, C++ or Java. In my experience 80% of developers like to believe that they are building highly performant applications where the maximum speed is a must have. Yet the top 10% of them are facing far far more critical performance requirements than the remainder. Unless you are convinced you are in the top 10%, you should certainly consider dynamic languages as an option. Moreover in case of most high performance requirements, these can sometimes be boiled down to a few inner loops / algorithms. Consider implementing the same in C, / Java or other .NET languages (depending upon the choice of your dynamic language interpreter implementation)
  • You have an architecture standard in place which does not allow using these languages. If you are convinced your applications are better served by using dynamic languages both from your individual application and an overall enterprise perspective, consider taking the feedback to your standards setting body to see if you can pilot a different approach. Also evaluate if the .NET or JVM versions can help you comply with the architecture guidelines.
  • You are unable to commit to the retraining requirements. While these languages are easy and powerful to use, leveraging that power can require some amount of retraining. If that does not fit your business plans, since the retraining effort could impact immediate and urgent requirements, that could be a reason to not use these languages. However in such situations do consider investing in building this skill sets before you get to another similar decision point.
  • You need a very high levels of multithreadinging as opposed to multi processing support. While this is not a typical situation for web applications, you should be aware that most dynamic languages have some limitations in terms of multi threading support. This actually is not necessarily an issue with the language as with the implementation eg. the C implementation of python has the notorious Global Interpreter Lock which constrains you from being able to use more than a handful of threads per processes efficiently. However the same restriction is not present in Jython (the jvm implementation of python). This is likely to be an issue for a miniscule percentage of the web applications market for the primary reason that multi process / shared nothing architecture styles often work quite well for many web applications and they don’t really need multi threading.

So where’s my return on investment ?

First of all lets talk of the investment part. If you get into it in a paced approach, the investment may not be that great. Start with a team size of anywhere between 2-6 people (depending upon your organisation and project size). Think of 15 days of intensive training followed by a 2-6 months coming up the curve effort (more likely 2 than 6). Make sure your first project is not a critical one under tremendous business pressure. This can be subsequently followed by more people getting retrained as necessary. In the longer term it might actually help reduce your incremental investment, since it might be much easier to ramp up new programmers in Ruby or Python than say Java or C#.

Secondly lets look at the incrementally higher costs. You are likely to need people who are a little bit more capable in terms of understanding and debugging the same logic expressed in fewer lines of code (that sometimes can be a challenge) and then be able to modify and enhance the same. This may increase your testing and fixing costs in the earlier days. Finally while the fewer lines of code can make refactoring easier, you could find that your total refactoring costs are a little higher.

Now the returns part. I am convinced that the increased business agility is the strongest return in business terms. Immediately after that is the substantial reduction in development, enhancement and maintenance times. If neither of these benefits are appealing, when contrasted with some other issues that you might perceive, maybe considering dynamic languages in your context is not such a great idea.

One more factor that I would of course encourage you to evaluate from a business perspective are the implications for you if your competition (assuming it is not already using them) started using these languages. The implications would vary from case to case, but it could also help you decide how important this issue is for you.

About the author – Dhananjay Nene

Dhananjay is a Software Engineer with around 17 years of experience in the field. He is passionate about software engineering, programming, design and architecture. He did his post graduation from Indian Institute of Management, Ahmedabad, and has been involved in Senior Management positions and has managed team sizes in excess of 120 persons. His tech blog, and twitter stream are a must read for anybody interested in programming languages or development methodologies. Those interested in the person behind the tech can check out his general blog, and personal twitter stream. For more details, check out Dhananjay’s PuneTech wiki profile.

Related articles by Zemanta
Reblog this post [with Zemanta]

Introduction to Engineering Simulations (CAE)

In Depth, Technology, , , , ,

(This article giving an overview of the field of Computer Aided Engineering has been written for PuneTech on our request by Dr. Ajey Walavalkar, regional manager of services and support at Ansys Fluent India, a company specializing in CAE and CFD applications and services, which has a development center in Pune. See the end of this article for more about Ajey.)

In the earlier PuneTech article “An Overview of CAD,” Yogesh and Amit have provided a very good overview of the CAD technology spectrum. There, we learnt where and how “Analysis” and “Simulations” fit in the overall scheme of CAD. This article is intended to provide a broad overview of the area called Engineering Simulations, which spans across the above mentioned “Analysis” and “Simulations” fields on the CAD canvas.

What are Engineering Simulations?

The analysis of vibrations and the dynamical behaviour due to excitation forces have been applied routinely to cable-stayed bridges using SMR devised analysis tools. Note that similar techniques have been used to analyse the resonance behaviour of large floating bridge structures.

The example shows an eigenmode of a large cable-stayed bridge (model and analysis with B2000 by The Dutch National Aerospace Research Institute (NLR)). The figure displays a vibration mode, the shape being artificially amplified to emphasize the deformation. Source http://www.smr.ch/services/csm

Let’s say you want to build a bridge. Not just any bridge, but a massive suspension bridge to rival the Golden Gate Bridge in San Francisco Bay Area. How do you decide the type of steel, the span length, the tower height, the thickness of the cables, the depth of the foundations, and other design parameters? You will wonder that if this problem was solved in the 1930s then why is it tough today? The simple answer is, ‘Solved? Yes! But at what cost and effort?’ Previously, the simple solution for most tough engineering problems was to ‘over-engineer’ the solution by building in a ‘huge factor of safety’. Today, this design process is lot more accurate and efficient. Today, the engineering team will start off by considering the effects of vehicles (weights and speeds) plying on the bridge, the wind forces that will sway the bridge, the waves that will hit the foundation, the steady long-term corrosive effects of weather, etc. These effects can studied by mathematically modeling these factors and ‘simulating’ them with a help of a computer. Such ‘simulations’ greatly help today’s engineers to come up with the exact design that is safe and that saves time and costs.

Wikipedia defines Simulation as follows:

Simulation is the imitation of some real thing, state of affairs, or process. The act of simulating something generally entails representing certain key characteristics or behaviors of a selected physical or abstract system.

Engineering simulations are simulations applied to engineering problems. These can range from simple “back of the envelope” calculations to full fledged systems simulations using High Performance Computing.
Mathematical modeling and physical modeling are two main aspects of engineering simulations. Mathematical modeling refers to the description of a particular phenomenon in terms of mathematical equations. E.g. If you wish to find out the distance a canon ball will travel when fired from a canon at certain angle, at a certain speed; you can write some mathematical expressions and solve them to calculate this distance. This is mathematical modeling. Now suppose you are firing that canon ball at a mountain cliff and the ball is going to bounce off and roll down the slopes of the mountain, how do you find out where that ball will end up? Here you need to also consider the physical structures and that leads you to physical modeling.

Uses of Engineering Simulations

Today, computer aided engineering simulations are used extensively in designing and development of almost all industrial products. In addition to product designing, simulations find their uses in troubleshooting of existing systems and research & development of new processes. Many times the field of computer aided engineering simulations is also termed as Computer Aided Engineering (CAE). There are numerous software products available in the market and they offer variety of engineering simulation capabilities. Many organizations also use their in-house built simulation software products that capture their time proven design practices.

There are many objectives in performing engineering simulations:

  1. During product design cycle, engineering simulations enable the designers to evaluate various design options without getting into costly prototyping and testing. Use of simulations in the design process can help narrow the design choices to very small set that can be taken to the prototyping and testing phases. Even with physical testing, one can inspect and measure only a small number of variables at few select locations. Whereas simulations can provide visual information on all the variables of interest at all locations inside the simulated space.
  2. Simulations can help troubleshoot deviation of actual performance of system/product from the desired one.
  3. Simulations can help designers and analysts to evaluate response from the product or system, to ‘highly-off’ design conditions, e.g. evaluating the safety of a nuclear reactor facility in hurricane conditions etc.

What is involved in performing engineering simulations?

Animation of electromagnetic simulation showing development of magnetic flux lines.

Animation of Electromagnetic Simulation showing development of magnetic flux lines. Source http://www.ansys.com/solutions/electromagnetics.asp

The domain of engineering simulations is mainly divided based on the physics that is captured in the simulation. Based on this criterion, the domain is broadly categorized as follows

  1. Structural analysis Simulations
  2. Thermal & Fluids Simulations or Computational Fluid Dynamics (CFD) simulations
  3. Electronic Design Automation

In any type of engineering simulation, there are three main stages.

  1. Pre-processing: This involves determining the region in space that one will need to use in simulation. This region, (with all the necessary geometry details that are needed to attain the goals of the simulation) is then drafted on a computer using various CAD software tools. This space domain is then discretized into various small volumes or elements called computational cells. This is called meshing or gridding of the domain. Depending on the physics involved, the extent of the physical domain, the accuracy desired and the computational resources that are available, this mesh or grid can range from a few hundred cells to few hundred million cells. Ansys recently broke the Billion cell barrier.
  2. Solving the governing equations: This step involves solving the governing mathematical equations that describe the physics that one desires to capture at all the computational cells. Finite Element Method (FEM), Finite Volume Method (FVM) and Finite Difference Method (FDM) are the most commonly used numerical techniques that enable solving the governing partial differential equations on discretized domain on computers. To perform these calculations, many inputs need to be provided. The results achieved from the simulations are directly dependent on the inputs provided. Techniques of parallel computing enable use of multiple cpus on a network to be used for solving a single simulation. In this technique, if you have a mesh with say 2 million cells, and have a network cluster of 8 cpus, each CPU can solve equations for 0.25 million cells and thus the simulation time can be reduced significantly as compared to a single CPU solving equations for all 2 million cells.
  3. Post-processing: This stage involves understanding and analyzing the results of the simulations. The simulation software tools provide a mix of visual as well as alpha-numeric reporting of various variable of interest to the user so that the user can derive the information from the simulation needed to fulfill their objectives.

Most of the engineering simulation software tools provide connectivity to variety of CAD drafting packages so that geometries can be imported in from various different sources. Many of them provide ability to customize the solvers such that users can add their own/ proprietary physics/knowledge in the simulations. The post-processing allows results to be ported to various other analysis tools including optimization tools. Through these customizations, many users of these software tools have embedded the engineering simulation technology deep into their design process. Of the software tools available in the market, many are general tools that can be used by any industry vertical where as there are few tools that are developed for only one or few industry verticals and are easier to use to simulate applications in that particular industry.

Future of engineering simulations

Animation of FLUENT CFD simulation of flow over an elite male swimmer in the glide position. Source: http://www.fluent.com/news/pr/pr69.htm.

At present, most of the software tools available in the market are solving various physics involved in the real life process or equipment separately. However, in reality all these physics occur simultaneously and affect one another. The world of engineering simulations is moving rapidly towards incorporating multiple physics and their interactions to provide more reliable predictions. The engineering product development community is exploring, what is known as “Simulation Driven Product Development”, so that full benefits of engineering simulation technology can be leveraged to their competitive advantage. Some of the major software providers in this space have already started offering multi-physics solvers that enable organizations to march in this direction.

Another new facet that is coming in focus now is of knowledge management. Use of these simulation software tools, is generating a lot of engineering knowledge which the companies would like to leverage in their future design processes. With this need in mind, integrated engineering knowledge management platforms that will work seamlessly with the engineering simulation tools are being developed.

Engineering Simulations scene in Pune

Pune has attracted most of the main players in this exciting domain.

Ansys Inc, one of the leading companies in developing the engineering simulation software tools is present in Pune. Ansys has development, testing, sales, support and services functions for India based as well as worldwide customers being conducted out of the Pune office. Ansys develops structural analysis, CFD, EAD as well as optimization tools that are used widely in almost all industry segments.

In addition to Ansys, companies such as Siemens PLM, MSC Software, Abaqus also have presence in Pune.

Pune also has a growing list of companies that are using these engineering simulation softwares. Companies such as Cummins, John Deere, Dow Chemicals, Eaton Corp, Honeywell, etc have set up their product development centers in Pune which use these tools. Tata Motors, Tata Technologies, research wing of TCS, TRDDC and Tata’s new venture in HPC, CRL are also exploring the field of engineering simulations actively. Additionally engineering services consultants such as Tridiagonal, Pacific Mindware are also based in Pune.

Education institutes such as COEP too are now well equipped and have elective courses that allow students exposure to this interesting field.

About the author – Dr. Ajey Walavalkar

Ajey has over 10 years of experience in the Computational Fluid Dynamics industry. Currently, he is a Regional Support and Services Manager for Ansys Fluent India, in Pune. He has experience in successfully simulating various applications across industry domains, building teams of engineers for delivering CFD projects, support and services in an offshore setting. Ajey has a Ph.D. in Mechanical Engineering from Pennsylvania State University, USA, and a B.E. in Mechanical Engineering from COEP.

Reblog this post [with Zemanta]

Why Ruby is cool – take 2

Technology, , ,
Official Ruby logo
Image via Wikipedia

Earlier today, we had an article by Nick Adams on why you need to learn Ruby and Ruby-on-Rails. That post attracted a detailed comment by  Ravindra Jaju which elaborated on some of the themes touched upon in the original article. Since it brings out nicely the power of Ruby, we decided to post the comment as a new article to ensure it reaches all the readers who don’t necessarily read the comments. (By the way, you should subscribe to PuneTech comments by email or via RSS.)

Those interested in Ruby should check out Ruby Fun Day in Pune on Saturday, 21st February. If you want to learn Ruby, take a look at RubyLearning.org, a site that provides free, online courses for Ruby, by Pune’s own Satish Talim. Also check out the Pune Ruby mailing list, which is reasonably active.

This “article” only makes sense in the context of the original article, so ensure that you’ve read it before you read this:

Depending on where you come from, the syntax 5.times… may appear either beautiful or plain weird. But whatever the initial impression, one starts appreciating it as the power unfolds. The syntax ‘5.times’ might look like something completely out of the blue at first glance, but once one recognizes that 5 (or any number for that matter) is a first-class object, things start falling in place.

Try playing with this further using ‘irb’ (the ‘interactive ruby’ command-line tool which is a part of the ruby distribution) to explore the object ‘5? further as follows: 5.class, 5.methods, 5.methods.sort [send message ‘methods’ to 5, which returns an ‘Array’ – further send a message ‘sort’ to this returned Array object if you like to see the methods ordered]. And if you haven’t guessed this already, do note that brackets to function calls are absolutely optional if the syntax is unambiguous (so, the above is equivalent to

5.times(){puts “Hello Pune!”}

which is equivalent to

5.times(){puts(”Hello Pune!”)}

What comes within the braces ({}) after ‘5.times’ is a block – ‘5.times’ is by itself uninteresting, unless you can do something 5 times, no? Blocks are a very powerful concept, which are defined at one place but executed elsewhere. They have access to the context where they are defined (closures). Use of blocks (and associated Proc objects) can lead to very interesting, succinct code!

And again, if you are a C/C++ or a Java programmer feeling wary about jumping into something new and have that feeling of your uber-cool libraries not being accessible from Ruby (and hence a new learning curve apart from the heart-burn) – fear not! Ruby, being an interpreted language, runs in a VM of its own. And the Ruby VM developers have made sure that access to your favourite libraries is as easy as possible. The ‘official’ Ruby VM is C code, but obviously the cool thing that the JVM is, you can run Ruby inside the JVM (Java Virtual Machine) too. Many have already made these libraries accessible via Ruby. I would term the Java VM as more powerful in this regard, giving you access to native libraries apart from all the Java goodies (look up ‘ruby+ffi’ on yahoo/google search).

Then again, metaprogramming takes Ruby fun to newer heights. Rails wouldn’t have been (easily) possible had it not been for Ruby’s abilities in this regard. How would you feel if told that there are database access methods named (for example) ‘find_by_name’ or ‘find_by_name_city’ that aren’t defined anywhere but still create appropriate SQL calls (’where name = …’ or ‘where name = … and city = …’) because you can intercept ‘function undefined’ events, look at the function name and arguments, and create the right SQL on the fly? Or even before that, a class mapped to a database table ‘knows’ which table it is supposed to represent in an object form, just by looking at its *own* name? The good news is that all the cool libraries developed as part of the Rails initiative can be used in plain Ruby code without all the rails framework overhead too.

Well, enough said for a comment. You’ve got to experience it to believe it!

(PS: Not saying here that Ruby is unique – Python, for example, is cool too – but a Python-junkie would be a better marketeer for it 😉 )

Related articles by Zemanta
Reblog this post [with Zemanta]