Category Archives: In Depth

Optimization: A case study

In Depth, Live Blogging, Overviews, , , ,

(PuneTech is honored to have Dr. Narayan Venkatasubramanyan, an Optimization Guru and one of the original pioneers in applying Optimization to Supply Chain Management, as our contributor. I had the privilege of working closely with Narayan at i2 Technologies in Dallas for nearly 10 years.

PuneTech has published some introductory articles on Supply Chain Management (SCM) and the optimization & decision support challenges involved in various real world SCM problems. Who better to write about this area in further depth than Narayan!

For Dr. Narayan Venkatasubramanyan’s detailed bio, please click here.

This is the first in a series of articles that we will publish once a week for a month. For the full series of articles, click here.)

the following entry was prompted by a request for an article on the topic of “optimization” for publication in punetech.com, a website co-founded by amit paranjape, a friend and former colleague. for reasons that may have something to do with the fact that i’ve made a living for a couple of decades as a practitioner of that dark art known as optimization, he felt that i was best qualified to write about the subject for an audience that was technically savvy but not necessarily aware of the application of optimization. it took me a while to overcome my initial reluctance: is there really an audience for this after all, even my daughter feigns disgust every time i bring up the topic of what i do. after some thought, i accepted the challenge as long as i could take a slightly unusual approach to a “technical” topic: i decided to personalize it by rooting in a personal-professional experience. i could then branch off into a variety of different aspects of that experience, some technical, some not so much. read on …

background

the year was 1985. i was fresh out of school, entering the “real” world for the first time. with a bachelors in engineering from IIT-Bombay and a graduate degree in business from IIM-Ahmedabad, and little else, i was primed for success. or disaster. and i was too naive to tell the difference.

for those too young to remember those days, 1985 was early in rajiv gandhi‘s term as prime minister of india. he had come in with an obama-esque message of change. and change meant modernization (he was the first indian politician with a computer terminal situated quite prominently in his office). for a brief while, we believed that india had turned the corner, that the public sector companies in india would reclaim the “commanding heights” of the economy and exercise their power to make india a better place.

CMC was a public sector company that had inherited much of the computer maintenance business in india after IBM was tossed out in 1977. quickly, they broadened well beyond computer maintenance into all things related to computers. that year, they recruited heavily in IIM-A. i was one of an unusually large number of graduates who saw CMC as a good bet.

not too long into my tenure at at CMC, i was invited to meet with an mid-level manager in electronics & telecommunications department of the oil and natural gas commission of india (ONGC). the challenge he posed us was simple: save money by optimizing the utilization of helicopters in the bombay high oilfield.

the problem

the bombay high offshore oilfield, the setting of our story
the bombay high offshore oilfield, the setting of our story

the bombay high oilfield is about 100 miles off the coast of bombay (see map). back then, it was a collection of about 50 oil platforms, divided roughly into two groups, bombay high north and bombay high south.

(on a completely unrelated tangent: while writing this piece, i wandered off into searching for pictures of bombay high. i stumbled upon the work of captain nandu chitnis, ex-navy now ONGC, biker, amateur photographer … who i suspect is a pune native. click here for a few of his pictures that capture the outlandish beauty of an offshore oil field.)

movement of personnel between platforms in each of these groups was managed by a radio operator who was centrally located.

all but three of these platforms were unmanned. this meant that the people who worked on these platforms had to be flown out from the manned platforms every morning and brought back to their base platforms at the end of the day.

at dawn every morning, two helicopters, flew out from the airbase in juhu, in northwestern bombay. meanwhile, the radio operator in each field would get a set of requirements of the form “move m men from platform x to platform y”. these requirements could be qualified by time windows (e.g., need to reach y by 9am, or not available for pick-up until 8:30am) or priority (e.g., as soon as possible). each chopper would arrive at one of the central platforms and gets its instructions for the morning sortie from the radio operator. after doing its rounds for the morning, it would return to the main platform. at lunchtime, it would fly lunchboxes to the crews working at unmanned platforms. for the final sortie of the day, the radio operator would send instructions that would ensure that all the crews are returned safely to their home platforms before the chopper was released to return to bombay for the night.

the challenge for us was to build a computer system that would optimize the use of the helicopter. the requirements were ad hoc, i.e., there was no daily pattern to the movement of men within the field, so the problem was different every day. it was believed that the routes charted by the radio operator were inefficient. given the amount of fuel used in these operations, an improvement of 5% over what they did was sufficient to result in a payback period of 4-6 months for our project.

this was my first exposure to the real world of optimization. a colleague of mine — another IIM-A graduate and i — threw ourselves at this problem. later, we were joined yet another guy, an immensely bright guy who could make the lowly IBM PC-XT — remember, this was the state-of-the-art at that time — do unimaginable things. i couldn’t have asked to be a member of a team that was better suited to this job.

the solution

we collected all the static data that we thought we would need. we got the latitude and longitude of the on-shore base and of each platform (degrees, minutes, and seconds) and computed the distance between every pair of points on our map (i think we even briefly flirted with the idea of correcting for the curvature of the earth but decided against it, perhaps one of the few wise moves we made). we got the capacity (number of seats) and cruising speed of each of the helicopters.

we collected a lot of sample data of actual requirements and the routes that were flown.

we debated the mathematical formulation of the problem at length. we quickly realized that this was far harder than the classical “traveling salesman problem”. in that problem, you are given a set of points on a map and asked to find the shortest tour that starts at any city and touches every other city exactly once before returning to the starting point. in our problem, the “salesman” would pick and/or drop off passengers at each stop. the number he could pick up was constrained, so this meant that he could be forced to visit a city more than once. the TSP is known to be a “hard” problem, i.e., the time it takes to solve it grows very rapidly as you increase the number of cities in the problem. nevertheless, we forged ahead. i’m not sure if we actually completed the formulation of an integer programming problem but, even before we did, we came to the conclusion that this was too hard of a problem to be solved as an integer program on a first-generation desktop computer.

instead, we designed and implemented a search algorithm that would apply some rules to quickly generate good routes and then proceed to search for better routes. we no longer had a guarantee of optimality but we figured we were smart enough to direct our search well and make it quick. we tested our algorithm against the test cases we’d selected and discovered that we were beating the radio operators quite handily.

then came the moment we’d been waiting for: we finally met the radio operators.

they looked at the routes our program was generating. and then came the first complaint. “your routes are not accounting for refueling!”, they said. no one had told us that the sorties were long enough that you could run out of fuel halfway, so we had not been monitoring that at all!

Dhruv
ONGC’s HAL Dhruv Helicopters on sorties off the Mumbai coast. Image by Premshree Pillai via Flickr

so we went back to the drawing board. we now added a new dimension to the search algorithm: it had to keep track of fuel and, if it was running low on fuel during the sortie, direct the chopper to one of the few fuel bases. this meant that some of the routes that we had generated in the first attempt were no longer feasible. we weren’t beating the radio operators quite as easily as before.

we went back to the users. they took another look at our routes. and then came their next complaint: “you’ve got more than 7 people on board after refueling!”, they said. “but it’s a 12-seater!”, we argued. it turns out they had a point: these choppers had a large fuel tank, so once they topped up the tank — as they always do when they stop to refuel — they were too heavy to take a full complement of passengers. this meant that the capacity of the chopper was two-dimensional: seats and weight. on a full tank, weight was the binding constraint. as the fuel burned off, the weight constraint eased; beyond a certain point, the number of seats became the binding constraint.

we trooped back to the drawing board. “we can do this!”, we said to ourselves. and we did. remember, we were young and smart. and too stupid to see where all this was going.

in our next iteration, the computer-generated routes were coming closer and closer to the user-generated ones. mind you, we were still beating them on an average but our payback period was slowly growing.

we went back to the users with our latest and greatest solution. they looked at it. and they asked: “which way is the wind blowing?” by then, we knew not to ask “why do you care?” it turns out that helicopters always land and take-off into the wind. for instance, if the chopper was flying from x to y and the wind was blowing from y to x, the setting was perfect. the chopper would take off from x in the direction of y and make a bee-line for y. on the other hand, if the wind was also blowing from x to y, it would take off in a direction away from y, do a 180-degree turn, fly toward and past y, do yet another 180-degree turn, and land. given that, it made sense to keep the chopper generally flying a long string of short hops into the wind. when it could go no further because they fuel was running low or it needed to go no further in that direction because there were no passengers on board headed that way, then and only then, did it make sense to turn around and make a long hop back.

“bloody asymmetric distance matrix!”, we mumbled to ourselves. by then, we were beaten and bloodied but unbowed. we were determined to optimize these chopper routes, come hell or high water!

so back we went to our desks. we modified the search algorithm yet another time. by now, the code had grown so long that our program broke the limits of the editor in turbo pascal. but we soldiered on. finally, we had all of our users’ requirements coded into the algorithm.

or so we thought. we weren’t in the least bit surprised when, after looking at our latest output, they asked “was this in summer?”. we had now grown accustomed to this. they explained to us that the maximum payload of a chopper is a function of ambient temperature. on the hottest days of summer, choppers have to fly light. on a full tank, a 12-seater may now only accommodate 6 passengers. we were ready to give up. but not yet. back we went to our drawing board. and we went to the field one last time.

in some cases, we found that the radio operators were doing better than the computer. in some cases, we beat them. i can’t say no creative accounting was involved but we did manage to eke out a few percentage point of improvement over the manually generated routes.

epilogue

you’d think we’d won this battle of attrition. we’d shown that we could accommodate all of their requirements. we’d proved that we could do better than the radio operators. we’d taken our machine to the radio operators cabin on the platform and installed it there.

we didn’t realize that the final chapter hadn’t been written. a few weeks after we’d declared success, i got a call from ONGC. apparently, the system wasn’t working. no details were provided.

i flew out to the platform. i sat with the radio operator as he grudgingly input the requirements into the computer. he read off the output from the screen and proceeded with this job. after the morning sortie was done, i retired to the lounge, glad that my work was done.

a little before lunchtime, i got a call from the radio operator. “the system isn’t working!”, he said. i went back to his cabin. and discovered that he was right. it is not that our code had crashed. the system wouldn’t boot. when you turned on the machine, all you got was a lone blinking cursor on the top left corner of the screen. apparently, there was some kind of catastrophic hardware failure. in a moment of uncommon inspiration, i decided to open the box. i fiddled around with the cards and connectors, closed the box, and fired it up again. and it worked!

it turned out that the radio operator’s cabin was sitting right atop the industrial-strength laundry room of the platform. every time they turned on the laundry, everything in the radio room would vibrate. there was a pretty good chance that our PC would regress to a comatose state every time they did the laundry. i then realized that this was a hopeless situation. can i really blame a user for rejecting a system that was prone to frequent and total failures?

other articles in this series

this blog entry is intended to set the stage for a series of short explorations related to the application of optimization. i’d like to share what i’ve learned over a career spent largely in the business of applying optimization to real-world problems. interestingly, there is a lot more to practical optimization than models and algorithms. each of the the links below leads to a piece that dwells on one particular aspect.

optimization: a case study (this article)
architecture of a decision-support system
optimization and organizational readiness for change
optimization: a technical overview

About the author – Dr. Narayan Venkatasubramanyan

Dr. Narayan Venkatasubramanyan has spent over two decades applying a rare combination of quantitative skills, business knowledge, and the ability to think from first principles to real world business problems. He currently consults in several areas including supply chain and health care management. As a Fellow at i2 Technologies, he tackled supply chains problems in areas as diverse as computer assembly, semiconductor manufacturer, consumer goods, steel, and automotive. Prior to that, he worked with several airlines on their aircraft and crew scheduling problems. He topped off his days at IIT-Bombay and IIM-Ahmedabad with a Ph.D. in Operations Research from the University of Wisconsin-Madison.

He is presently based in Dallas, USA and travels extensively all over the world during the course of his consulting assignments. You can also find Narayan on Linkedin at: http://www.linkedin.com/in/narayan3rdeye

Reblog this post [with Zemanta]

Improve your web based software development and maintenance ROI with dynamic programming languages

In Depth, Technology, , , ,

After we carried a few quick articles on why you should learn more about Ruby and Ruby on Rails (take 1, take 2) last month, we decided that we wanted to give people a much deeper article on why these new languages (Ruby, Python, PHP) and frameworks (Rails, Django) are setting the web world on fire. We invited Dhananjay Nene to write an article with an in depth discussion of the technical reasons how these new languages differ from the older ones and when to choose one over the other. He responded with this article which, as an added bonus, also includes the business reasons for your decisions. At the request of the community, Dhananjay is also giving a talk on the relative strengths and weaknesses of different programming languages on Saturday, 28th March, 4pm, at SICSR. All those who found this article interesting should definitely attend.

Introduction

Programing language selection is often a topic that elicits a lot of excitement, debate and often a bit of acrimony as well. There is no universally superior programming language that one can recommend, so I tend to generally disregard most language opinions which say ‘X language is the best’, without specifying the context under which it is superior. Finally most language debates often deal with the technical issues and not the ROI issues. Hopefully I shall be able to address this topic without being guilty of any of these problems.

So what languages are we referring to here ?

Official Ruby logo
Image via Wikipedia

The range of languages that fall under Dynamic Programming Languages category is rather extensive. My experience is primarily limited to Python and to a lesser extent PHP, Ruby, Javascript, and Groovy. For the rest of this article, I shall be primarily referring to Python or Ruby when I use the word dynamic languages, though many of the references may continue to be applicable and relevant for a number of other dynamic programming languages.

As I describe the technical characteristics, I shall also continue to attempt to address the business aspects as well, so you might find this article at a little techno-business level. Assuming I am able to excite their interest, the tech guys would not find sufficient technical details and would be hungry to hunt for more, and while the business guys would get a little teased with the possibilities, they will not quite get the ROI served in the traditionally formatted excel spreadsheets. Being aware of that, I continue down this path with a feeling that this perhaps will be the most appropriate level for me to abstract this article to.

Characteristics of Dynamic Programming Languages.

Let us quickly review some of the characteristics :

CPython
Image via Wikipedia

Object Oriented : Many dynamic languages support full object orientation. There are many who don’t necessarily buy the benefits of Object Orientation, but it is my strong belief, that once a piece of software grows beyond a certain threshold of complexity and / or size, Object Orientation starts delivering very strong dividends. There are a few areas such as highly complex, algorithmic processing which might be better suited for functional programming. However a majority of the medium-to-large sized web applications are better served by OO. The empirical evidence at least bears out the fact that most of the most popular languages today (except C) are Object Oriented. However this still is a very very large class of languages which in them include C++, Java, PHP, Python, Ruby etc. The one area where some dynamic languages separate themselves from the others is in the notion of “everything is an object”, ie. primitives such as numbers, functions are all objects by themselves.

Business implications: OO code well designed and implemented allows for a substantial reduction in maintenance costs. When working with a team which is up the curve on OO, it is likely to lead to lower costs and time on inital coding as well. On the other hand, both training costs and skill requirements are higher for fully OO languages. If you are already using partialy OO / hybrid languages such as PHP, C++ or Java, and are convinced about OO, using fully OO languages such as Python or Ruby will help you leverage the OO capabilities even further.

Duck Typing : In very loose terms, duck typed languages do not require you to declare an explicit interface. You send an object a message (ie. invoke a function or access an attribute) and if it can respond to it, it will, and if it can’t it will result in an error. Duck typing is a specific typing system which is a subset of a broader system called Dynamic Typing, which often makes for an interesting debate with its counterpart – Static typing : Static and Dynamic Type checking in practice. For people well grounded in static typing alone, this can sometimes seem to be sacrilegious. I am convinced that duck typing makes writing code much much faster for two reasons – a) You now require to write fewer lines of code and b) You often don’t have to keep on regularly waiting for the compiler to do its work. There is also a substantial capability enhancement that dynamic typing makes to the language type system, which allow the frameworks to build dynamic types on the fly. This in turn offers the framework users many more capabilities than frameworks written in other languages. That is why it is nearly impossible to write frameworks like Rails or Django in Java (You can modify the class loaders and use byte code generation to generate the new types, but the compiler can’t see them so you cant use them). That is also why there is a lot of anticipation of using JRuby, Jython and Grails on the JVM since the languages underlying them (Ruby, Python and Groovy respectively) bring the dynamic typing capabilities to the JVM platform.

Business Implications :Writing code is much much faster. Maintenance depending upon the situation can sometimes be more or less difficult in case of dynamic typed languages. Refactoring is usually a lot more difficult in case of dynamically typed languages since the underlying type system is not able to infer sufficiently about the code to help the refactoring tools, as is possible in case of statically typed languages. It is my opinion that a skilled and trained development team using dynamic languages can generally substantially outperform another equally capable team using static languages. Insufficiently or poorly skilled development teams however can lead to very very different kind of pitfalls in these class of languages. In both cases the code becomes difficult to change or maintain due to a) cryptic code in case of dynamically typed languages and b) extremely large code bases in case of statically typed languages. Both are undesirable situations to be in but if I had to choose between one of the two, I would go for being in the cryptic mess since it is at least manageable by bringing in external skilled help.

Metaprogramming : Metaprogramming is in loose terms the ability of programs to write programs. A large proportion of developers may not use this capability too frequently. Specifically in web application development it gets used as a mechanism to transform one set of datastructures which a programmer specifies into code at runtime. As I point out later in this article, it in fact is a very important element in designing common frameworks and libraries which in turn offer substantial capabilities including small code and easier maintenance. A quick note to state that metaprogramming is not code generation. In case of code generation, one uses the generator to generate code which is then compiled. A big limitation with this is the fact that often people modify the generated code leading to really tough maintenance nightmares and the fact that it is a two stage process which is prone to more errors. Metaprogramming results in new code “coming to life” so to speak while your program is running.

Business Implications : Read on, they will get covered in the final roundup. They are large and they are positive.

Function blocks/objects, iterators, closures, continuations, generators: I will not go into any substantial details of this issue except to say that small pieces of code logic can be handled in a much much more concise way than if these weren’t supported. While many situations may not need closures support, you will be glad to have them on your side when needed.

Business Implications : Helps having shorter, cleaner code leading to lesser development and maintenance costs. Another significant positive is that your developers are just likely to be so much happier since they get some truly nice building blocks for concise and elegant expression of their logic. Can’t think of any significant negatives.

There are a full range of other capabilities, but none come to mind immediately as something that have strong business implications as well.

The role of frameworks

Ruby on Rails
Image via Wikipedia

When did these languages say Ruby and Python originate ? Most people are likely to be a little surprised if the answer is in the last millenium. Yet Guido von Rossum started working on Python in 1986 and Ruby was released in 1992. Python has been rather well known within the scientific community and perhaps a bit within the systems / OS utility programming communities for quite some time. However both languages grabbed a large mindshare only post 2005. A big reason for their popularity (especially in case of Ruby’s case) came from the popularity the frameworks which used them. Ruby on Rails for ruby and Django (to the best of my knowledge) for python. These frameworks combined the language capabilities with the learnings of good design practices for internet applications (eg MVC, declarative validations, simple ORM etc) into a simple usable package, which developers could take and build web applications quickly. There are examples of people having built simple web apps within a day and medium complexity apps in 1-3 weeks using these frameworks. The languages are the ingredients, the frameworks are the cooks – a great combination for serving great meals. Now you will find many such frameworks in these languages, including some which have better capabilities for building more sophisticated / complex applications eg. Merb and Pylons.

I am not too sure of how many people are exactly aware of the role of metaprogramming in the frameworks’ successes. I am willing to believe that but for metaprogramming, these frameworks simply would not have achieved anywhere close to the success they achieved. It is metaprogramming which takes the datastructures as defined by a developer and converts it into runtime code implicitly, saving the developer lots of time and effort. So even if most developers don’t actively write metaprograms, their lives are so much easier. Metaprogramming capabilities are also the reason why it is virtually impossible to write similar frameworks in Java. However if you are on the .NET or JVM environments, things are definitely looking encouraging with the possibilities to use IronPython or IronRuby on .NET or JRuby or Jython or Groovy+Grails on the JVM.

Business implications : If you are focused on scientific or desktop or highly algorithmic applications, where python especially is used extensively, you are likely to get benefits from these languages on their own merit alone. For web applications you will see the maximum benefits by using the web MVC frameworks along with the languages. I submit that on the whole you are likely to see very substantial reduction in development, enhancement and maintenance times – sweet music for any end user, investor or project manager.

Increased Business Agility

There is one more reason why I believe these languages are especially helpful. They help by increasing development agility to an extent where it now allows for the business to be more agile. You can get a first prototype version up in weeks, take it around to potential users, and gather feedback on the same. Incorporate elements of this feedback into the next release of working code quickly. The business benefits of such a scenario are tremendous. You might wonder that this is a process issue, so what does it have to do with a language selection. I would submit, that languages which allow changes to be made faster, help support this process in a far superior way. Another equally important facet is the superior risk management. Since you are able to build features with lower investments, you are able to get a series of customer feedbacks into your decision making process much faster. This helps being able to come up with a product that really meets the customer expectations much earlier. This happens by allowing the better features to come in earlier and also by allowing the lesser important or lesser relevant features to be decided to be deferred earlier. That’s precisely the reason why the dynamic languages have found a strong acceptance in the startup world. I believe the increasing agility which is often required in the startup world, is and will continue to be increasingly required of established enterprises. Precisely the reason why I believe these languages will continue to do better in the enterprise space as well. Finally, these languages make it relatively easier to tell your business sponsor – We will work with you on imprecise requirements rather than spending months on nailing down requirements which anyways are likely to change later. This has both a pro and a con especially for outsourcing situations. It is likely to allow for tremendous customer delight in terms of a vendor that works with him in such a flexible manner, yet it does introduce challenges in terms of how the commercials and management of the project are handled.

The reason I would like to especially point out increased business agility is because programmers don’t often visualise or evangelise it much, but when I wear a manager’s hat, it is perhaps the most compelling benefit of these languages.

Concluding

As I said earlier, there is no single universal language which is the best for all scenarios. There are some scenarios where using dynamic languages will not be helpful

Programming language book sales 4Q2008

A Treemap view of sales of programming language books by O’Reilly Media in 4Q2008. The size of a box represents the total sales of a book. The color represents the increase or decrease in sales compared to same quarter in 2007. Green = increase, bright green = big increase, red = decrease, bright red = large decrease. See full article at O’Reilly Radar for lots of interesting details.

When not to use these languages

  • You are building a simple / small application and don’t have the available skill sets. One exception to this is where you decide to use it in a simple application to allow yourself a non risky mechanism of building these skillsets.
  • Extremely High performance requirements. However please make sure that you really need the high performance capabilities of say a C, C++ or Java. In my experience 80% of developers like to believe that they are building highly performant applications where the maximum speed is a must have. Yet the top 10% of them are facing far far more critical performance requirements than the remainder. Unless you are convinced you are in the top 10%, you should certainly consider dynamic languages as an option. Moreover in case of most high performance requirements, these can sometimes be boiled down to a few inner loops / algorithms. Consider implementing the same in C, / Java or other .NET languages (depending upon the choice of your dynamic language interpreter implementation)
  • You have an architecture standard in place which does not allow using these languages. If you are convinced your applications are better served by using dynamic languages both from your individual application and an overall enterprise perspective, consider taking the feedback to your standards setting body to see if you can pilot a different approach. Also evaluate if the .NET or JVM versions can help you comply with the architecture guidelines.
  • You are unable to commit to the retraining requirements. While these languages are easy and powerful to use, leveraging that power can require some amount of retraining. If that does not fit your business plans, since the retraining effort could impact immediate and urgent requirements, that could be a reason to not use these languages. However in such situations do consider investing in building this skill sets before you get to another similar decision point.
  • You need a very high levels of multithreadinging as opposed to multi processing support. While this is not a typical situation for web applications, you should be aware that most dynamic languages have some limitations in terms of multi threading support. This actually is not necessarily an issue with the language as with the implementation eg. the C implementation of python has the notorious Global Interpreter Lock which constrains you from being able to use more than a handful of threads per processes efficiently. However the same restriction is not present in Jython (the jvm implementation of python). This is likely to be an issue for a miniscule percentage of the web applications market for the primary reason that multi process / shared nothing architecture styles often work quite well for many web applications and they don’t really need multi threading.

So where’s my return on investment ?

First of all lets talk of the investment part. If you get into it in a paced approach, the investment may not be that great. Start with a team size of anywhere between 2-6 people (depending upon your organisation and project size). Think of 15 days of intensive training followed by a 2-6 months coming up the curve effort (more likely 2 than 6). Make sure your first project is not a critical one under tremendous business pressure. This can be subsequently followed by more people getting retrained as necessary. In the longer term it might actually help reduce your incremental investment, since it might be much easier to ramp up new programmers in Ruby or Python than say Java or C#.

Secondly lets look at the incrementally higher costs. You are likely to need people who are a little bit more capable in terms of understanding and debugging the same logic expressed in fewer lines of code (that sometimes can be a challenge) and then be able to modify and enhance the same. This may increase your testing and fixing costs in the earlier days. Finally while the fewer lines of code can make refactoring easier, you could find that your total refactoring costs are a little higher.

Now the returns part. I am convinced that the increased business agility is the strongest return in business terms. Immediately after that is the substantial reduction in development, enhancement and maintenance times. If neither of these benefits are appealing, when contrasted with some other issues that you might perceive, maybe considering dynamic languages in your context is not such a great idea.

One more factor that I would of course encourage you to evaluate from a business perspective are the implications for you if your competition (assuming it is not already using them) started using these languages. The implications would vary from case to case, but it could also help you decide how important this issue is for you.

About the author – Dhananjay Nene

Dhananjay is a Software Engineer with around 17 years of experience in the field. He is passionate about software engineering, programming, design and architecture. He did his post graduation from Indian Institute of Management, Ahmedabad, and has been involved in Senior Management positions and has managed team sizes in excess of 120 persons. His tech blog, and twitter stream are a must read for anybody interested in programming languages or development methodologies. Those interested in the person behind the tech can check out his general blog, and personal twitter stream. For more details, check out Dhananjay’s PuneTech wiki profile.

Related articles by Zemanta
Reblog this post [with Zemanta]

Introduction to Engineering Simulations (CAE)

In Depth, Technology, , , , ,

(This article giving an overview of the field of Computer Aided Engineering has been written for PuneTech on our request by Dr. Ajey Walavalkar, regional manager of services and support at Ansys Fluent India, a company specializing in CAE and CFD applications and services, which has a development center in Pune. See the end of this article for more about Ajey.)

In the earlier PuneTech article “An Overview of CAD,” Yogesh and Amit have provided a very good overview of the CAD technology spectrum. There, we learnt where and how “Analysis” and “Simulations” fit in the overall scheme of CAD. This article is intended to provide a broad overview of the area called Engineering Simulations, which spans across the above mentioned “Analysis” and “Simulations” fields on the CAD canvas.

What are Engineering Simulations?

The analysis of vibrations and the dynamical behaviour due to excitation forces have been applied routinely to cable-stayed bridges using SMR devised analysis tools. Note that similar techniques have been used to analyse the resonance behaviour of large floating bridge structures.

The example shows an eigenmode of a large cable-stayed bridge (model and analysis with B2000 by The Dutch National Aerospace Research Institute (NLR)). The figure displays a vibration mode, the shape being artificially amplified to emphasize the deformation. Source http://www.smr.ch/services/csm

Let’s say you want to build a bridge. Not just any bridge, but a massive suspension bridge to rival the Golden Gate Bridge in San Francisco Bay Area. How do you decide the type of steel, the span length, the tower height, the thickness of the cables, the depth of the foundations, and other design parameters? You will wonder that if this problem was solved in the 1930s then why is it tough today? The simple answer is, ‘Solved? Yes! But at what cost and effort?’ Previously, the simple solution for most tough engineering problems was to ‘over-engineer’ the solution by building in a ‘huge factor of safety’. Today, this design process is lot more accurate and efficient. Today, the engineering team will start off by considering the effects of vehicles (weights and speeds) plying on the bridge, the wind forces that will sway the bridge, the waves that will hit the foundation, the steady long-term corrosive effects of weather, etc. These effects can studied by mathematically modeling these factors and ‘simulating’ them with a help of a computer. Such ‘simulations’ greatly help today’s engineers to come up with the exact design that is safe and that saves time and costs.

Wikipedia defines Simulation as follows:

Simulation is the imitation of some real thing, state of affairs, or process. The act of simulating something generally entails representing certain key characteristics or behaviors of a selected physical or abstract system.

Engineering simulations are simulations applied to engineering problems. These can range from simple “back of the envelope” calculations to full fledged systems simulations using High Performance Computing.
Mathematical modeling and physical modeling are two main aspects of engineering simulations. Mathematical modeling refers to the description of a particular phenomenon in terms of mathematical equations. E.g. If you wish to find out the distance a canon ball will travel when fired from a canon at certain angle, at a certain speed; you can write some mathematical expressions and solve them to calculate this distance. This is mathematical modeling. Now suppose you are firing that canon ball at a mountain cliff and the ball is going to bounce off and roll down the slopes of the mountain, how do you find out where that ball will end up? Here you need to also consider the physical structures and that leads you to physical modeling.

Uses of Engineering Simulations

Today, computer aided engineering simulations are used extensively in designing and development of almost all industrial products. In addition to product designing, simulations find their uses in troubleshooting of existing systems and research & development of new processes. Many times the field of computer aided engineering simulations is also termed as Computer Aided Engineering (CAE). There are numerous software products available in the market and they offer variety of engineering simulation capabilities. Many organizations also use their in-house built simulation software products that capture their time proven design practices.

There are many objectives in performing engineering simulations:

  1. During product design cycle, engineering simulations enable the designers to evaluate various design options without getting into costly prototyping and testing. Use of simulations in the design process can help narrow the design choices to very small set that can be taken to the prototyping and testing phases. Even with physical testing, one can inspect and measure only a small number of variables at few select locations. Whereas simulations can provide visual information on all the variables of interest at all locations inside the simulated space.
  2. Simulations can help troubleshoot deviation of actual performance of system/product from the desired one.
  3. Simulations can help designers and analysts to evaluate response from the product or system, to ‘highly-off’ design conditions, e.g. evaluating the safety of a nuclear reactor facility in hurricane conditions etc.

What is involved in performing engineering simulations?

Animation of electromagnetic simulation showing development of magnetic flux lines.

Animation of Electromagnetic Simulation showing development of magnetic flux lines. Source http://www.ansys.com/solutions/electromagnetics.asp

The domain of engineering simulations is mainly divided based on the physics that is captured in the simulation. Based on this criterion, the domain is broadly categorized as follows

  1. Structural analysis Simulations
  2. Thermal & Fluids Simulations or Computational Fluid Dynamics (CFD) simulations
  3. Electronic Design Automation

In any type of engineering simulation, there are three main stages.

  1. Pre-processing: This involves determining the region in space that one will need to use in simulation. This region, (with all the necessary geometry details that are needed to attain the goals of the simulation) is then drafted on a computer using various CAD software tools. This space domain is then discretized into various small volumes or elements called computational cells. This is called meshing or gridding of the domain. Depending on the physics involved, the extent of the physical domain, the accuracy desired and the computational resources that are available, this mesh or grid can range from a few hundred cells to few hundred million cells. Ansys recently broke the Billion cell barrier.
  2. Solving the governing equations: This step involves solving the governing mathematical equations that describe the physics that one desires to capture at all the computational cells. Finite Element Method (FEM), Finite Volume Method (FVM) and Finite Difference Method (FDM) are the most commonly used numerical techniques that enable solving the governing partial differential equations on discretized domain on computers. To perform these calculations, many inputs need to be provided. The results achieved from the simulations are directly dependent on the inputs provided. Techniques of parallel computing enable use of multiple cpus on a network to be used for solving a single simulation. In this technique, if you have a mesh with say 2 million cells, and have a network cluster of 8 cpus, each CPU can solve equations for 0.25 million cells and thus the simulation time can be reduced significantly as compared to a single CPU solving equations for all 2 million cells.
  3. Post-processing: This stage involves understanding and analyzing the results of the simulations. The simulation software tools provide a mix of visual as well as alpha-numeric reporting of various variable of interest to the user so that the user can derive the information from the simulation needed to fulfill their objectives.

Most of the engineering simulation software tools provide connectivity to variety of CAD drafting packages so that geometries can be imported in from various different sources. Many of them provide ability to customize the solvers such that users can add their own/ proprietary physics/knowledge in the simulations. The post-processing allows results to be ported to various other analysis tools including optimization tools. Through these customizations, many users of these software tools have embedded the engineering simulation technology deep into their design process. Of the software tools available in the market, many are general tools that can be used by any industry vertical where as there are few tools that are developed for only one or few industry verticals and are easier to use to simulate applications in that particular industry.

Future of engineering simulations

Animation of FLUENT CFD simulation of flow over an elite male swimmer in the glide position. Source: http://www.fluent.com/news/pr/pr69.htm.

At present, most of the software tools available in the market are solving various physics involved in the real life process or equipment separately. However, in reality all these physics occur simultaneously and affect one another. The world of engineering simulations is moving rapidly towards incorporating multiple physics and their interactions to provide more reliable predictions. The engineering product development community is exploring, what is known as “Simulation Driven Product Development”, so that full benefits of engineering simulation technology can be leveraged to their competitive advantage. Some of the major software providers in this space have already started offering multi-physics solvers that enable organizations to march in this direction.

Another new facet that is coming in focus now is of knowledge management. Use of these simulation software tools, is generating a lot of engineering knowledge which the companies would like to leverage in their future design processes. With this need in mind, integrated engineering knowledge management platforms that will work seamlessly with the engineering simulation tools are being developed.

Engineering Simulations scene in Pune

Pune has attracted most of the main players in this exciting domain.

Ansys Inc, one of the leading companies in developing the engineering simulation software tools is present in Pune. Ansys has development, testing, sales, support and services functions for India based as well as worldwide customers being conducted out of the Pune office. Ansys develops structural analysis, CFD, EAD as well as optimization tools that are used widely in almost all industry segments.

In addition to Ansys, companies such as Siemens PLM, MSC Software, Abaqus also have presence in Pune.

Pune also has a growing list of companies that are using these engineering simulation softwares. Companies such as Cummins, John Deere, Dow Chemicals, Eaton Corp, Honeywell, etc have set up their product development centers in Pune which use these tools. Tata Motors, Tata Technologies, research wing of TCS, TRDDC and Tata’s new venture in HPC, CRL are also exploring the field of engineering simulations actively. Additionally engineering services consultants such as Tridiagonal, Pacific Mindware are also based in Pune.

Education institutes such as COEP too are now well equipped and have elective courses that allow students exposure to this interesting field.

About the author – Dr. Ajey Walavalkar

Ajey has over 10 years of experience in the Computational Fluid Dynamics industry. Currently, he is a Regional Support and Services Manager for Ansys Fluent India, in Pune. He has experience in successfully simulating various applications across industry domains, building teams of engineers for delivering CFD projects, support and services in an offshore setting. Ajey has a Ph.D. in Mechanical Engineering from Pennsylvania State University, USA, and a B.E. in Mechanical Engineering from COEP.

Reblog this post [with Zemanta]

TouchMagix: Convert any wall or floor into an interactive touchscreen

In Depth, Interview, , , ,

TouchMagix is a Pune-based startup with a product that can convert any wall of floor into an interactive touch screen. Targeted towards “high footfall” areas like malls, conferences and lobbys, it is a technology with a lot of potential, and frankly, something that we don’t expect to see out of Indian companies. After being very impressed with the demo at their Bhonsale Nagar office, we interviewed founder Anup Tapadia for PuneTech. TouchMagix technology will be on display on both days of the Hi Tech Pune Maharashtra 2009 conference in Pune on 26th and 27th – it’s definitely worth checking out. Anup will also be speaking at the conference on Friday. Read on for the interview.


Q: Can you give an overview of TouchMagix?

A: TouchMagix is a technology that allows any projection screen to be made interactive. Imagine a picture being projected on a regular wall, and a person can walk up to the wall and start interacting with the objects in the picture, and they actually react to the touch. TouchMagix technology allows adding such multi-point interactivity to any projected surface i.e walls, floors or screens. The projected surface reacts to the human gestures. It reacts to a touch, or it can react to gestures from a distance. And this can be done with any existing surface.

It is a medium that attracts crowds of all ages and helps brands “Get Noticed” with fun and interaction. It has extensive applications in retail, malls, advertising, events, promotions and gaming. We believe that this is the way brands would communicate to their consumers in future. At TouchMagix we are focussed towards building technologies and services which will aid brand owners to create an everlasting impression in consumers mind. Right now we have launched this product, and our team is overwhelmed with the response from all over the world. Now the real challenge we are facing is to accelerate the process of deployment across the globe in short amount of time. We are looking for awesome people to join this wagon!

Demo of TouchMagix. Click here if you can’t see the video.
Q: Can you dig a little into the technology that goes into making TouchMagix?

As off now we have 2 products, MagixFloor™ and MagixWall™ which are like large multi-point touch-screens. Both the technologies use same set of hardware and software. The hardware comprises of a high-end PC, a projector and our proprietary TouchMagix sensor. The sensor feeds the signals to the PC software, which recognizes human gestures and generates an XML feed on loopback interface. This feed can be integrated with any designing software like Adobe Flash, Adobe Director, 3DS Max etc… to build games and content. We offer an Open SDK interface which can be used by creative agencies or game designers to build content for deployment on TouchMagix. We also provide a remote controlled media scheduler for rotating content on the media. We are also in the process of designing audience measurement tools that will give web based interaction statistics to an advertiser.

Q: There are existing systems in US, Israel and elsewhere which do similar things. For example, Reactrix Systems. How do you differentiate yourself from them, both from an IPR perspective and a marketing perspective.

We have a superior and cost effective technology than most of the competition around. Technology, is just one aspect of the media. The possibilities with the content that one can build on this media would establish our position. We are offering an Open SDK interface for developers that no other competitor is offering. We are also introducing a feature of audience measurement, and web management which will set us apart. Reactrix systems was a close competitor for us, but they recently shutdown despite investment of $75 Million. Any good idea badly executed can also lead to failure. Being in India, we have an cost advantage along with the technology edge to deliver this product to the world. We have a team with diverse experience in technology, advertising and gaming and we believe we can make this happen.

Q: What were the unexpected challenges that you faced in making this happen.

A: Loads of challenges surfaced during the process of development of this technology. It was like playing the game of snakes and ladders for last 2 years day-in and day-out. What we thought was working in the labs and we said hurray, was a basket of new challenges on the field. Elimination of pseudo sensing was one of the big challenges that we had to overcome.

Pseudo sensing is a problem in which the movements of the objects that are in the picture being projected need to be distinguished from the motion of the human. Earlier the system used to do false detections on the projected image thinking it was the human motion. Now we have overcome that problem.

Q: Unlike most other “IT” and “web” startups, your technology has a significant hardware component. This is not something common in India. Has that been a challenge for you?

A: I was fortunate to get an experience in both hardware and software during my journey of learning. My dad always says, “You are a jack of all trades, and master of some!”. This trait helped me in integrating hardware and software required to create the product. Having our own hardware solved lot of problems, which would have been difficult otherwise. It also helped in building the intellectual property around it.

Q: What conventional (and unconventional) means, you are trying out for marketing your product?

A: Basically we have 3 types of customers.

  1. Private companies advertising their own product in private spaces. e.g – An mobile company putting it in their every showroom.
  2. Public place owners where advertisers would like to advertise their products. e.g – A mall who would lease out their space to advertisers.
  3. Short term installations for events, exhibitions, activations, promotions, parties etc.

We would be handling 1st type of customers by direct marketing initiatives. 2nd and 3rd type of customers would be handled through channel partners who have local presence. In addition to this, we will also be doing online direct marketing.

Q: What funding options have you considered?

A: The first phase was funded by Family and Friends. Along with the business plan, the next phase funding plan also would be frozen.

Q: At a very young age, you have done a lot of interesting and varied things? Can you tell us a bit about your background and the various areas you’ve worked in?

A: I started exploring computers at a very young age. At the age of 14 I became the world’s youngest Microsoft Certified professional to achieve MCSD, MCSE, MCDBA. I was fortunate to receive written compliments and motivation from Bill Gates, Azim Premji, Dr. Raghunath Mashalkar, Dr.Irwin Jacobs and many others for this achievement. At the age of 16 I started off my company TechnoKarma Labs with a vision to build innovative indian products for the global market. Our first product was 802.11b based mesh router. This router was a low cost and more efficient alternative to the way wireless networks were being deployed. During this time, I was also associated with International Institute of Information Technology, Pune (I2IT) for research in High Performance Computing under mentoring of Dr.Vijay Bhatkar. After my 12th standard at Fergusson College, I was offered to join in as a full time MS student in Networks & Telecom at I2IT skipping my engineering degree. At the age of 19 I received my 1st Masters degree. As part of my masters thesis, I had worked on a low level distributed memory management system for virtual machine clusters. After my Masters, I worked for 1 year with Qualcomm at San Diego, USA. I developed 2 products in distributed mobile computing at Qualcomm’s Corp R&D which were patented. I received an offer to join University of California, San Diego (UCSD) for Ms-PhD program in Communication Theory and Systems. I decided to join UCSD for PhD. My topic of research was Ad-Hoc networks and large scale distributed systems. During my stay in San Diego, I got a chance to work with a Venture Capital firm and get a view of several start-ups that they had invested in, or were planning to invest-in. This experience was invaluable. After completing the Master’s courses and some MBA courses I decided that I would like to come back to India, and start off TouchMagix. Presently I am on leave of absence from UCSD. I have been consulting to several start-ups in India and US on technology and product development.

Q: You could have chosen to do your startup in San Diego, or in Silicon Valley. Why did you choose to come back to India?

A: I see India as a place with ample opportunities for an entrepreneur. My dream always has been to see India as a product innovation hub rather than a BPO hub. I decided to setup my company in Pune to pursue this dream. TouchMagix Media Pvt. Ltd. is the first entity of the TechnoKarma Labs. I dream to have many more product companies like this in times to come. I love to interact with like minded people who share this vision.

Reblog this post [with Zemanta]

Should you use a file-system or a database

In Depth, Technology, ,

Whether to use a file-system or a database to store the data of your application has been a contentious issue since the 80s. It was something we worried about even when I was doing my Ph.D. in Databases in the 90s. Now Jaspreet Singh, of Pune-based startup Druvaa has weighed in on this issue on Druvaa’s blog. His post is republished here with permission.

This topic has been on my plate for some time now. It’s interesting to see how databases have come a long way and have clearly out-shadowed file-systems for storing structured or unstructured information.

Technically, both of them support the basic features necessary for data access. For example both of them ensure  –

  • Data is managed to ensure its integrity and quality
  • Allow shared access by a community of users
  • Use of well defined schema for data-access
  • Support a query language

But, file-systems seriously lack some of the critical features necessary for managing data. Lets take a look at some of these feature.

Transaction support
Atomic transactions guarantee complete failure or success of an operation. This is especially needed when there is concurrent access to same data-set. This is one of the basic features provided by all databases.

But, most file-systems don’t have this features. Only the lesser known file-systems – Transactional NTFS(TxF), Sun ZFS, Veritas VxFS support this feature. Most of the popular opensource file-systems (including ext3, xfs, reiserfs) are not even POSIX compliant.

Fast Indexing
Databases allow indexing based on any attribute or data-property (i.e. SQL columns). This helps fast retrieval of data, based on the indexed attribute. This functionality is not offered by most file-systems i.e. you can’t quickly access “all files created after 2PM today”.

The desktop search tools like Google desktop or MAC spotlight offer this functionality. But for this, they have to scan and index the complete file-system and store the information in a internal relational-database.

Snapshots
Snapshot is a point-in-time copy/view of the data. Snapshots are needed for backup applications, which need consistent point-in-time copies of data.

The transactional and journaling capabilities enable most of the databases to offer snapshots without shopping access to the data. Most file-systems however, don’t provide this feature (ZFS and VxFS being only exceptions). The backup softwares have to either depend on running application or underlying storage for snapshots.

Clustering
Advanced databases like Oracle (and now MySQL) also offer clustering capabilities. The “g” in “Oracle 11g” actually stands for “grid” or clustering capability. MySQL offers shared-nothing clusters using synchronous replication. This helps the databases scale up and support larger & more-fault tolerant production environments.

File systems still don’t support this option 🙁  The only exceptions are Veritas CFS and GFS (Open Source).

Replication
Replication is commodity with databases and form the basis for disaster-recovery plans. File-systems still have to evolve to handle it.

Relational View of Data
File systems store files and other objects only as a stream of bytes, and have little or no information about the data stored in the files. Such file systems also provide only a single way of organizing the files, namely via directories and file names. The associated attributes are also limited in number e.g. – type, size, author, creation time etc. This does not help in managing related data, as disparate items do not have any relationships defined.

Databases on the other hand offer easy means to relate stored data. It also offers a flexible query language (SQL) to retrieve the data. For example, it is possible to query a database for “contacts of all persons who live in Acapulco and sent emails yesterday”, but impossible in case of a file system.

File-systems need to evolve and provide capabilities to relate different data-sets. This will help the application writers to make use of native file-system capabilities to relate data. A good effort in this direction was Microsoft WinFS.

Conclusion

The only disadvantage with using the databases as primary storage option, seems to be the additional cost associated. But, I see no reason why file-systems in future will borrow features from databases.

Disclosure

Druvaa inSync uses a proprietary file-system to store and index the backed up data. The meta-data for the file-system is stored in an embedded PostgreSQL database. The database driven model was chosen to store additional identifiers withe each block – size, hash and time. This helps the filesystem to –

  1. Divide files into variable sized blocks
  2. Data deduplication – Store single copy of duplicate blocks
  3. Temporal File-system – Store time information with each block. This enables faster time-based restores.
Related articles by Zemanta
Reblog this post [with Zemanta]

An overview of Computer Aided Design (CAD)

In Depth, Overviews, , , , , ,

Pune is a hotbed of activity for CAD Software – both, for users as well as developers. We asked Yogesh Kulkarni, who has more than a decade of experience in this industry to team up with Amit Paranjape to give PuneTech readers an overview of this area.

What is CAD?

Nonlinear statistics analysis of a 3D structure subjected to plastic deformations. Image by Joël Cugnoni courtesy the Wikimedia Project
Nonlinear statistics analysis of a 3D structure subjected to plastic deformations. Image by Joël Cugnoni courtesy the Wikimedia Project

CAD is defined as the use of computer technology to aid in the design of a part, a sub-assembly, or an entire product. Design can include Technical Drawings with Symbol based Representations, Visualization, 3D Rendering, and Simulation. Note, the term ‘Product’ could range from a small Widget, to an iPOD, too a large Building. Components of CAD technologies have also found widespread use in somewhat unrelated fields such Animation & Gaming.

Consider a World-War II era vintage B-17 Flying Fortress bomber; probably the only bomber ever to manufactured on an assembly line. How was it designed? Each and every part was painstakingly drawn on a drafting board. The various components and sub-assemblies were represented through various engineering drawing conventions. Yet the true visualization of how all these complex pieces fit and worked together, was left to that of the top engineers’ minds. And what about the complex 3-D shapes such as the wings? How were they designed and tested? Actual wooden models had to be created for this to visualize their shapes and test out their air-flow characteristics in wind tunnels. You can think of an army of literally hundreds of Draftsmen working on various pieces of this complex machine. Cars were designed the same way. ‘Machine Designing’ had elements of ‘Art’ in it. This style of designing was with us until recently. It’s only in the past 2-3 decades (even more recent in many SMEs in India) that computers have started replacing these ubiquitous ‘A1’ sized drawing boards that ruled the designers shop for so many decades.

Fast forward to today, and now let’s look at how Boeing’s latest 787 Dreamliner is being designed. This truly 21st century aircraft is built with composites instead of the traditional aluminum structures, and a whole bunch of other interesting innovations. All put together, Boeing claims to improve fuel efficiency by over 20% compared to other modern day commercial airplanes. All the designs of the Dreamliner are done using CAD. From the smallest widget to the entire airframe, the drawing, designing, assembling, and visualization is done on computer monitors. These designs are also evaluated for their validity and performance via Computer Aided Engineering (CAE). CAE works in conjunction with CAD to simulate and analyze various mechanical and other aspects of the design. Similarly, Computer Aided Manufacturing (CAM) works closely with CAD to help manufacture the complex parts on Computer Numerically Controlled (CNC) machines.

History

CAD has evolved a great deal over the past few decades along with the rise in computing and graphics power. Earlier CAD solutions were simple 2-Dimensional solutions for drawing machines and structures. These still represented a big step forward over drawing boards in terms of ability to save, edit and reuse drawings. Initial 3-Dimensional solutions were based on ‘wireframe models’ and ‘surface modeling’. Loosely speaking, these represented the outer edges and the external surfaces of a solid object in mathematical terms. Real 3-D capability involves representing the real object as a solid model. Mathematically, this involves a series of complex equations and data points. This had to wait for computing power to catch-up. Only in the late 1980s did this power become available to a wider engineering community via desktop workstations.

At a high level, you can think of a CAD package to have 2 important pieces: 1) The backend mathematical engine and 2) The front-end graphical rendering service.

Earlier CAD programs were primarily written in FORTRAN. Present day, CAD packages are typically developed in C or C++. Rendering was not a strong point of the earlier solutions. However over the past 2 decades, life-like rendering and simulation (rotation, motion, etc.) have become a reality. This capability has also taken this technology into the Animation & Gaming fields.

Associated Areas

CAD works closely with other allied areas such as CAE (Computer Aided Engineering), CAM (Computer Aided Manufacturing), as well as PLM (Product Lifecycle Management). In fact, CAD/CAM or CAD/CAE are often used together to describe the entire workflow. In this section we will take a brief look at these allied areas. In future, PuneTech will feature more detailed and specific articles about each of these areas.

Computer Aided Engineering is the use of computer technology to support engineering tasks such as analysis, simulation and optimization. These tasks are often performed by the engineer in close synchronization of the actual CAD activities. An example of ‘Analysis’ could be leveraging mathematical techniques such as ‘FEA/FEM’ (Finite-Element-Analysis/Finite-Element-Method) for designing a safe Bridge. ‘Simulation’ can be used to study how a mechanical assembly with various moving parts work together, on a computer screen, before actually building it. ‘Optimization’ can build on top of Analysis and Simulation to come up with the ‘most optimal’ design that meets the designer’s requirements. ‘Most Optimal’ could mean least weight, smallest number of parts, least friction, highest reliability, etc. depending on the designer’s primary objective.

Computer Aided Manufacturing is the use of computer technology to manufacture complex parts on automated machine tools. These machine tools are commonly referred to as ‘CNC’ or ‘Computer Numerically Controlled’ machines. Here’s a simple example. Let’s say an engineer has created a complex 3-D shape consisting of various contours for a new car’s exterior. The exterior parts are made by die-stamping in huge presses. The ‘dies’ are essentially molds made of hard metal. Principally, they are similar to a clay mold that is used to create various artifacts out of Plaster-of-Paris. These metal dies themselves have to be created by machining a ‘die-block’ to create a solid mirror image of the final part. This complex 3-D shape needs a sophisticated machine tool that can machine (cut/drill/shape) metal across multiple (3 or more) dimensions.

Controlling the motion of these machine tools is similar to controlling a robotic arm. CAM packages convert the solid designs in CAD packages into a set of coordinates and path instructions, along with desired speeds & acceleration/deceleration profiles for the machine tools, and communicate these instructions to the CNC machines.

PLM or ‘Product Lifecycle Management’ is not directly related to CAD like CAE or CAM. Instead, PLM as the name suggests focuses on managing the entire lifecycle of designing activity across multiple groups and departments in a company. A complete design is not limited to the machine designer. Various other players come into the picture. These include Purchasing Managers who have to source design components and sub-assemblies from vendors; Cost Accountants who want to keep a tab on the overall material and manufacturing costs of a design, Compliance Experts who want to review the design for various agency compliance requirements, etc. Similarly there are requirements to maintain the design as it goes through various versions/upgrades through its life-cycle. PLM enables collaboration across different departments on the key aspects of the design. PLM also enables collaboration between designers in terms of sharing parts data, etc.

Major Players

AutoCAD® by Autodesk is one of the most popular CAD packages out there. It focuses more on 2-D drawings such as part drawings, architect plans, electronic circuit designs, etc.

Packages like Catia® by Dassault, NX® by Siemens-Unigraphics, Pro/E® by Parametric Corporation are popular 3-D Solid Modeling Solutions. These solutions find wide use in Automotive, Aerospace and Other Manufacturing Industry Segments.

CAD in Pune

Due to the strong industrial and manufacturing base, Pune not only contains some of the biggest users of CAD/CAM software, but it also hosts some of the biggest developers of CAD/CAM software in the world.

Leaving the sobriquets such as ‘Detroit of East’ aside, it is safe to say that Pune is indeed the primary automotive hub of India. Pioneering Indian automotive companies such as Tata Motors, Bajaj Auto, Bharat Forge and Kirloskar Oil Engines are headquartered here. Along with these, a number of top multi-nationals such as Mercedes-Benz, General Motors and Volkswagen are also based here.

These big auto-majors, along with other industrial powerhouses such as Cummins Diesel have created a strong industrial manufacturing ecosystem in Pune. These OEM (Original Equipment Manufacturers) in turn drive requirements for sub-assemblies and components to Tier-1 and Tier-2 vendors.

A large number of small and med-sized industrial automation companies have also sprung up in Pune. These companies design and develop various factory automation and material handling solutions for automotive as well as other industries.

Designing activity is important at all levels, in all these companies – big or small. As a result, Pune has become probably the biggest user of various 2-D and 3-D CAD applications and other associated CAE/CAM applications, in India.

However, the ecosystem for CAD doesn’t stop here! Given Pune’s dominance in Information Technology and the huge CAD users’ base, many CAD/CAM/CAE software companies worldwide have found Pune to be the natural choice for their R&D and Service Centers. All CAD majors described in the earlier section have some development presence in Pune. Pune also has software service companies focusing on this area, such as Geometric Systems.

About the Authors

Yogesh Kulkarni has more than a decade’s experience with CAD Software Development (PTC, SDRC, UGS and now Autodesk). He is based in Pune and can be contacted at yogeshkulkarni@yahoo.com. More details are available at: http://www.linkedin.com/in/yogeshkulkarni

Amit Paranjape is one of the driving forces behind PuneTech. He has been in the supply chain management area for over 12 years, most of it with i2 in Dallas, USA. For more details, see his PuneTech profile.

Related articles by Zemanta
Reblog this post [with Zemanta]

Enrich your website (with content & money) – An interview with Hover.in

In Depth, Interview, , , ,

hover_logoHover.in is a Pune startup that provides a service for web publishers (i.e. website/blog owners) to automatically insert extra content into the webpages, in the form of a bubble that appears when the mouse is hovered over underlined words. The bubble can be informational (like a map appearing wherever a name of a place appears, or a background information about a company appearing wherever a name of a company appears), or it can be contextual, in-text, advertisement from hover’s network of partners, and most importantly it is fully under the publisher’s control. While services like this have been around in other forms, hover.in believes that its ability to handle any language, and the focus on Indian market sets it apart from the competition. See the PuneTech profile of hover.in to get a better idea of what hover.in provides.

Hover.in was one of the startups chosen to be showcased at proto.in’s Jan ’08 edition. Earlier this, week, they announced that they have received seed funding from Media2Win, and will soon be seen in action on some large Indian portals.

PuneTech interviewed Arun Prabhudesai, CEO of Hover.in, (he also runs popular Indian Business blog trak.in) to get a deeper look at hover.in. To be true to the “tech” part of PuneTech, we also asked some technical questions that were answered by Bhasker V. Kode (Bosky), CTO of Hover.

Q: Congratulations on getting funded – especially under these economic conditions. How do you plan on using this funding – what will be the focus areas?

The seed funding was finalized few months back before the whole “recession” thing started constantly ringing in our ears.

Actually, from hover.in perspective we feel this funding as more of strategic investment where Media2Win – being a leading digital media agency – will help us to go to the market. We have immensely benefitted from the experience Me2W brings on table.

The funding is being mostly used to ramp up our technical resources and infrastructure.

Q: Your main “customers” are website publishers. Are you targeting any specific geography, like India (as the .in domain name would suggest)?

Hover.in in-text platform is global and open for web publishers and bloggers from all geographies. However, we are actively targeting Indian market first. India currently does not have any in-text platform and that’s puts us in a great position to capture this market. Infact, hover.in is world’s first in-text platform that is also language agnostic, which opens up a large chunk of regional language websites.

Q: I keep hearing that “there isn’t enough money to be made from online advertisements alone in India, except for a few specific verticals.” And you seem to be going squarely after this market. What is your take on this issue?

You know, this people have started talking about because there are too many ad networks that have come up in last couple of years…more than 15 odd I can count on my fingers !

But if you look at the larger picture, online advertisements are the only ones that are growing year on year. Traditional advertising is hardest hit…

For us the advantage is, we DO NOT compete with traditional ad networks as they are 99% display advertising. We are in-text and this market has not even tapped. From publisher perspective, it is an additional channel for content and monetization.

From Advertisers, this is the most targeted way of displaying their advertisement. Also, as we follow CPA / CPC kind of model, advertisers have full ROI on investment.

Co-founders of Hover.in - Bhasker V. Kode, CTO (left) and Arun Prabhudesai, CEO
Co-founders of Hover.in - Bhasker V. Kode, CTO (left) and Arun Prabhudesai, CEO

Q: If I remember right, you are using Erlang for your development – a very non-standard choice. Can you share the reasons behind the choice? Isn’t it difficult to get Erlang developers? In retrospect are you happy with this decision?

(by Bosky)

Erlang has been used to build fault-tolerant and distributed applications for quite some time in areas like telecom, especially for allowing highly granular choices in networking. Off-late projects like ejabberd, mnesia, yaws and tsung have shown how products spanning several hundred nodes can be implemented with the erlang stack and in particular – web technologies.

It most definitely is a paradigm shift courtesy of it’s functional programming concepts, and we are glad we took that decision because of its inherent focus on distributed systems, and although the erlang developer community in India is non-existent, with the right attitude towards learning now a day’s it does’nt matter. Moreover it only took a couple of months for our developers to get used to the semantics, following which as with any stack – it’s about what you do with that expertise.

Erlang gives you that power, but at the same time – there are areas where it might not seem a right fit and perhaps look to perl or ruby for tasks that suit them. For example, we use python wherever it seems required as well. The good part is erlang open-source community has quite a closely-knit presence online, which does help quite a lot. We ourselves are now looking at contributing and opening up internal projects.

Q: One of the important challenges for hover.in will be scalability. How are you gearing up to handle that?

(By Bosky)

Right from day one, erlang based systems like ours are designed built for horizontal scaling – which allows plug-n-play addition to our growing cluster. Regardless of the development stack you work on – some things need to be built in early and that’s something we spend a whole lot of time during our first year fine tuning.

Especially for us – where practically every page hit – for every one of our users – reflects a page visit to us where we need to compute and render hovers in a matter of milliseconds. To this end – before starting out application-logic, we first built out our own distributed priority-queuing systems, our own distributed crawler and various indexing mechanisms, a time-splicing based cpu allocation for various tasks, which made things like adding jobs, executing them a more controlled operation regardless of what the actual job is and has been handling burst mode quite well.

Moreover, we can also add workers on-the-fly to almost all major tasks much like an Amazon ec2 instance where each work itself is supervised for crash recovery thanks to erlang’s open telecom platform libraries and guidelines. Caching is something else we have and continue to work on consistently. No matter how many papers, algorithms or implementations there are out there – every system needs to fine tune their own unique set of optimizations vs compromises that reflect their infrastructure, traffic, memory & space needs,etc ..

Having granular control of this is something that is a real challenge as well as a pleasure with the stack (Linux, Yaws, Mnesia, Erlang). We ‘ve also been quick to adopt cloud-computing initiatives like Amazon s3, and more recently cloudfront for our static content delivery needs.

We’re also working on a parallel map-reduce implementation, exploring options with xmpp, and better logging for our developers to find and fix glitches or bottlenecks, eventually translating to a faster and better user experience for our users.

Q: You moved to Pune specifically to start hover.in. What made you choose Pune?

Yes, I did move to Pune to start hover.in, however, it would not be fair to say that is the only reason why I moved here. I have lived most of my formative years here in Pune, before going to USA. And as you know, once a Puneite, always a Puneite!

Actually we had to choose from 2 cities – Chennai (Our Co-founder, Bhasker VK, is from Chennai) and Pune. Few important aspects tilted the balance in favour of latter. Better weather, Pune’s proximity to Mumbai where majority of big publishers, investors and advertisers have their offices. To add to it all Pune has great startup & tech community.

Q: In the journey so far, have you made any significant mistakes that you’d like to share with others, so they can learn from your experience?

Absolutely… Mistakes are important aspect of learning process and especially for first generation entrepreneurs like Bosky and Me. I think “attention to detail” is one of the most important aspects that an entrepreneur should watch for. You need to have clear in-depth blueprint in your mind about the direction your startup is going to take, otherwise it’s very easy to fall off the cliff!

Optimizing, especially during these tough times – be it resources, infrastructure or even your time. Optimize everything. Startups can’t afford any leaks.

The third thing and the one which I don’t see very often. Partner with other startups; see if there are any synergies between you. In most cases it is a win-win situation for both of them

Q: Are you partnering with other startups? At this stage, would it be possible for you to share info about any of these partnerships?

Yes, we are…one example would be Alabot (another Pune startup -ed.). Where we have got their NLP application (Travel bot) inside our hoverlet. So for any travel related publishers, it becomes a boon. So a win-win situation for both of us.

Another example would be – Before we got our own office, 2 other startups were kind enough to accommodate us for few weeks – These kind of partnerships in any way possible go a long way !

Q: What would your advice for struggling Pune entrepreneurs be?

Entrepreneurship is a roller coaster ride … It ain’t easy, but the thrills along the way make it all more than worth it!

Just jump into the rough waters and by the time you reach the other side, you will be glad you did it….

Reblog this post [with Zemanta]

Understanding Data De-duplication

In Depth, Technology, , ,

Druvaa is a Pune-based startup that sells fast, efficient, and cheap backup (Update: see the comments section for Druvaa’s comments on my use of the word “cheap” here – apparently they sell even in cases where their product is priced above the competing offerings) software for enterprises and SMEs. It makes heavy use of data de-duplication technology to deliver on the promise of speed and low-bandwidth consumption. In this article, reproduced with permission from their blog, they explain what exactly data de-duplication is and how it works.

Definition of Data De-duplication

Data deduplication or Single Instancing essentially refers to the elimination of redundant data. In the deduplication process, duplicate data is deleted, leaving only one copy (single instance) of the data to be stored. However, indexing of all data is still retained should that data ever be required.

Example
A typical email system might contain 100 instances of the same 1 MB file attachment. If the email platform is backed up or archived, all 100 instances are saved, requiring 100 MB storage space. With data deduplication, only one instance of the attachment is actually stored; each subsequent instance is just referenced back to the one saved copy reducing storage and bandwidth demand to only 1 MB.

Technological Classification

The practical benefits of this technology depend upon various factors like –

  1. Point of Application – Source Vs Target
  2. Time of Application – Inline vs Post-Process
  3. Granularity – File vs Sub-File level
  4. Algorithm – Fixed size blocks Vs Variable length data segments

A simple relation between these factors can be explained using the diagram below –

Deduplication Technological Classification

Target Vs Source based Deduplication

Target based deduplication acts on the target data storage media. In this case the client is unmodified and not aware of any deduplication. The deduplication engine can embedded in the hardware array, which can be used as NAS/SAN device with deduplication capabilities. Alternatively it can also be offered as an independent software or hardware appliance which acts as intermediary between backup server and storage arrays. In both cases it improves only the storage utilization.

Target Vs Source Deduplication

On the contrary Source based deduplication acts on the data at the source before it’s moved. A deduplication aware backup agent is installed on the client which backs up only unique data. The result is improved bandwidth and storage utilization. But, this imposes additional computational load on the backup client.

Inline Vs Post-process Deduplication

In target based deduplication, the deduplication engine can either process data for duplicates in real time (i.e. as and when its send to target) or after its been stored in the target storage.

The former is called inline deduplication. The obvious advantages are –

  1. Increase in overall efficiency as data is only passed and processed once
  2. The processed data is instantaneously available for post storage processes like recovery and replication reducing the RPO and RTO window.

the disadvantages are –

  1. Decrease in write throughput
  2. Extent of deduplication is less – Only fixed-length block deduplication approach can be use

The inline deduplication only processed incoming raw blocks and does not have any knowledge of the files or file-structure. This forces it to use the fixed-length block approach (discussed in details later).

Inline Vs Post Process Deduplication

The post-process deduplication asynchronously acts on the stored data. And has an exact opposite effect on advantages and disadvantages of the inline deduplication listed above.

File vs Sub-file Level Deduplication

The duplicate removal algorithm can be applied on full file or sub-file levels. Full file level duplicates can be easily eliminated by calculating single checksum of the complete file data and comparing it against existing checksums of already backed up files. It’s simple and fast, but the extent of deduplication is very less, as it does not address the problem of duplicate content found inside different files or data-sets (e.g. emails).

The sub-file level deduplication technique breaks the file into smaller fixed or variable size blocks, and then uses standard hash based algorithm to find similar blocks.

Fixed-Length Blocks v/s Variable-Length Data Segments

Fixed-length block approach, as the name suggests, divides the files into fixed size length blocks and uses simple checksum (MD5/SHA etc.) based approach to find duplicates. Although it’s possible to look for repeated blocks, the approach provides very limited effectiveness. The reason is that the primary opportunity for data reduction is in finding duplicate blocks in two transmitted datasets that are made up mostly – but not completely – of the same data segments.

Data Sets and Block Allignment

For example, similar data blocks may be present at different offsets in two different datasets. In other words the block boundary of similar data may be different. This is very common when some bytes are inserted in a file, and when the changed file processes again and divides into fixed-length blocks, all blocks appear to have changed.

Therefore, two datasets with a small amount of difference are likely to have very few identical fixed length blocks.

Variable-Length Data Segment technology divides the data stream into variable length data segments using a methodology that can find the same block boundaries in different locations and contexts. This allows the boundaries to “float” within the data stream so that changes in one part of the dataset have little or no impact on the boundaries in other locations of the dataset.

ROI Benefits

Each organization has a capacity to generate data. The extent of savings depends upon – but not directly proportional to – the number of applications or end users generating data. Overall the deduplication savings depend upon following parameters –

  1. No. of applications or end users generating data
  2. Total data
  3. Daily change in data
  4. Type of data (emails/ documents/ media etc.)
  5. Backup policy (weekly-full – daily-incremental or daily-full)
  6. Retention period (90 days, 1 year etc.)
  7. Deduplication technology in place

The actual benefits of deduplication are realized once the same dataset is processed multiple times over a span of time for weekly/daily backups. This is especially true for variable length data segment technology which has a much better capability for dealing with arbitrary byte insertions.

Numbers
While some vendors claim 1:300 ratios of bandwidth/storage saving. Our customer statistics show that, the results are between 1:4 to 1:50 for source based deduplication.

Related articles by Zemanta
Reblog this post [with Zemanta]

Inside a Retail Supply Chain: How did ‘Maha’ DeshMart Survive the Economic Slowdown and Thrive

In Depth, Overviews, , , ,
Wal-Mart Hermosillo
Image via Wikipedia

‘Maha’ DeshMart is a large (fictional) supermarket chain with a pan-India presence. Amit Paranjape, our resident expert on Supply Chain Management, discusses some of their management strategies and best practices. Special emphasis is put on the importance of ‘Information Technology’ and how it enables ‘Maha’ DeshMart to run one of the most efficient Supply Chain & Operations. Benchmarking is also done with global industry leaders such as Wal-Mart. 2008 represented a challenging year and we will take a look at how specific Supply Chain Processes and the other aspects of Retail Operations react to the global economic challenges, and can still deliver on the overall goals and objectives of the company. This fictional story about the fictional ‘Maha’ DeshMart is in continuation to our series of articles on Supply Chain Management.

‘Maha’ DeshMart

‘Maha’ DeshMart as the name suggests stands for everything ‘large’ (‘Maha’ in Hindi / Marathi / Sanskrit = ‘Large’ or Huge’). Some say ‘Maha’ also stands for ‘Maharashtra’ the home state of the founder, Raj Deshpande. ‘Desh’ comes from the founder’s last name. This chain is also often referred to as simply ‘DeshMart’. Ever since opening its first store in Pune about 15 years back, it has gone through a rapid expansion and now had a presence in every major Tier 1 and Tier 2 city in India, with aggressive plans to expand to Tier 3 cities as well. DeshMart’s vision is to be the most preferred shopping destination for the Indian consumer; period. To achieve this, they want to have the widest choices, the best prices, and the most comfortable shopping experience for their consumers.

It is no secret that the DeshMart founder was inspired by world leader in retail, Wal-Mart, and its philosophy of scale and constantly driving down costs. The Wal-Mart business model is actively pursued here in their Pune headquarters, as well as in all their stores and throughout their supply chain. The management team though has taken a series of strategic decisions to ‘Indianize’ the model to suit the local context. For example, while ‘ELDP’ (Every Day Low Prices, as opposed to Promotions) was the generally preferred strategy, some key exceptions were made based on local preferences. The other one is to enshrine a ‘Neighborhood Kirana Store’ owner type mentality into the Store Manager and his team. In India, the small neighborhood ‘Kirana’ (or grocery) store is run by a family with the patriarch serving as the CEO, Head of Operations, and all other roles combined. The rest of family fills up various other support roles. One thing this model provides is an ‘amazing’ ownership of the business and the consumer. DeshMart wanted its Store Managers to think and act, like the Kirana Store Owner. Metrics have been suitably adjusted to encourage the right behavior; however the core difference has been through intense focus on Hiring & Recruitment. Extraordinary importance is put on finding the right talent for these critical positions.

The Importance of Information Technology

Another key Wal-Mart strategy that has been espoused by DeshMart is focus on Information Technology. At DeshMart, IT is considered to be one of their biggest strategic differentiators. They don’t want to rely on any one or even a few different application software vendors for their business process applications. Instead, they have followed the example of Wal-Mart, Dell, FedEx, and Toyota and have taken complete ownership of their IT Applications & Infrastructure. These great companies have supported their unique business processes by designing and developing their own IT solutions where necessary. Here again, DeshMart put lot of emphasis on hiring the right CIO (Chief Information Officer). In fact in the early 1990s, when this position was virtually unknown in India and in many other places around the world – they had a CIO and his senior team in place.

The IT Department’s mission is to deliver the requisite data and decision making capabilities at the disposal of every DeshMart employee throughout the organization, in order to deliver on the overall goals & objectives of the company. Organizationally, IT was aligned along with the business process teams in such a way that for every project there was no ‘IT vs. Business’ division. The combined team had the singular goal to achieve the necessary process improvement metric.

The 2008 Global Economic Slowdown

The 2008 Global Economic Slowdown was not predicted by even the top experts on Wall Street. Thus, even the best of the supply chain leaders didn’t have any ability to accurately forecast the impending shortfall in demand. The only way for a company to react to something like this was through some rapid adjustments to their plans, and execute as efficiently as possible. The first signs of the slowdown were visible too late for planning the 2008 season. In the following sections, we will look at how DeshMart’s IT and Business Processes reacted to this challenge.

Merchandizing & Assortment Planning

Let us take a step back here and understand how large multi-product category retailers worldwide do their long term planning. For many retailers, especially for those heavily weighted towards fashion segment, the holiday season (last 5-6 weeks of the year, in the US – from Thanksgiving weekend to Christmas/New Year) could account for anywhere from 20% to even 50% of their annual revenues. In India, the holiday season traditionally runs from Diwali (The Festival of Lights), in October/November to the December End wedding season. Similar holiday season spikes are also observed in the Indian market.

To react to this end of the year demand, retailers start planning a year in advance. The overall merchandizing decisions (deciding what products to buy and position in the stores) are the first step. This process is called as ‘Merchandize Planning’. A ‘top down view’ is often a starting point, where a revenue forecast is broken down across different product groups (commonly referred to as ‘categories’) and then finally to the individual product (e.g. a pair of pants). Similarly, a geographic top down view is taken where the global revenue number is broken down by regions and eventually down to the store level. A ‘bottom up’ view can be taken based on actual product level forecasts. Often times, ‘middle-out’ view is also done at one of the intermediate points in the hierarchies. All these views are synchronized closely before arriving at final numbers and targets. This process sounds straight forward, but is complicated by the sheer size of the retail problem. Imagine doing these aggregations for 100,000 products! (A typical Wal-Mart stocks well over 200,000 products). Now add the store dimension (Major US chains have 1000s of stores), and you will get the idea of the scale problem. As a result, merchandize planning drives some challenging scalability requirements for databases and servers.

Table 1: Retail SCM & Operations – Business Processes & IT Drivers
Business Process Focus Key IT Drivers
Merchandise Planning Selecting the right products to purchase for the season/year, based on corporate objectives and setting sales targets Scalability Multi-level aggregation/dis-aggregation
Assortment Planning Grouping stores based on similar characteristics. Selecting what store groups have what products Optimization Algorithms for identifying the right store groups (clusters).
Transportation Planning & Logistics Delivering the product to the stores while minimizing transportation cost. Transaction Management for handling large volume of shipment orders
Optimization Algorithms for truck load building and route planning
Replenishment & Inventory Management Getting the right amount of the product to the store, while minimizing excesses and reducing out of stocks situations Scalability
‘Fast’ algorithms for calculating replenishments and inventory targets
Store Operations Presenting the products appropriately
Collecting POS data
Managing customer loyalty		 Scalability POS Data Collection Systems
Business Intelligence & Data Mining
Algorithms for ‘Planograms’

In case of DeshMart, the problem is not as massive as Wal-Mart, but still quite big. At present, DeshMart has over 500 stores with anywhere from 10,000 to 50,000 products in a given store. Initially, at the beginning of the year, they had done the merchandize planning assuming a specific forecast value for the end of the year period. Now, they need to alter it, and propagate it down the hierarchy. This adjusts the forecast for individual products. This will in turn affect the purchasing decisions. In some cases (as it turned in DeshMart’s case…) it is too late to do this, since long lead time items are already on order. In such cases, various promotions/ discounting strategies are worked out to push the excess merchandize out. Note that given DeshMart’s unique customer loyalty and cost advantages, a down market can also be an opportunity. For example, Wal-Mart actually had more sales compared to previous quarters in the 2008 slowdown since the cost conscious consumer from other competing stores increasingly moved their shopping to Wal-Mart. Hence while adjusting the merchandize, DeshMart also considered this aspect as an input.

Once the product level forecast is available, the next challenge in a multi-store retail environment is deciding what stores carry what products. Not all DeshMart stores are the same. They differ by types of cities, as well as location of stores within cities. The typical product mix of a store in Central Deccan Gymkhana area of Pune is different from that in the Hadapsar suburb. A Mumbai store would differ in its product selection than say, a store in Indore. These product selections are referred to as ‘Assortments’ and planning them is called as ‘Assortment Planning’. Here too, scale is a big issue. Various algorithms are used to group stores based on their commonalities into groups or ‘clusters’. Then assortment decisions are made at the cluster level.

When the economic down turn hit, not only did the merchandizing decisions change, but also certain assortments by stores. To give an example stores in up market areas were now stocked with certain products normally allocated to more middle income areas. DeshMart was able to make these changes quickly by redoing certain clustering and changing allocations.

In both merchandizing and assortment planning IT plays a key role in providing the planners with a key capability to make fast and accurate decisions, while dealing with a huge amount of data. Handling large volumes of data, large scale aggregations and dis-aggregations, scenario planning &  what-ifs are also important IT requirements for these business processes.

Logistics & Transportation

If ‘Supply Chain Management‘ is loosely defined as ‘Getting the right product at the right place, at the right time’; then ‘Logistics & Transportation’ is one of the core execution pillars of realizing this. Logistics deals with the overall handling, distribution and shipment of products through the supply chain, while transportation is focused more on the physical shipment of goods and products. In a sense transportation can be considered as a subset of logistics, but often times these two terms are used interchangeably.

For large global retailers, the transportation problem typically begins at their warehouses. Product manufacturers are responsible for shipping material into these warehouses. (Note there are some exceptions such as Vendor Managed Inventory & Direct Store Shipments, but we will not discuss those here…).

The primary goal of efficient logistics & transportation planning is to get the product to the store at the right time, while minimizing cost. Warehouse management – ensuring smooth and efficient inflow and outflow of products is the first step. One relatively new technology that is being used in some places is ‘RFID’ (Radio Frequency ID). These are small tags (like a semiconductor chip) that are attached to stocking pallets, cases or other products that need tracking. A radio frequency based reader then ‘reads’ from this tag. This helps in easy tracking, sorting and distribution of products in a warehouse, while minimizing manual intervention. Some of these warehouses span many acres and have 1000s of pallets, miles of conveyor belt and typically handle over 100,000 individual products. Hence automated tracking is very important. RFID systems also need efficient large scale data acquisition and storage systems to handle this high volume data.

Truck load planning and truck routing are two important pieces of transportation planning. The challenge in truck load planning is filling up the truck to a full truck load with multiple products, while minimizing excessive shipments to the store (Note – It is always easy to ship a full truck load, if ‘stuffing’ it with unnecessary excess stuff is allowed…). The challenge comes in when this restriction of not shipping too much excess products comes in. Remember, excess product takes up excess space, as well as locks-up excess capital. This is bad for the overall financial performance. Various optimization based approaches that trade-off between cost of shipping early, vs. sending partial truck loads (thus trading off between excess inventory carrying costs vs. excess transportation costs…) are used to figure out the right loading mix inside a truck. In case full truck loads are not possible, then smaller shipments are loaded in, to be delivered to different destinations (typically different stores). This is called as ‘LTL Shipments’ (LTL = Less Than Truck Load). Here the challenge is to come up with a efficient truck route so as to minimize the distance traveled.

Hence if the DeshMart warehouse in Vashi, New Mumbai is shipping LTL shipments to the suburban stores in Bandra, Andheri and Mulund, the optimized route could consist of Vashi-Mulund-Andheri-Bandra. This seems relatively straight forward, but there are a lot of other constraints in the real world. There are some loading constraints, precedence constraints, traffic constraints, regulatory constraints that can influence the most optimal route on a map. Also, for a large retailer like Wal-Mart this process has to be done for 1000s of trucks every day. The IT systems have to deal with managing these whole sets of transactions as well as the decision making processes. They need efficient data integration with the IT systems of external logistics providers as well as suppliers.

In DeshMart’s case, there are also a series of local constraints that are peculiar to the Indian market. Shipping times in India can be fairly non-standard as compared to that in the US. Most truck operators are not organized into big companies. Regulatory requirements like Octroi (a local city tax) can add delays to the process. Enforcement of driving standards/time can also be a problem. Hence similar to Wal-Mart, DeshMart had made a strategic decision to own a private fleet of trucks for their transportation needs. This enables them with greater control over the whole process.

As we saw earlier, the impact of the economic slowdown led to last minute adjustments in assortments. Such changes, as well as those coming from sudden surges in demand can be better handled with a private fleet. DeshMart’s trucks also use the latest GPS based navigation and logging technology that enable the logistics master controller to keep exact track of each shipment in real time.

IT enabled dynamic logistics & transportation business processes helped DeshMart to better respond to the challenges of the ‘real’, ‘unplanned’ world, while keeping transportation costs at minimum.

Store Level Replenishment & Inventory Management

Store Level Replenishment & Inventory Management deals with calculating the right levels of products that are maintained at the individual stores. Too few and you run the risk of running out of stock, often resulting in lost sales and unhappy customers. Too many, and you take up excess space as well as occupy excess working capital.

Specialized replenishment and inventory management IT systems can react to daily fluctuations in the demand signals and pre-calculated forecasts at the store level and identify the right quantity of each product that needs to be shipped to the store. A variety of algorithms are used to do this, and like all other retail problems – scale is a big challenge. Imagine planning 100,000 products across 1,000 stores. The number of products-stores combination goes into millions! Now consider that this planning has to be done for the forecasted demand for the time horizon of next 2 weeks. Each day represents a new demand entry. Thus this further increases the problem size by an order of 14!

Daily demand fluctuations are usually computed based on ‘POS’ (Point-Of-Sale’) data. This data is often directly generated at the systems at the point where the sale takes place (e.g. cash register). POS Systems collect validate and transfer this data to the Replenishment System.

Through efficient store replenishment, DeshMart can make sure that they have the right product at the store, while keeping their costs low. The same cannot be said about their competitors! In fact, walk into any retail store today in India, and chances are you would find many of your preferred items to be out of stock. This goes a long way in generating the customer loyalty that DeshMart has been able to create over the years. What many of the new retail chains don’t seem to realize today (and what some ‘Neighborhood Kirana Stores’ do very well…) is that it is not the fancy layouts, air-conditioning, jazzy promotional material, etc. that attracts the customers! It is being repeatable and consistent and always having the fast moving goods in stock at all times – this is what the customer ultimately cares about!

Store Operations

Store Operations for DeshMart represents a great challenge and a terrific opportunity. Store operations in India can be quite different from that seen in the US or other developed countries. To start with, the store footprints are much smaller. The assortments are also smaller, but there is a lot of localization. For example, a large percentage of the assortment of a store in Chennai will differ from that in New Delhi. Managing store layouts and displays and locating the right stuff in the front, at the end aisles is very important. DeshMart uses CAD like software capabilities to do ‘Planograms’ (Planograms refer to the designing of products placement on the various shelves, while accounting for display priorities as well as the products’ volume dimensions). A fairly unique service in India is ‘home delivery’ service, provided by many grocery retailers. Ordering is either done over the phone or in person. A store level order entry and tracking system captures the order and coordinates the home delivery. DeshMart will be the first retailer in India to launch a completely web-based & call-center based ordering system, starting early next year. Here the order will be accepted and promised centrally and delivered from a central warehouse or a nearby store. This ‘hybrid’ web and brick and mortar model will be fairly unique not only in India but globally as well.

Customer loyalty is key for DeshMart. They have implemented a sophisticated customer loyalty program. A unique card/id number is assigned to a customer and rewards are given based on amount of purchase, as well as other special criteria. DeshMart collects, analyzes and mines customer buying preferences in their centralized business intelligence system and comes up with pricing and product placement strategies. Customer specific targeted emails and other specials are also managed through this system. For example, DeshMart’s data mining system can literally predict what special brand hair oil Mrs. Shah from Ahmadabad is likely to buy and at what frequency – and automatically send specials (or alerts) for the same!

All these entire store-centric systems ensured that even when the consumer spending all over India was going down, DeshMart still had their loyal customer continue to find and spend on the right product mix at their stores.

Conclusions and lessons learnt

In retrospect, the 2008 economic slowdown turned out to be a blessing in disguise for DeshMart. Through their superior business processes, management teams, and IT systems – they were able to not only react effectively to the changing market dynamics; they were also able to grab even higher market share. This same slowdown saw big troubles for many of DeshMart’s competitors, some of which like ‘Bharat Bears Mart’ actually went out of business.

Raj Deshpande reflects on this ‘interesting year’ and ponders how these 3 basic principles go a long way for his business success:

  • Focus on the customer – learn from the neighborhood Kirana Store.
  • Focus on costs at all costs – learn from the world leader, Wal-Mart.
  • Focus on leveraging IT for business – learn from industry leaders.
Related articles by Zemanta
Reblog this post [with Zemanta]

Peepaal – Bringing Industry and Academics Together

In Depth, Overviews, , , ,

improving engineering education in PuneThis is the third in PuneTech’s Improving Education series of posts detailing the efforts being taken by people in the tech industry in Pune to give our students a chance at giving their career a solid foundation by acquiring skills that are valued in the industry with guidance from experienced mentors from the industry. The first in the series was Techstart.in, a program to nurture the passions of students by guiding them to implement specific, and interesting projects. The second was KQInfoTech’s Post Graduate Diploma in Systems Programming where the students don’t actually pay for the course, but end up being funded by industry in return for completing projects.

The third is “Peepaal – Continuous Community Learning” is an online and offline community of industry professionals, college faculty, and students that aims at providing continuous guidance to students through knowledge, best practices and processes. This is a description of Peepaal in the words of one of the founders:

1. Introduction – Bringing Industry and Academics Together

Bill Gates (Windows) , Steve Jobs(Mac, iPod, iPhone), Larry Page & Sergey Brin (Google), Orkut Büyükkökten (Orkut), Mark Zuckerburg (Facebook) , Sabeer Bhatia (Hotmail)  apart from being successful have one important fact in common. They were all young when they dreamed of transforming the world into a better place to live. The truth is the pioneers in the technology field have been youngsters who have time and again brought new path breaking perspectives and solutions.

After 25 years in the IT field India has become the outsourcing hub of the world albeit with some deficiencies. On the pioneering front we are still some steps behind. Why in spite having such a large pool of vastly talented students are we still lagging behind? This is a million dollar question. But the answer to the question is easy.

Our students have always learned in a vacuum far away from the reality. Firstly they only have a very distant view of the needs of the industry and secondly they have passive view on the problems faced by the people and society. Good industrial skills are needed to become successful professionals of the IT industry.  Identifying practical problems and solving them will make the leaders and pioneers of tomorrow.

Peepaal aims to become that easily accessible Bridge between the students and the industry. Peepaal aims to become that great ladder on which the pioneers and leaders of tomorrow will climb and make a difference to the society we live in. Peepaal aims to become that Guru who will always show the right path to the students. Peepaal aims to become that Family who will nurture the students to become as independent as possible. Peepaal aims to become that Friend who will bring students with similar thoughts and interest together.

2. Do our talented students need improvement? Can it be done?

The Peepaal initiative was born out of our various interactions with students, faculty and our experience with freshers entering the industry. Each of these experiences reemphasized the significant need for Industry to provide a very strong collaborative role in shaping students and supporting faculty in delivering industry relevant knowledge to students to ensure they are more well rounded professionals by the time they enter the industry. It was also clear that this is not possible without continuous support from the industry and a collaborative model for delivering knowledge, best practices and processes.

While there are a number of areas that require focus we have identified some core focus areas for Peepaal and use projects as a mechanism for being able to deliver these improvements.

Why Projects? Well Projects have been something the students have been actively working on during their course curriculum and their is a significant focus on projects among students and faculty and we think it is a great mechanism to deliver knowledge. The issue however is that the current process of executing projects leaves a lot to be desired.

We expect to significantly alter the way student projects are executed and help them in the following:

  • Innovate: Projects can be a key medium to innovate and learn new ideas and principles and apply them in real world. We expect Peepaal guided projects to be able to help students to Innovate in the real world and follow cutting edge technology trends. In addition we believe the core CS fundamental concepts are key to ensuring that students deliver value from their project. Concepts like Data Structures, DBMS, operating Systems for example have to be understand and applied well on projects so that students learn by doing.
  • Execute: If a project does not succeed, the value of the effort is reduced to naught and it turns into an academic exercise. There are various industry proven methods like “Proof of Concept” implementation which helps support this. We expect the Peepaal Barcamps and the online portal to serve as checkpoints and support mechanism to significantly increase the number of successful projects.
  • Showcase: While an Idea looks great on paper and even after it is successfully executed it all boils down to how well you can showcase it. This applies to student projects as well. There is a considerable lack of awareness of the need and techniques to successfully showcase a project to an audience. Through a continuous stream of Barcamps and presentations we expect to significantly raise the bar on project presentations.

3. Objectives of Peepaal

How does Peepaal plan to solve the problem areas? Following are the high level steps Peepaal is undertaking at a brisk pace:

  • Establish strong collaboration between students and the industry to bring the students into reality.
  • Establish strong collaboration between students in various campuses.
  • Engage the faculty in healthy discussions and suggest solutions to the problems they face to enable successful projects.
  • Help students in the entire life cycle of their projects.
  • Provide technical and industrial knowledge,best practices to the students in an easy to understand format.
  • Compliment the students academic education with the learning and applications of the latest trends and technologies.
  • Provide guidance to the students in every manner so that they do not have to reinvent the wheel like the students of the past.
  • Encourage innovative ideas from the students that will have profound effect on the society we live in.
  • Encourage discussions so that students open up to share problems and seek solutions.

4. Peepaal Online Portal

Peepaal online portal http://www.peepaal.org/campus provides the much needed platform where the students, faculty and the industry come together. Following are the objectives it aims to meet:

  • Establish a strong communication mechanism between the students and the industry. Students, Faculty post questions on the discussion forums and the industry experts answer them in a simple easy to understand format.
  • Provide Students with articles that give them insight on the latest trends and technologies.
  • Prevent Students from re-inventing the wheel by suggesting them right tools,techniques and designs.
  • Provide Students with a forum to improve their interpersonal skills.
  • Provide industry experts a simple way to interact with students instead of visiting their colleges.
  • Provide industry experts to talk to a greater audience and use this greater audience in useful ways.
  • Community features that will get students with like thoughts together.

5. Peepaal Barcamps

Peepaal Barcamps are an innovative process model to guide the students in the entire life cycle of their project. Peepaal aims to be as close to students as possible in this regard. Peepaal engages face to face with the students during the course of their project through various Project BarCamps and campus events.

We have already conducted a number of BarCamps in the first season where we addresses close to 1000 students across various colleges across Pune. Starting January we will conduct an additional 18 Barcamps across 10 colleges in Pune where expect to work with close to 1500 students on their projects.

Some of the colleges that we are currently work with include Symbiosis, Nowrosjee Wadia, PICT, MIT, VIT, Fergusson, Modern, SP College.

6. Peepaal Linked-in community

Peepaal has setup a linked in community. This is the place where the industry experts, students and faculty will get together for discussions and debates. These discussions will give Peepaal great insights into the initiative and further improving the initiative.

http://www.linkedin.com/e/gis/1454357/1D32262ADD49

7. Get Involved !!!

All of us are aware that our educational system needs a lot of support if we want to sow the seeds for transformation in society for the future. The source for innovation and execution lie in our future students and it is all of us as experienced professionals who are best suited to provide this support. All it would take is a few hours of your time online or offline to make this impact. You can join us in our discussions online, on the portal, in the linked in community or join us for the Barcamps.

The number of Industry professionals have grown rapidly since we have started and we have close to 50 Industry professionals registered for the initiative and over 800 students impacted by the program. We do however need a lot more industry participation to ensure we have significantly broader impact across a larger number of students. If you are an Industry Professional, Faculty or a student and if all of this makes sense to you and you would like to join us on this initiative drop us a line at peepaalcampus@gmail.com and we would be happy to meet up with you to get started.

What to expect – Industry Professionals

An industry professional who wants to get involved with Peepaal should expect to:

  • Conduct Mock Interviews
  • Discuss and Refine Project Ideas
  • Mentor Projects Online
  • Own Technical Focus Areas
  • Drive Technology Trends
  • Support Campus Contact Programs
  • Collaborate with your campus
  • Discuss and Collaborate with Peers online

All you need is 2 hours a week

What to expact – Students

A student who gets involved in Peepaal can expect the following help:

  • Prepare for Placements
  • Identify, Plan and Execute Projects
  • Identify Technical and Soft Skill Needs
  • Showcase Competencies
  • Understand Technology Trends
  • Plan and execute Barcamps
  • Collaborate with Industry Experts
  • Discuss and Collaborate with Peers online

All you need is a few hours a week.

Related links:

Know any other Pune initiatives that belong to this series? Let us know.

Reblog this post [with Zemanta]