Tag Archives: analysis

Liveblogging CSI Pune Lecture: Applications of Business Intelligence

I am liveblogging CSI Pune‘s lecture on Applications of Business Intelligence by Narender C.V. of SAS R&D India. These are quick and dirty notes of the lecture – not intended to be a well organized article, but hopefully it gives you enough of a flavor for the area to get you interested and excited enough to check it out on google and wikipedia.

The amount of data is doubling every 11 months. And we have easier and easier access to all this data from all over the world. The problem is making sense of all this data. The amount of time at our disposal remains the same. So we have to use sophisticated software and algorithms to figure out how to use this data to improve business and efficiency. That is Business Intelligence (BI).

This talk is the second in a series of talks on BI. PuneTech covered the first talk which gave an overview of BI and data warehousing. This lecture focuses on who uses BI and why. A major portion of this talk will be a bunch of examples of use of BI in real companies. So on to the examples:

Example 1: Getting a better grip on Reality (i.e. Seeing problems earlier)

First case study will focus on using BI to simply get a good picture of the situation as it exists. Seeing Reality. Last year, US based companies paid $28 billion in servicing warranties or recalls. This is money you don’t really want to spend. Biggest problem in this is identifying these problems as early as possible. Seeing reality early. Typically, an issue first appears. A little while later, the issue becomes visible to the company, and it is prioritized. Later it is “defined” and decisions taken by the decision makers. Finally the issue is resolved, and money paid out. A study by SAS shows that the “detect” part of this cycle takes about 90 days, the prioritize part takes 20 days, and the define part takes 75 days. That’s a total of 185 days to fix the problem.

A business intelligence system helps to reduce each phase of that sequence because of better data gathering and statistical analysis. This results in 27 days detection, 5 days, prioritization and 46 days to prioritize, for a total of 78 days. This is a huge improvement, and each day saved results in money saved.

How is this done? First simple reports: defects per thousand, per product. Dashboard with easy to see defect reports. Then a library of reports that various people in the company can use easily to see and analyze defects and warranty claims. Then a statistical analysis engine to detect “emerging issues”. Use algorithms that can detect, from early trends, issues that are likely to become “big” later on. Text mining and analysis to read unstructured reports of service technicians and being able to determine, simply by looking at the keywords, which product or part or defect was the cause of that particular incident. And there are other analytics, like forecasting and trend analysis that are used. Bottomline? Shanghai GM was able to reduce detection and definition time by 70%, resulting in reduction of costs by 34%. Which is pretty cool for simply running a bunch of mathematical algorithms.

Example 2: Manage and Align Resources to Strategy

Everybody agrees that it is important for a company to have a strategy. And that everyone should understand and execute according to that strategy. Obvious?

This is a reality based on a survey: Only 5% of the workforce of a large company understand the company strategy. Only 25% of the managers were incentivized based on the strategy. 60% of organizations do not link budgets to the strategy. 86% of executive teams spend less than one hour per month discussing strategy.

How can BI help in this case?

It is possible to define objectives for each person/team in the company. Then it is possible to define how this objective can/should be measured. Then BI software can be used to capture and analyze this data, and figure out how everybody is contributing to the end objectives of the business.

Example 3: Retail Optimization

The problem to be solved. Need to stock the exact quantity that people are going to buy. Stock too much and you lose money on unsold items. Order too little and you get out-of-stock situations and lose potential profits. Need to be able to forecast demand. Optimize which sizes/assortments to stock. All of you must have an experience of going to a shop, liking an item, and not having that available in your size. Sale lost. Profit lost. Can this loss be reduced?

Use BI for this. In case study, a department store sent the same mix of different sizes to all stores. SAS did clustering of stores, to create 7 different sub-groups that have different size mixes for each sub-group of stores.

Example 4: Personalized, real-time marketing

Take the example of marketing. Consider a traditional marketing mail sent from a company. Customers hate that and the success rate is a pathetic 3% or so. That’s just stupid, but exists when there is no alternative. Better is event based marketing. When you do something, it triggers a marketing push from the company. This is often convenient for the customer, and has a 20% success rate. But the best is customer initiated interaction which has a 40% success rate.

Note that as you go down that list, it gets more difficult to quickly, in real time, determine what marketing message exactly to push to the customer. If you call a pizza delivery place and they point out that that they have a buy-one-get-one-free offer, it might or might not be interesting for you. Better would be an offer focused specifically on your needs. Use BI to analyze individual customers and forecast their needs and then tailor the offer for you. An offer you cannot refuse.

Another example. Customer puts digital camera in online shopping cart. The online shopping software contacts the BI system for offers to push to customer. It looks at customer history. Figures out that customer is non-tech savvy customer who buys high-end products. Also, customer’s demographic information is consulted, and finally some accessories are suggested. Since this is very specific recommendation, this can result in a high chance of being accepted. This significantly increases profit on this transaction.

Example 5: Understanding Customers

Mobile company, simplistic view: Customer is leaving. Offer them a lower value plan. The might or might not leave. BI gives you better tools. Cost is not the only thing to play with. Understand why people are leaving, and also understand the effect of them leaving on your business. (Sometimes it might be best to let them leave.) And based on this, determine the best course of action – what / how much to offer them.

First, use predictive analysis to get an estimate of how much profit you are going to make from a customer over the course of next N years based on the data you have gathered about them so far. Use this figure, the “customer value”, to drive decisions on how much effort to expend on trying to get this customer to stay. Forget the low value customers, and focus on the high value ones!

Another possibility. If you have marketing money to spend on giving offers to some customers. Let us say there are 3 different kinds of offers. Use BI analysis to figure out which offers to send to which customers, based on customer value, and also chances of customer accepting that offer. This optimizes the use of the “offer” dollars.

Reblog this post [with Zemanta]

CSI Lecture: Applications of Business Intelligence – 16th Oct

Computer Society of India – Pune Chapter presents the second lecture in a series on Data warehousing. The first lecture gave an overview of BI and DW. The second lecture will describe how these techniques are used by businesses:

What: Applications of of Business Intelligence  by Narendar C.V. of SAS R&D India.

When: Thursday, October 16th, 2008, 6:30pm to 8:30pm
Where: Dewang Mehta Auditorium, Persistent Systems, Senapati Bapat Road
Entry: Free for CSI Members, Rs. 100 for others. Register here.

Details – Overview of BI & Data warehousing

This lecture will cover the various applications of Business Intelligence solutions. These include Customer Intelligence, solutions specific to Industries and also will touch upon real time BI applications. Narender will explore the value and use of advanced Business Intelligence, areas such as Performance Management, Customer Management and Analytics: forecasting, data mining and Optimization. He’ll present examples of advanced business Intelligence methods and uses, and suggest ways companies can implement and incorporate these types of analysis. He will also discuss ways to measure the success and ROI. 

If you’ve always wanted to know why, how and when you should be using advanced BI, you won’t want to miss this!

It is not necessary to have attended the previous lecture.

For more information about other lectures in this series, and in general other tech events in Pune, see the tech events calendar at upcoming.

About the speaker – Narender C.V.

Narender is a Principal Consultant at SAS. He currently spearheads the Solution development for the Retail & Manufacturing Solution.

Use Google Insights to find a niche market for your (non-web) product

Image representing Google Labs as depicted in ...Image via CrunchBase, source unknown

(In this interesting article, Trevas of Druvaa uses keyword search trending data from Google Insights and Google Labs Experimental Search to fine-tune his idea of what exactly is the market niche into which his products are most likely to have a demand.

While search term analysis is a very common technique used by web-based companies for search engine optimization and finding long-tail customers, what is surprising in this case is that the products Trevas wants to sell have nothing to do with the web. He is using the keyword analysis to simply get a feel for which needs of users seem to have been met in the last few years, and which needs seem to be increasingly unmet. That gives him ideas for potential niche markets in which to position his products. Even if you have no interest in laptop backup and disaster recovery and the other terms used in this article, you should still read the article to get a hang of the technique, which can be applied in other fields. This article first appeared at Druvaa’s blog and is reproduced with permission. For more information about Druvaa and its technology, see this in-depth punetech article.)

While doing some keyword research for Druvaa it began to become clear how interesting search engine statistic can be when you look closely at the data. From simple keyword suggestion tools, and graphs you can ascertain information that you never thought possible.

The terms “backup” or “recovery”, for instance, get over 300,000 searches per month each with Google. In other words people are searching for good solutions to keep their data safe. That information by itself is useful (at least to us), but it’s when you begin to look at more specific search terms that things really get interesting. In fact, you can even begin to clearly see trends within the industry when you compare specific terms over any given length of time.

With a look at some simple charts, you can begin to see things like:

  • Interest in laptop backup solutions has greatly increased over the past 10 years.
  • Some users are finding solutions to their data backup needs and disaster recovery isn’t as much of a problem as it was 4 years ago (but it still is a problem).
  • Enterprise users who have laptops in the office are still seeking a suitable solution to their backup needs.
  • Enterprise users who have offsite backup needs are still seeking a solution to business continuity.

To demonstrate how I can get all of that from a few search terms, let’s take a closer look at some charts.

A Look at Trends Using Search Engine Statistics

Using Google Labs and their experimental search tool you come up with the following charts for the terms “data backup” and “laptop backup”.  This particular tool uses search volumes, online news statistics, number of websites, and more to show interest in any given topic. The charts clearly show that, while data backup has retained the same amount of interest over the past 10 years, interest in laptop backup has (and is) increasing.

Of course, this idea makes sense. Laptops have decreased greatly in price since 1998, and as such have become a more common tool both for enterprise users and at home. On the other hand, data integrity has been a problem for business users for a couple of decades now, so interest in the topic of “data backup” have remained relatively the same.

This information alone isn’t necessarily new. It’s the reason we created Druvaa InSync in the first place. The industry needed a reliable data backup solution, which is also fast enough to work well with computers that are on the go.  To further look at what’s needed let’s look at some more charts. This time based on search volumes alone.

Laptop Backup as Important as Ever

Search volumes for any given term are an easy way to see what is happening within an industry, to gauge interest for a product or service, or even to see how one product relates to another. In the developed world more than 73% of the population has internet access, and over 88% of internet users go online when they seek a solution to a problem.

With that in mind let’s briefly look at some search engine statistic.  In this case I have used Google Insights to compare related search terms. The charts are based on normalized data, over time. If you looked at the actual search volumes they would have increased with time (since Internet use has grown). To get a more accurate look, Insights uses normalized data displayed on a scale of 1 – 100.

Click Here to See the Chart for Yourself

The first chart compares the terms data backup and disaster recovery. There are two things that can be gained from this chart.

  • 1. Since search volumes for both terms have declined over the past few years, it shows that some users are finding solutions to their backup needs, and disaster recovery is less of a problem today than it was in 2004/2005.
  • 2. As the lines of the chart come together, they begin to show a direct correlation to each other. Very likely this is due to the fact that proper data backup is becoming the solution to disasters in the office. It really was only a few years ago that disaster recovery often meant taking that broken hard drive to have the data extracted. In the past couple of years, enterprise users have begun to see that simple backups are a cheaper (and more reliable) solution.

Since the term data backup may also relate to home users, with the next chart I used the term “enterprise backup” and compared it to “laptop backup”. Again we can see a couple of things from this chart. Once again we see a slight decline in the search volumes for enterprise backup. This confirms the idea that some enterprise users are finding a suitable solution to their backup needs.

Click Here to See the Chart for Yourself

By adding the term laptop backup though, something else begins to become clear. The term started the chart off at 61 and finished three years later at 62. There have been slight ups and downs in search volumes, but overall they have remained relatively the same. The two terms also begin to correspond closely with each other as the chart moves through 2007 and into 2008. To me this says that these terms are also beginning to become synonymous.  In other words, although some enterprise users are finding a backup solution, those with laptops in the office aren’t.

I could repeat these same results with terms like “offsite backup” or “remote backup”.

With a simple look at search engine statistics we begin to see that enterprise users have a need for a laptop backup solution that works. With our own product, which provides 10x faster laptop backup and a 90% reduction in storage and bandwidth, there is a solution to suit.

Reblog this post [with Zemanta]

Overview of Business Intelligence and Data Warehousing

I am liveblogging the CSI Pune lecture on Business Intelligence and Data Warehousing. These are quick-n-dirty notes, so please forgive the uneven flow and typos. This page is being updated every few minutes.

There’s a large turnout – over 100 people here.

Business Intelligence is an area that covers a number of different technologies for gathering, storing, analyzing and providing access to data that will help an large company make better business decisions. Includes decision support systems (i.e. databases that run complex queries (as opposed to databases that run simple transactions)), online analytical processing (OLAP), statistical analysis, forecasting and data mining. This is a huge market, with major players like Microsoft, Cognos, IBM, SAS, Business Objects, SPSS in the fray.

What kind of decisions does this help you with? How to cut costs. Better understanding of customers (which ones are credit worthy? which one are at most risk of switching to a competitor’s product?) Better planning of flow of goods or information in the enterprise.

This is not easy because amount of data is exploding. There’s too much data. Humans can’t make sense of all of them.

To manage this kind of information you need a big storage platform and a systematic way of storing all the information and being able to analyze the data (with the aforementioned complex queries). Collect together data from different sources in the enterprise. Pull from various production servers and stick it into an offline, big, fat database. This is called a data warehouse.

The data needs to be cleaned up quite a lot before it is usable. Inconsistencies between data from different data sources. Duplicates. Mis-matches. If you are combining all the data into one big database, it needs to be consistent and without duplicates. Then you start analyzing the data. Either with a human doing various reports and queries (OLAP), or the computer automatically finding interesting patterns (data mining).

Business Intelligence is an application that sits on top of the Data Warehouse.

Lots of difficult problems to be solved.

Many different data sources: flat files, CSVs, legacy systems, transactional databases. Need to pick updates from all these sources on a regular basis. How to do this incrementally and efficiently?  How often – daily, weekly, monthly? Parallelized loading for speed. How to do this without slowing down the production system. Might have to do this during a small window at night. So now you have to ensure that the loading finishes in the given time window.

This is the first lecture of a 6-lecture series. Next lectures will be Business Applications of BI. This will give an idea of which industries benefit from BI – specific examples: e.g. banking for assessing credit risk, fraud, etc. Then Data Management for BI. Various issues in handling large volumes of data; data quality, transformation and loading. These are huge issues, and need to be handled very carefully, to ensure that the performance remains acceptable in spite of the huge volumes. Next lecture is technology trends in BI. Where is this technology going in the future. Then one lecture on role of statistical techniques in BI. You’ll need a bit of a statistical background to appreciate this lecture. And final session on careers in BI. For detailed schedule and other info of this series, see the Pune Tech events calendar, which is the most comprehensive source of tech events info for Pune.

SAS R&D India works on Business Applications of BI (5 specific verticals like banking), on Data management, on some of the solutions. A little of the analytics – forecasting. Not working on core analytics – that is only at HQ.

We are trying to get the slides used in this talk from the speaker. Hopefully in a few days. Please check back by Monday.

Reblog this post [with Zemanta]