Tag Archives: BI

Why Analytics Matter in Business Intelligence – CSI Pune Lecture – 6th March

Computer Society of India – Pune Chapter presents the 5th lecture in a series on Data warehousing. The first lecture gave an overview of BI and DW. The second lecture was about how these techniques are used by businesses. The third was about data management for business intelligence. The fourth lecture talked about technology trends in BI. This is the fifth in the series:

What: Why Analytics Matter in Business Intelligence by Ajit Ghanekar of SAS R&D India.

When: Friday March 6th, 2008, 6:30pm to 8:30pm

Where: Dewang Mehta Auditorium, Persistent Systems,402, Senapati Bapat Road, Pune
Entry: Free for CSI Members & Students, Rs. 100 for others. Rs. 50 for Persistent employees.  Register here.

Details – Technology trends in Business Intelligence

One of the areas which adds significant value to business is application of analytics to solving complex problems. These can be in the areas of scoring, risk management, fraud detection, forecasting and so on. The focus of this session will be to give an introduction to the role of statistical techniques in BI applications.

It is not necessary to have attended the previous lectures.

For more information about other tech events in Pune, see the PuneTech events calendar.

About the speaker – Ajit Ghanekar

Ajit is a Senior Software Specialist – Analytics at SAS Research & Development, India, and has 10 years of experience in developing various Analytical solutions in the areas like Statistical Inference, Modeling, Time Series in Banking and Pharma domains. Currently, he is engaged in SAS Credit Risk Management Solution.

Ajit has a Masters in Statistics from Pune University & PG Diploma in Banking and Finance from SIBM

Reblog this post [with Zemanta]

Business Intelligence Technology Trends: CSI Pune Lecture – 30 Jan

Computer Society of India – Pune Chapter presents the 4th lecture in a series on Data warehousing. The first lecture gave an overview of BI and DW. The second lecture was about how these techniques are used by businesses. The third was about data management for business intelligence. This is the fourth in the series:

What: Technology trends in Business Intelligence by Prasad Kulkarni of SAS R&D India.
When: Friday January 30th, 2008, 6:30pm to 8:30pm
Where: Damle Hall, Damle Path, Behind Indsearch, Off Law College Road
Registration and Fees: Free for CSI Members & Students, Rs. 100 for others. Register here.

Details – Technology trends in Business Intelligence

This lecture will cover technological advances in BI domain. It will start with a discussion on general trends in BI and will relate them to technology. Primary focus is on different technologies used currently, their necessity and type of problem they are solving in the business intelligence domain. It will discuss areas like SOA (Service oriented architecture), SaaS (Software as a service), MDM (Master data management), Real time warehousing, Click stream data warehouses, Federated/integrated search, Web 2.0, Data visualization and so on. The participant will know how such technologies are solving problems specific to BI domain.

It is not necessary to have attended the previous lecture.

For more information about other lectures in this series, and in general other tech events in Pune, see the PuneTech events calendar.

About the speaker – Prasad Kulkarni

Prasad Kulkarni is working with SAS Research And Development India Pvt. Ltd for past 8 years as Associate Director – Platform Research and Development. He leads the core technology group at SAS R&D Pune. Prasad holds post graduation degree in computer management from University Of Pune and has 12 years of experience in the field of information technology. He has worked with product development setups in India. With SAS his focus areas are Metadata Management, Data Warehousing, Data visualization and Data access.

Related articles by Zemanta

Reblog this post [with Zemanta]

Data management and data quality in business intelligence

I am liveblogging CSI Pune‘s lecture on Data Management and Data Quality in Business Intelligence, by Ashwin Deokar of SAS R&D Pune.

Huge amounts of data being generated these days. Different technologies (from databases, to RFID tags and GPS units), different platforms (PCs, servers, cellphones), different vendors. And all this data is often duplicated and inconsistent. All of this data needs to be collected in one place, and cleaned up?

Why? Three reasons:

  • Competitive business environment: With better, and more granular data, you can increase your profits, and reduce costs. For example, Walmart forcing RFID tags on all items that are supplied to them by suppliers – and tracking their locations for very accurate and up-to-date inventory control
  • Regulatory and Compliance requirements: e.g. US government has seriously strict data gathering and storage requirements for hospitals (HIPAA). If you can’t generate this data, you go to jail. That certainly reduces your ability to increase profits.
  • Adherence to Industry standards: If you can’t produce and consume data in the format that everybody else understands, you can’t play with the big boys

The key areas of study in this area are:

  • Data governance: Policies that govern the use of data in an organization. Done usually from the point of view of increasing data security (prevent hackers from getting in, prevent data from leaking out inadvertently), ensuring compliance with regulations, and optimal use of data for organizational growth.
  • Data architecture and design: Overall architecture – data storage, ETL process design, BI architecture, etc.
  • Database management: Since there are huge quantities of data, making a mistake here will pretty much doom the whole project to failure through overload. Which database? Optimizing the performance. Backup, recovery, integrity management, etc.
  • Data security: Who should have access? Which data needs to be kept private?
  • Data quality: Lots of work needed to ensure that there is a single version of the truth in the data warehouse. Especially difficult for non-transactional data (i.e. data that is not there in a database). e.g. Ashwin Deokar is the same as A.P. Deokar. Need fancy software that will do these transformations on the data.
  • Data Warehousing and Business Intelligence: What this component does is covered in a previous PuneTech article.

Data Quality. Why this is an important problem:

  • 96000 IRS tax refund cheques did not get delivered because of incorrect addresses.
  • An acquiring company, which acquired another company mainly for the customer base found that the acquisition was vastly overvalued – because the got 50% fewer customers than expected. Due to duplicates in the database.
  • A cable company lost $500,000 because a mislabeled shipment resulted in a cable being laid at a wrong location.
  • A man defrauded a CD company by taking their “introductory” offer (of free CDs) over 1600 times, by registering that many different accounts with different address. Since he did not really have that many different addresses, he managed to fool their computers by making slightly different address using minor changes like extra punctuation marks, fictitious apartment numbers, slightly different spellings, etc. Total damage: $250,000.

There is a process, combination of automated algorithms, and human assistance to help with improving data quality. And it is not just about duplicate data, or incorrect data. You also need to worry about missing data. And fetching it from the appropriate “other” sources.

What do you do?

  • Clean up your data by standardizing it using rules – have canonical spellings for names, addresses, e etc.
  • Use fancy algorithms to detect duplicates which are obvious by just looking at the strings. For example, “IBM” and “International Business Machines” do not look similar. But if they have the same address, same number of employees, etc., then you can say they are the same. (And you can have thresholds that adjust the sensitivity of this matching process.)
  • Use external data to clean up the data for de-duplication. For example, US Postal service publishes a CD of every valid address in the US. Businesses buy that CD and use that to convert all their address data to this standard format. That will result in major de-duplication.

SAS provides tools for all the steps in this process. And since it has all the pieces, it has the advantage of ensuring that there is a single meta-data repository for all the steps in this process – which is a huge advantage. SAS has the best ETL tools. It also exists in analytics, and BI. It has OLAP capabilities, but it really excels in business intelligence applications.

SAS R&D Pune has engineers working on various core products that are used in this area – meta-data, ETL, BI components. It also has a consulting group that helps companies deploy SAS products and use them – and that ends up working on all the parts of the data management / data quality process.

CSI Lecture: Data Management for Business Intelligence

Computer Society of India – Pune Chapter presents the second lecture in a series on Data warehousing. The first lecture gave an overview of BI and DW. The second lecture was about how these techniques are used by businesses. This is the third in the series:

What: Data Management for Business Intelligence by Ashwin Deokar of SAS R&D India.

When: Wednesday, November 19th, 2008, 6:30pm to 8:30pm
Where: Dewang Mehta Auditorium, Persistent Systems, Senapati Bapat Road
Entry: Free for CSI Members, Rs. 100 for others. Register here.

Details – Data Management for Business Intelligence

This lecture will cover the various issues in Data Management of Business Intelligence solutions: Why is Data management and data quality important, What is Data management, Components of Data management, Factors affecting Data management, Key Challenges in Data management, Data Quality, Data Quality process

It is not necessary to have attended the previous lecture.

For more information about other lectures in this series, and in general other tech events in Pune, see the PuneTech events calendar.

About the speaker – Ashwin Deokar

Ashwin is working as a business unit head with SAS R&D Pune. Heading the OnDemand Solution group. Ashwin has over 10 year of experience in ERP, DW, BI & Analytics across multiple domains like manufacturing, CPG, Retail, Banking & Insurance. He has been with SAS for 6 years under various roles like Project Manager, Senior Consultant, Business Unit head.

Liveblogging CSI Pune Lecture: Applications of Business Intelligence

I am liveblogging CSI Pune‘s lecture on Applications of Business Intelligence by Narender C.V. of SAS R&D India. These are quick and dirty notes of the lecture – not intended to be a well organized article, but hopefully it gives you enough of a flavor for the area to get you interested and excited enough to check it out on google and wikipedia.

The amount of data is doubling every 11 months. And we have easier and easier access to all this data from all over the world. The problem is making sense of all this data. The amount of time at our disposal remains the same. So we have to use sophisticated software and algorithms to figure out how to use this data to improve business and efficiency. That is Business Intelligence (BI).

This talk is the second in a series of talks on BI. PuneTech covered the first talk which gave an overview of BI and data warehousing. This lecture focuses on who uses BI and why. A major portion of this talk will be a bunch of examples of use of BI in real companies. So on to the examples:

Example 1: Getting a better grip on Reality (i.e. Seeing problems earlier)

First case study will focus on using BI to simply get a good picture of the situation as it exists. Seeing Reality. Last year, US based companies paid $28 billion in servicing warranties or recalls. This is money you don’t really want to spend. Biggest problem in this is identifying these problems as early as possible. Seeing reality early. Typically, an issue first appears. A little while later, the issue becomes visible to the company, and it is prioritized. Later it is “defined” and decisions taken by the decision makers. Finally the issue is resolved, and money paid out. A study by SAS shows that the “detect” part of this cycle takes about 90 days, the prioritize part takes 20 days, and the define part takes 75 days. That’s a total of 185 days to fix the problem.

A business intelligence system helps to reduce each phase of that sequence because of better data gathering and statistical analysis. This results in 27 days detection, 5 days, prioritization and 46 days to prioritize, for a total of 78 days. This is a huge improvement, and each day saved results in money saved.

How is this done? First simple reports: defects per thousand, per product. Dashboard with easy to see defect reports. Then a library of reports that various people in the company can use easily to see and analyze defects and warranty claims. Then a statistical analysis engine to detect “emerging issues”. Use algorithms that can detect, from early trends, issues that are likely to become “big” later on. Text mining and analysis to read unstructured reports of service technicians and being able to determine, simply by looking at the keywords, which product or part or defect was the cause of that particular incident. And there are other analytics, like forecasting and trend analysis that are used. Bottomline? Shanghai GM was able to reduce detection and definition time by 70%, resulting in reduction of costs by 34%. Which is pretty cool for simply running a bunch of mathematical algorithms.

Example 2: Manage and Align Resources to Strategy

Everybody agrees that it is important for a company to have a strategy. And that everyone should understand and execute according to that strategy. Obvious?

This is a reality based on a survey: Only 5% of the workforce of a large company understand the company strategy. Only 25% of the managers were incentivized based on the strategy. 60% of organizations do not link budgets to the strategy. 86% of executive teams spend less than one hour per month discussing strategy.

How can BI help in this case?

It is possible to define objectives for each person/team in the company. Then it is possible to define how this objective can/should be measured. Then BI software can be used to capture and analyze this data, and figure out how everybody is contributing to the end objectives of the business.

Example 3: Retail Optimization

The problem to be solved. Need to stock the exact quantity that people are going to buy. Stock too much and you lose money on unsold items. Order too little and you get out-of-stock situations and lose potential profits. Need to be able to forecast demand. Optimize which sizes/assortments to stock. All of you must have an experience of going to a shop, liking an item, and not having that available in your size. Sale lost. Profit lost. Can this loss be reduced?

Use BI for this. In case study, a department store sent the same mix of different sizes to all stores. SAS did clustering of stores, to create 7 different sub-groups that have different size mixes for each sub-group of stores.

Example 4: Personalized, real-time marketing

Take the example of marketing. Consider a traditional marketing mail sent from a company. Customers hate that and the success rate is a pathetic 3% or so. That’s just stupid, but exists when there is no alternative. Better is event based marketing. When you do something, it triggers a marketing push from the company. This is often convenient for the customer, and has a 20% success rate. But the best is customer initiated interaction which has a 40% success rate.

Note that as you go down that list, it gets more difficult to quickly, in real time, determine what marketing message exactly to push to the customer. If you call a pizza delivery place and they point out that that they have a buy-one-get-one-free offer, it might or might not be interesting for you. Better would be an offer focused specifically on your needs. Use BI to analyze individual customers and forecast their needs and then tailor the offer for you. An offer you cannot refuse.

Another example. Customer puts digital camera in online shopping cart. The online shopping software contacts the BI system for offers to push to customer. It looks at customer history. Figures out that customer is non-tech savvy customer who buys high-end products. Also, customer’s demographic information is consulted, and finally some accessories are suggested. Since this is very specific recommendation, this can result in a high chance of being accepted. This significantly increases profit on this transaction.

Example 5: Understanding Customers

Mobile company, simplistic view: Customer is leaving. Offer them a lower value plan. The might or might not leave. BI gives you better tools. Cost is not the only thing to play with. Understand why people are leaving, and also understand the effect of them leaving on your business. (Sometimes it might be best to let them leave.) And based on this, determine the best course of action – what / how much to offer them.

First, use predictive analysis to get an estimate of how much profit you are going to make from a customer over the course of next N years based on the data you have gathered about them so far. Use this figure, the “customer value”, to drive decisions on how much effort to expend on trying to get this customer to stay. Forget the low value customers, and focus on the high value ones!

Another possibility. If you have marketing money to spend on giving offers to some customers. Let us say there are 3 different kinds of offers. Use BI analysis to figure out which offers to send to which customers, based on customer value, and also chances of customer accepting that offer. This optimizes the use of the “offer” dollars.

Reblog this post [with Zemanta]

CSI Lecture: Applications of Business Intelligence – 16th Oct

Computer Society of India – Pune Chapter presents the second lecture in a series on Data warehousing. The first lecture gave an overview of BI and DW. The second lecture will describe how these techniques are used by businesses:

What: Applications of of Business Intelligence  by Narendar C.V. of SAS R&D India.

When: Thursday, October 16th, 2008, 6:30pm to 8:30pm
Where: Dewang Mehta Auditorium, Persistent Systems, Senapati Bapat Road
Entry: Free for CSI Members, Rs. 100 for others. Register here.

Details – Overview of BI & Data warehousing

This lecture will cover the various applications of Business Intelligence solutions. These include Customer Intelligence, solutions specific to Industries and also will touch upon real time BI applications. Narender will explore the value and use of advanced Business Intelligence, areas such as Performance Management, Customer Management and Analytics: forecasting, data mining and Optimization. He’ll present examples of advanced business Intelligence methods and uses, and suggest ways companies can implement and incorporate these types of analysis. He will also discuss ways to measure the success and ROI. 

If you’ve always wanted to know why, how and when you should be using advanced BI, you won’t want to miss this!

It is not necessary to have attended the previous lecture.

For more information about other lectures in this series, and in general other tech events in Pune, see the tech events calendar at upcoming.

About the speaker – Narender C.V.

Narender is a Principal Consultant at SAS. He currently spearheads the Solution development for the Retail & Manufacturing Solution.

Overview of Business Intelligence and Data Warehousing

I am liveblogging the CSI Pune lecture on Business Intelligence and Data Warehousing. These are quick-n-dirty notes, so please forgive the uneven flow and typos. This page is being updated every few minutes.

There’s a large turnout – over 100 people here.

Business Intelligence is an area that covers a number of different technologies for gathering, storing, analyzing and providing access to data that will help an large company make better business decisions. Includes decision support systems (i.e. databases that run complex queries (as opposed to databases that run simple transactions)), online analytical processing (OLAP), statistical analysis, forecasting and data mining. This is a huge market, with major players like Microsoft, Cognos, IBM, SAS, Business Objects, SPSS in the fray.

What kind of decisions does this help you with? How to cut costs. Better understanding of customers (which ones are credit worthy? which one are at most risk of switching to a competitor’s product?) Better planning of flow of goods or information in the enterprise.

This is not easy because amount of data is exploding. There’s too much data. Humans can’t make sense of all of them.

To manage this kind of information you need a big storage platform and a systematic way of storing all the information and being able to analyze the data (with the aforementioned complex queries). Collect together data from different sources in the enterprise. Pull from various production servers and stick it into an offline, big, fat database. This is called a data warehouse.

The data needs to be cleaned up quite a lot before it is usable. Inconsistencies between data from different data sources. Duplicates. Mis-matches. If you are combining all the data into one big database, it needs to be consistent and without duplicates. Then you start analyzing the data. Either with a human doing various reports and queries (OLAP), or the computer automatically finding interesting patterns (data mining).

Business Intelligence is an application that sits on top of the Data Warehouse.

Lots of difficult problems to be solved.

Many different data sources: flat files, CSVs, legacy systems, transactional databases. Need to pick updates from all these sources on a regular basis. How to do this incrementally and efficiently?  How often – daily, weekly, monthly? Parallelized loading for speed. How to do this without slowing down the production system. Might have to do this during a small window at night. So now you have to ensure that the loading finishes in the given time window.

This is the first lecture of a 6-lecture series. Next lectures will be Business Applications of BI. This will give an idea of which industries benefit from BI – specific examples: e.g. banking for assessing credit risk, fraud, etc. Then Data Management for BI. Various issues in handling large volumes of data; data quality, transformation and loading. These are huge issues, and need to be handled very carefully, to ensure that the performance remains acceptable in spite of the huge volumes. Next lecture is technology trends in BI. Where is this technology going in the future. Then one lecture on role of statistical techniques in BI. You’ll need a bit of a statistical background to appreciate this lecture. And final session on careers in BI. For detailed schedule and other info of this series, see the Pune Tech events calendar, which is the most comprehensive source of tech events info for Pune.

SAS R&D India works on Business Applications of BI (5 specific verticals like banking), on Data management, on some of the solutions. A little of the analytics – forecasting. Not working on core analytics – that is only at HQ.

We are trying to get the slides used in this talk from the speaker. Hopefully in a few days. Please check back by Monday.

Reblog this post [with Zemanta]

CSI Pune Lecture: Overview of Business Intelligence and Data Warehousing – 27 Aug 2008

Computer Society of India – Pune Chapter presents a lecture series on Data warehousing. This is the first lecture in that series:

What: Overview of Business Intelligence & Data warehousing by Vibhas Joshi, head of R&D at SAS R&D India.

When: Wednesday, August 27th, 2008, 6:30pm to 8:30pm
Where: Dewang Mehta Auditorium, Persistent Systems, Senapati Bapat Road
Entry: Free for CSI Members, Rs. 100 for others. Register here.

Details – Overview of BI & Data warehousing

Concepts of data warehouse, data marts, OLAP and data mining, understand relationship between transactional systems and data warehouse.

About the Speaker – Vibhas Joshi

Vibhas is with SAS R&D india as Head R&D , Program Manager – Industry Intelligence solutions, Member of Management Team.
Vibhas holds a Masters degree in Physics from the University of Pune, a Diploma in Computer Management from Jamnalal Bajaj Institute as well as a Masters in Management Studies from University of Pune. He is certified PMP.

He has over 25  years of experience in the IT. He has special skills in General Management, Program Management, Project Management, Software Product Development, Requirement Engineering, Database Management, Software Development Methodologies, and Infrastructure set-up.

Vibhas has conducted numerous training programs covering Project Management, Requirement Management and Software Engineering.

Vibhas in the course of his assignments has worked in the following business domains: Banking, Financial Services, Insurance, Manufacturing, Telecom.

For more information about other lectures in this series, and in general other tech events in Pune, see the tech events calendar at upcoming.

Reblog this post [with Zemanta]