Tag Archives: measurement

CMG Pune Meet: Scalable Data Streaming; Mobile Tracking & Analytics; Benchmarking OpenStack – 12 Sept

CMG India, a professionals’ forum for measurement and quantitative analysis in IT enterprises, invites performance engineering and capacity management professionals to their fourth half-day event in Pune, on September 12th, 2014, 2:30pm, BMC Software (near Pune Golf Course). This event is free for anyone to attend. See here for details on how to register.

Agenda

Agenda

  • 2:15pm: Event kick off
  • 2:30pm: Reliable and Scalable Data Streaming in Multi-Hop Architecture, Sudhir Sangra, BMC
  • 3:15pm: Optimization for Large Scale Wi-Fi Mobile Device Tracking & Analytics System, Santosh Kangane, Persistent
  • 4:00pm: Break
  • 4:15pm: Rally – Performance benchmarking of openstack – private cloud, Deepak Mane, TCS
  • 5:00pm: Windup

Reliable and Scalable Data Streaming in Multi-Hop Architecture – Sudhir Sangra

The changes in technologies, customer requirements, and visualization and consumption of data demands has led multiple point products to integrate and evolve into a solution. The value of the solution is in the seamless flow of information through multiple product layers and consuming this information in real time. Many times, if the “time to consume” the data to build information is not within the agreed service levels, the data loses its value. Thus, timeliness and reliability are the two most important aspects of multi-layered application integration.

In the IT Management domain, the data collection and the system analytics engines need to go hand-in-hand. The data collectors should feed the analytics serves in real-time, to enable it to build intelligent analysis and to do various sorts of data modelling to help IT Managers have the holistic view of the underlying system resources and let them perform a root cause analysis of any problem proactively. The more granular, and consistent data, the more accurate will be the analytics and capacity estimates.

Sudhir Sangra is the Product Development Architect at BMC, leading data centre performance and availability solution suite.  One of the focus area in recent time is fault tolerant, Load balance, real time data management.

Optimization for Large Scale Wi-Fi Mobile Device Tracking & Analytics System – Santosh Kangane

Mobile Device Tracking System handles the movement of 70K+ mobile devices in WiFI network and present valuable insides like, Most famous paths, crowded places on flowers, patterns of device movement in Wi-Fi network, device count on different flowers and Zones.

The case study present optimization around handling heavy write operation and performing analytics on Oracle database. It demonstrate the effective use of Oracle performance monitoring tools like OEM, AWR & ADDM report. How to draw conclusions from reports and co-relate that with Oracle internal functioning to gain maximum benefit.

Rally – Performance benchmarking of openstack – private cloud – Deepak Mane

OpenStack is open-source software which is used for building public or private clouds. Competitors include VMware, vCloud and Amazon Web Services (News – Alert), so steps need to be taken to make OpenStack as popular as possible in the competitive marketplace.

OpenStack has become increasingly important to enterprises of all sizes as its reach extends beyond Web 2.0/SaaS companies, such as Workday, Webex and PayPal. OpenStack is now a major element of both enterprise cloud computing and broader cloud initiatives.

Customers also need to know how well OpenStack performs, but OpenStack is complex with many subsystems and components, which makes it hard for customers or potential customers to predict how “different implementation decisions and changes proposed to OpenStack affect the whole system’s behavior and performance,” according to a statement from Mirantis. On top of this, there are many alternative cloud configurations where OpenStack can be installed and used. Benchmarking becomes very difficult because it’s hard to get results from just parts of OpenStack. What was needed is a way to find benchmarking on the entire OpenStack system, with a predefined cloud configuration, the company explained. The benchmarking tool would also need to come up with complex and reproducible scenarios on actual OpenStack deployments.

In other words, customers basically need to monitor how well of a job OpenStack is doing. So IBM, SoftLayer (News – Alert) (part of IBM) and Mirantis have developed Rally. It’s a benchmarking tool that reports on OpenStack performance.

In this session, we describe the details behind the process, talk about the Rally – Performance benchmarking tool , performance assessment strategies we used. We’ll also share our findings on key scalability and performance bottlenecks, validations approaches, and suggest solutions.Performance benchmarking of Openstack component – nova.

About CMG Pune

Computer Measurement Group is a not-for-profit, worldwide organisation of IT professionals committed to sharing information and best practices focused on ensuring the efficiency and scalability of IT service delivery to the enterprise through measurement, quantitative analysis, and forecasting. CMG Inc, which is headquartered in New Jersey, USA was setup in 1975 and it now has more than 25 US and International Chapters.

CMG India has been recently setup with the objective of networking performance engineering and capacity management professionals across India. Large IT systems across India need to process millions of transactions per day and CMG India will allow for the experts to share their experiences and learn from one another. The facilitation will be done through regional events across major cities, an annual conference, and posting technical articles on this site.

See the CMG India website for more information.

Fees and Registration

Seating capacity is limited. Event participation is only for CMG India members – but you can sign up for a free membership here: http://www.cmgindia.org/punevent-registration/. Members can register for this event here: http://www.cmgindia.org/punevent-registration/

Event Website: http://www.cmgindia.org/events/event/4th-cmg-india-pune-event/

Please double-check the date/time/venue of the event at the above link. We try to ensure that PuneTech calendar listings are accurate, but occasional errors creep in.

Note: you are currently subscribed to the PuneTech main newsfeed. However, there are lots of technology events happening in Pune every week, and most of those are not posted on the main newsfeed. If you want to be informed by email about all the tech events happening in Pune, you need to separately subscribe to the PuneTech Calendar. It’s free so what are you waiting for? Check out the PuneTech Calendar to get an idea of the kinds of events that you’re missing out on.

CMG Pune half-day conference on Performance, Capacity, Instrumentation, Measurement

CMG India, a professionals’ forum for measurement and quantitative analysis in IT enterprises, invites performance engineering and capacity management professionals to their second half-day event in Pune, on Friday, 20th December, 2:30pm, at TCS, Nagar Road. This event is free for anyone to attend. See CMGIndia website for details on how to register.

After a competitive round of paper submissions and stringent review by a technical program committee, the following have been selected for presentation at this event:

Agenda

  • 2:30pm: Welcome Note
  • 2:40pm: A High Performing Scalable Architecture to Instrument Software Applications, Anand Kumar, TCS
  • 3:20pm: Messaging Protocols for Real Time System Data Sharing, Deepti Nagarkar, Persistent
  • 4:00pm: Break
  • 4:10pm: Guerilla Capacity Assessment using USL, Prajakta Bhatt, Infosys
  • 4:50pm: Prize Distribution for 1st Pune Event Quiz Contest & Windup

A High Performing Scalable Architecture to Instrument Software Applications

By Anand Kumar

Software applications have become larger and more complicated in design. This increased complexity and scale in software application design present difficult challenges in asserting software quality. Software instrumentation is an effective solution in measuring and improving a software application’s quality. By examining the data gathered by the various instruments, it is possible to determine the cause for the software application failure and a causal analysis of the context in which the failure happened can lead to improvement in the overall software quality. In this paper, we discuss about a generic instrumentation for software applications which can be used to assert software quality. We also discuss about architecture of this generic instrumentation. We illustrate the capabilities of this generic instrumentation using couple of case studies.

Anand is currently a Senior Scientist at TCS Innovation Labs, Pune with research interests in Architecture, Processes, Systems, and Services. He has two decades of experience in the IT industry spanning across programming, architecture, and research.

Messaging Protocols for Real Time System Data Sharing

By Deepti Nagarkar

An introduction to protocols primarily used to share data for Real time systems along with a brief comparison with each other. Realtime systems have different requirements with respect to data delivery. There are protocols explicitly designed to deliver according to these specifications. Protocols analyzed are – DDS, STOMP and AMQP.

Deepti is currently with Persistent Systems Performance Engineering Group. She has over 6.5 years experience in J2EE across a number of projects and domains. Deepti has done her B.E. Computers from Pune University.

Guerilla Capacity Assessment using USL

By Prajakta Bhatt

Traditional Capacity Assessment exercises come with their own challenges. This paper aims to cover how these challenges can be surpassed using Guerilla techniques, originally proposed by Dr. Neil Gunther. It introduces the concept of scalability and shows how Universal Scalability law (USL) can be employed to model scalability of the Systems realistically, combining both Hardware/Software aspects together. USL aids in arriving at maximum point of throughput, beyond which the system performance degrades. This paper, shows how this theoretical limit can be used to predict capacity of a live database server through a detailed case study.

Prajakta is working as a Technology Architect, in NFRs technical focus group that specializes in Performance Engineering Activities at Infosys. She has over 9.5+ years of experience in software development, project management and Performance Engineering across various domains and technologies of applications. She has done M.S. (Software Systems) from BITS, Pilani and B.E. (CSE) from Govt. College of Engineering, Aurangabad.

About CMG Pune

Computer Measurement Group is a not-for-profit, worldwide organisation of IT professionals committed to sharing information and best practices focused on ensuring the efficiency and scalability of IT service delivery to the enterprise through measurement, quantitative analysis, and forecasting. CMG Inc, which is headquartered in New Jersey, USA was setup in 1975 and it now has more than 25 US and International Chapters.

CMG India has been recently setup with the objective of networking performance engineering and capacity management professionals across India. Large IT systems across India need to process millions of transactions per day and CMG India will allow for the experts to share their experiences and learn from one another. The facilitation will be done through regional events across major cities, an annual conference, and posting technical articles on this site.

See the CMG India website for more information.

Fees and Registration

Seating capacity is limited. Event participation is only for CMG India members – but you can sign up for a free membership here. Members can register for this event by sending an email to pune-events@cmgindia.org from their registered email ids. For details about the venue contact Abhay_Pendse@persistent.co.in

Event Website: http://www.cmgindia.org/events/event/2nd-cmg-india-pune-event/

Please double-check the date/time/venue of the event at the original website/registration link. We try to ensure that PuneTech calendar listings are accurate, but occasional errors creep in.

Note: you are currently subscribed to the PuneTech main newsfeed. However, there are lots of technology events happening in Pune every week, and most of those are not posted on the main newsfeed. If you want to be informed by email about all the tech events happening in Pune, you need to separately subscribe to the PuneTech Calendar. It’s free so what are you waiting for? Check out the PuneTech Calendar to get an idea of the kinds of events that you’re missing out on.

Internet Traffic Tracking and Measurement

comScore Search Ratings, Dec. 2005-2006, Live,...
Image by dannysullivan via Flickr

(As the web upgrades to web-2.0, it becomes a difficult challenge to figure out the value of companies that are serving this market. Since most web-2.0 companies are in an early stage of their evolution, they can’t be measured on the basis of the revenues they are earning. Instead, one needs to guess at the future earnings based on measuring the thing that they’ve currently managed capture – i.e. the number and demographics of visitors, and the amount of attention they are paying to the site. Pune-based entrepreneur Vibhushan Waghmare, who has co-founded a marketing analytics startup, MQuotient, gives us an overview of this space, points out some problems, and wonders if there is an opportunity for some entrepreneur to step in and provide solutions.)

Introduction

A good product or service will always attract appreciation and success, but what will make it stand out from crowd and fetch the premium is knowledge of exactly how good it is from the rest. Qualitative strategy decisions are important to set the direction, but real numbers and insights from these numbers are required to actually know how fast/slow is one moving in that direction and how far it is from the target.

This applies to online internet businesses as well. As against the established brick-and-mortar businesses which are driven primarily by monetary profitability, evolving online businesses have been searching for the parameters to judge and measure the success or failure of the business.

A few days before the Dot Com bust (of early 2000s) happened, we had seen how internet companies’ valuation shot off the roof based on parameters like eyeballs they generated – and hypothetically – could be monetized. We had ExciteAtHome paying $780 million for BlueMountain.com, an online greeting cards company with 11 million monthly visitors and negligible revenues (which was sold to American Greetings after 2 years for just $35 million!). Back then, page-hits on the servers was what each site measured and investors bought into.

Today we are into web2.0 world, and parameters for measuring success of online business have also evolved to 2.0 version. Before the Lehman Brothers folded up their shop, we had valuations of these socionets soaring to astronomical levels – all based on the unique users they can generate. Page-hits have given way to page-views per unique user, and now we talk about more evolved and derived parameters like unique users visiting the site and time spent by each unique user on the website. With the ghosts of Dot Com bust not yet laid to rest, investors and entrepreneurs are much more cautious and are becoming scientific in tracking and measuring the internet traffic. Still every now and then, we keep getting news about socionets with their latest 2.0 apps being chased by good money because of their platform of involved users, although all that they do there is poke each other and take up a challenge of some random quiz. We all know the problems giants like Google are facing when it comes to monetizing a socionet like Orkut.com. Economists have predicted 8 of the last 5 economic meltdowns, and I don’t want to sound like one. I just want to point out to you the issues faced by online businesses today.

User Panel based traffic estimation

I was reminded of these measurement arguments when last weekend I attended an interesting talk organized by Pune OpenCoffee Club. We had owner of a reputed online gaming portal talking about the kind of traffic his games attract. He used comScore extensively to compare himself in the online gaming world and stated that getting into the top 5 of the comScore list of online gaming sites worldwide is the target he has set for himself. (I don’t know whether the list was of page views or unique users these gaming sites are generating). Definitely a great target to chase!

Image representing comScore as depicted in Cru...
Image via CrunchBase

While comScore does provide an elaborate analysis of the website traffic and is considered a standard worldwide, before we set our business targets based on it, we need to understand the methodology used for this tracking. comScore has a panel of around 2 million internet users worldwide (16,000 in India) and these users install monitoring software from comScore on their computers. This monitoring software is used to determine which websites are being visited by these users, and how much time they are spending on each site. comScore then uses extensive statistical methods to extrapolate these numbers to the behaviour of all the users (not just comScore’s user panel). More details on methodology are here). They have elaborate analysis like time spent by each user, IP tracking, repeating users, incoming and outgoing traffic and many more such details.

But what needs to be noticed is the fact that comScore excludes traffic from cyber-cafes and users under age 15. For India, I am sure that is a sizable mass of internet users. And when it comes to activities like online gaming, I am afraid, absolute numbers shown by comScore might be drastically away from the reality. Cyber cafe still remains an important point of access for Indians and excluding this traffic can result in misleading conclusions. Internet is being taught in schools and at least in cities, school kids are using internet extensively for both studies and entertainment purposes. In such situation, excluding users under age 15 might not always provide the best traffic numbers, especially for activity like online gaming.

Image representing Alexa as depicted in CrunchBase
Image via CrunchBase

The other good bet in terms of tracking online traffic is Alexa.com. Alexa again, is a panel research based on the information gathered through a browser toolbar that their panel of users download and install in their browser. However in over more than a decade that I have spent on internet, I have not seen a single browser with Alexa toolbar. Apart from the high-end users of internet, I wonder if an average internet user would actually go to www.alexa.com and download and install their toolbar.

There are some other tracking and measurement services available, but mostly it has been Alexa and comScore who are quoted for such purposes.

One can argue that both comScore and Alexa work based on a random sample and hence same error in reporting would appear across traffic measurement for all sites. Given this, Alexa and comScore can be reliably used to compare two internet destinations or to detect any deviation from normal trend. However for absolute numbers, I guess there is lot more needed to be done.

For developed countries where most of the traffic originates from home, school or offices and very less from cyber cafes, these numbers might work, but for India with its huge cyber cafe traffic, I guess a more extensive tracking system is required. Cyber cafes continue to be important point of access, often the only access point in tier II and III cities. I have seen young school kids flocking these cyber cafes which serve more as gaming parlours; parents creating matrimony profiles of their children with the help of assistant (generally the owner) at the cyber cafe; and young college students playing pranks on their friends through Orkut and also getting their first experience to mature content over internet. comScore is missing this traffic by excluding cyber cafes.

Although this traffic might not be very huge in terms of absolute numbers, general observation is that these new users of internet (who learn how to use internet in cyber cafes) are more likely to click on ads as online behaviour has not matured to differentiate an online advertisement from a genuine article. I once saw a school kid trying to fill up a life insurance form because the advertisement offered some lucky draw prize on filling the form (Of course he never completed the form for the lack of PAN number :-)). This audience would be of the least interest to all the online advertisers and brands since they hardly convert into any transaction; however these would be the guys who would most likely click on all those online advertisements and hence form important part of the online advertisement industry.

Is there an entrepreneurship opportunity here

I am sure that all hosting servers do have the exact numbers about traffic coming to them, however key is in profiling this traffic and consolidating and analysing this information into useful insights. Quite often websites who try to track and measure their traffic resort to putting javascripts on their pages for this purpose. This adds to page-weight and slows down the site, a significant problem in country like India where high-speed broadband is still a luxury. These efforts give reasonable tracking and measurement of traffic from server side alone. However to prove the worthiness of traffic generated by website, system needs to track the demographic details of this traffic. System should provide information about the age, education profile, income level and other details in which advertisers and investors would be interested. Of course proxy variables need to be used for this tracking along with all the principles of market research and with due care of privacy of the user. Also the system should be encompassing enough to take care of diversity in internet usage as we see in India and in developed western countries and also in non-English speaking countries.

Creating such a tracking and measurement system for India would need investment, and given the current level of online advertisement spends in the country it needs to be analyzed whether this investment is justified.
Do you guys see an entrepreneurship opportunity in this?

About the author – Vibhushan Waghmare

Vibhushan is a co-founder of MQuotient, a Pune-based startup that uses cutting-edge quantitative analytics and mathematical modeling to build software products for marketing analytics, and in general deliver solutions for enterprise marketing challenges. Before co-founding MQuotient, Vibhushan was managing the Search product at Yahoo! India. He is an MBA from IIM Ahmedabad and an Electrical Engineer from REC, Nagpur. He has also held positions with Amdocs & Cognizant Technology Solutions. Check out his blog, his linked-in page, or his twitter page for more about him.

Reblog this post [with Zemanta]