Tag Archives: website

Suggest ways for Pune Techies to collaborate online and win a Google Wave invitation

Update: the competition is over – Sandeep Gautam has won the Google Wave invitation for this suggestion.

The offline tech scene in Pune is thriving, as one look at the PuneTech events listing and the PuneTech calendar will show.

And there are a whole bunch of online places for techies in Pune to hang out:

Most of these are basically mailing lists, and forums. I wonder whether there are other ways in which techies in Pune can find other like-minded people, and collaborate in more ways. Would chat be interesting, like proto.in uses? Or IRC? Should we be focusing on Orkut or Facebook or both? Is there something intersting that can be done with YouTube?  Can we use some new technology in new ways to bring people closer together? Maybe Google Wave?

Give your suggestions in the comments section below. The best suggestion gets a Google Wave invitation. You can get the invitation for yourself, or you can use it to invite someone else. If you’re not interested in the invitation, please say so in your comment.

Give a specific suggestion for online collaboration/communication amongst Pune’s techies. Don’t just give the mechanism of collaboration – also give the purpose. For example, saying, “use an online chat room” is useless. Much more useful is something like “use an online chat-root where students from engineering colleges can ask questions about career to people from industry.” Also, a suggestion that is easy to implement is much more valuable than a suggestion that is going to require a lot of setup and/or effort. And, you get lots of plus points if you’re also willing to drive the effort. (And if you like somebody else’s suggestion, and would be willing to help/join that effort, please leave a comment indicating that.)

(Thanks to Amit Somani for graciously agreeing to donate one of his Google Wave invitations for this purpose.)

Reblog this post [with Zemanta]

Web Scalability and Performance – Real Life Lessons (Pune TechWeekend #3)

Last Saturday, we had TechWeekend #3 in Pune, on the theme of Website Scalability and Performance.  Mukul Kumar, co-founder, and VP of Engineering at Pubmatic, talked about the hard lessons in scalability they learnt on their way to building a web service that serves billions of ad impressions per month.

Here are the slides used by Mukul. If you cannot see the slides, click here.
Web Scalability & Performance

The talk was live-tweeted by @punetechlive and @d7ylive. Here are a few highlights from the talk:

  • Keep it simple: If you cannot explain your application to your sales staff, you probably won’t be able to scale it!
  • Use JMeter to monitor performance, to a good job of scaling your site
  • Performance testing idea: Take 15/20 Amazon EC2 servers, run JMeter with 200threads on each for 10 hours. Bang on your website! (a few days later, @d7y pointed out that using openSTA instead of JMeter can give you upto 500 threads per server even on old machines.)
  • Scaling your application: have a loosely coupled, shared nothing, stateless, distributed architecture
  • Mysql scalability tip: Be careful before using new features, or new versions. Or don’t use them at all!
  • Website scalability: think global. Some servers in California, some servers in London, etc. Similarly, think global when designing your app. Having servers across the world will drive architecture decisions. When half your data-center is 3000 miles from the other half, interesting, non-trivial problems start cropping up. Also, think carefully about horizontal scaling (lots of cheap servers) vs vertical scaling (few big fat servers)
  • memcache tip: pre-populate memcache with most common objects
  • Scalability tip: Get a hardware load balancer (if you can afford one). Amazon AWS has some load-balancers, but they don’t perform so well
  • Remember the YouTube algo for scaling:
    while(1){
    identify_and_fix_bottlenecks();
    eat_drink();
    sleep();
    notice_new_bottleneck();
    }

    there’s no alternative to this.
  • Scalability tip: You can’t be sure of your performance unless you test with real load, real env, real hardware, real software!
  • Scalability tip – keep the various replicated copies of data loosely consistent. Speeds up your updates. But, figure out which parts of your database must be consistent at all times, and which ones can have “eventual consisteny”
  • Hard lessons: keep spare servers at all times. Keep servers independent – on failure shouldn’t affect other servers
  • Hard lessons: Keep all commands in a script. You will have to run them at 2am. Then 3am. Then 7am.
  • Hard lessons: Have a well defined process for fault identification, communication and resolution (because figuring these things out at 2am, with a site that is down, is terrible.)
  • Hard lessons: Monitor your web service from 12 cities around the world!
  • Hard lesson, Be Paranoid – At any time: servers can go down, DDOS can happen, NICs can become slow or fail!

Note: a few readers of of the live-tweets asked questions from Nashik and Bombay, and got them answered by Mukul. +1 for twitter. You should also start following.

Reblog this post [with Zemanta]

TechWeekend #3: Website Performance, Scalability and Availability: Sept 5

Scalability (Source: Domas Mituzas, Wikipedia)
Click on the image to see other PuneTech articles on Scalability (Image Source: Domas Mituzas, Wikipedia)
What: TechWeekend featuring “Website Scalability and Performance” by Mukul Kumar, VP Engineering at Pubmatic, and “Website Availability and Recovering from Failures and Disasters” by Sameer Anja, Associate Director at KPMG
When: Saturday, 5th Sept, 4pm
Where: Symbiosis Institute of Computer Studies and Research, Atur Centre, Model Colony. Map.
Registration and Fees: This event is free for all to attend. Please register here.

Website Scalability and Performance – Mukul Kumar

Mukul will talk about the various aspects of what it takes to run a very high traffic website – something that he has a lot of experience with at Pubmatic, the ad optimization service for web publishers, where they serve over a billion requests per month.

Mukul Kumar (mukul.kumar [at] pubmatic [dot] com) is a Co-Founder and VP of Engineering at Pubmatic, and Mukul is responsible for PubMatic’s engineering team and resides in Pune, India. Mukul was previously the Director of Engineering at PANTA Systems, a high performance computing startup. Previous to that he joined VERITAS India as the 13th employee and helped it grow to over 2,000 individuals as Director of Engineering for the NetBackup group, Veritas’ main product. He has filed for 14 patents in systems software, storage software, and application software and proudly proclaims his love of Π and can recite it to 60 digits. Mukul is a graduate of IIT Kharagpur with a degree in electrical engineering.

Website Availability and Recovery from Disasters – Sameer Anja

While everyone looks at security and focuses on confidentiality, privacy and integrity; an oft neglected parameter is of availability. While “neglected” may be seem like a strong term, the truth is that we overlook basic data on availability and do not even implement simple to-dos which would help in remediating the situation. The session is aimed at identifying such simple remedies, look at impacts, the assessment model and put forward various scenarios and possible solutions available. The session does not focus on specific products and instead endeavours to use existing technologies used for web site development and how they can be used for ensuring availability. Some principles of disaster recovery will also be covered.

Sameer is a Senior Manager in the IT Advisory practice and is working with KPMG since January 2007 and has 12+ years of work experience in the areas of Information Security, Product design and development, system and network administration. Worked on process and technology areas of Information Security. Worked on Governance and Compliance areas like SOX, Basel II, ISO 15048, SSE -CMM, Data Privacy apart from ISO 27001, Identity Management and Business Continuity design and testing. Experience working with startups and established setups. Speaker at various conferences/ seminars within India and abroad. Trained for six sigma green belt.

Editorial Policy

Click on the logo to get all PuneTech articles about PuneTech
Click on the logo to get all PuneTech articles about PuneTech

PuneTech is a for-the-community, non-commercial site, with an intention to keep technology professionals in Pune informed of interesting technologies, events, companies, user groups, organizations, mailing lists, and people in, or related to Pune. A lot of the content that is put up on PuneTech actually comes from other sources, and is often a cut-n-paste job, if not a direct copy (with permission). Thus, a lot of the value that PuneTech provides is in the selection of the content, and the editorial opinions and biases that we introduce in the process.

This note lays out our editorial policy to help the community understand what goes up on the site, and why:

  • PuneTech does not accept any payments or favors (monetary or otherwise) from any body, for any reason. All content on PuneTech is there because we feel that it needs to be there, and is in keeping with the purpose and the spirit of the site.
  • PuneTech strives hard to be free of vested interests. With precisely that in mind, right from the first day, we’ve always made it clear that PuneTech is not monetized in any way, and it does not provide any direct revenues, or indeed any direct monetary benefit to anybody associated with PuneTech. (For example, we don’t promote our own companies, and before promoting, or featuring our friends or their companies, we try very hard to ensure that we use the same criteria for judging them as we use for everybody else. And thankfully, our friends all respect this fact.)
  • However, PuneTech is not objective. We believe in evaluating the world around us and making our opinions known. We strongly believe that our opinions have value, and that our readers value our opinions. All PuneTech posts will always reflect the opinions of the authors (at least those opinions that we feel are worth publicizing to the rest of the world).
  • PuneTech often has posts that promote a certain company, event or person. In fact, it can be argued that all the posts on PuneTech are promoting something or someone. When we write about a company, we are essentially promoting it, and implicitly saying that the company is good enough to feature on PuneTech. When we write about an upcoming event, we are promoting the event, and essentially saying that we feel this event will be good, and worth attending. When we interview a person, or otherwise write about him/her, we are promoting that person, and saying here is a person that you should know.
  • We welcome and encourage opposing points of view in the comments. We usually approve all such comments, as long as they are in keeping with the PuneTech comments policy. Occasionally, we do delete comments that violate our policy, so please read the comments policy carefully to get an idea of what is not allowed.

Note: the PuneTech wiki is an open wiki that anybody can add to, and edit. Any content that anybody puts up there will usually be left there, as long as it is relevant to Pune, and relevant to technology, and is not illegal in some way. Note, however, that we, or someone else, might edit the content to make it more useful and usable.

In contrast, all content goes up on the the PuneTech blog (i.e. content which goes up on the front page, and to all PuneTech subscribers) is controlled by the PuneTech editor (currently Navin Kabra, with help from Amit Paranjape, and also some other friends of PuneTech). These are the rough guidelines currently used in determining what is appropriate for the front page:

  • Company profiles for interesting companies. Service companies are boring – unless the service happens to be something unique. Product companies are more interesting, especially if the product is technologically interesting, or has some aspect that is innovative. Or the product or service is itself useful for the tech community Pune.
  • Interviews of interesting tech people. Guidelines here are similar to that for company profiles – the person must be working on something that is technologically interesting, or innovative, or unique in some way, or useful to the tech community in Pune.
  • Events: The event must be in Pune,  and should be related to technology. By and large, we try to focus on events that are free, or have a nominal fee. In general, we will not feature commercial events (i.e. those which have a hefty registration fee), unless we feel that the content is especially interesting, and the price especially reasonable. We usually don’t feature information about paid training programs. These days, there are so many tech events in Pune, that we have started being selective about featuring only the more interesting events on our front page – all other events that fit our criteria are posted on the PuneTech calendar.  Note: the PuneTech calendar, like the PuneTech wiki, is a by-the-community calendar. Anyone is free to add events to the calendar.
  • PuneTech does not post job requirements. We are constantly asked about unpaid or paid job postings, but we feel that having job postings on PuneTech would distract from the primary purpose of the site. Note, however: if somebody writes a long, detailed and interesting article for PuneTech about the technology domain that their company is working in, one that would be of wider general interest to the readers of PuneTech, then, at the end of the article, we allow a short pitch about the company, and that can include a description of the kind of people they’re looking to hire. We feel that is a fair trade-off.

A few things to note:

  • These editorial guidelines have evolved over time, and will continue to evolve. Thus, it is possible that some article that featured on the PuneTech front page in the page violates one of these guidelines. The most likely reason would be that that guideline did not exist a the time the article was posted.
  • It is also possible that we make mistakes and violate our own guidelines. When you feel that such a mistake has happened, please point it out to us publicly (as a comment on the post). That will give us a chance to explain our choice to you and to everyone else, or to apologize, and learn from the mistakes.
  • If you can think of additions/modifications to the guidelines, please let us know.

Introducing PuneTech Video – your feedback needed

PuneTech logoWe are now experimenting with an additional format for PuneTech updates – short video clips hosted on YouTube.

Almost since the time PuneTech started, people have been suggesting to us that we should use video. Finally, something clicked last week, and we decided to take the plunge and start it on an experimental basis.

Our first few video posts are up here (proto.in reactions) and here (Druvaa update). Admittedly the video quality is not great. The sound quality is also quite variable. The main issue is that we are basically trying to balance the quality of the video against the amount of effort required to produce each video. Right now, we are in favor of going with a minimally acceptable quality of video that we can produce with as little effort as possible. This ensures that we continue to produce videos regularly (as opposed to starting with a bang and then discontinuing it because it is too much work.)

So the question for you is this:

  • Is the current quality good enough? (Sound quality will probably improve a little bit over time as we get more experience. Picture quality is unlikely to improve.)
  • Are you able to stream/download it conveniently and watch it, or is it too painful?
  • More generally: Is this useful? (Please note: doing the video takes about 5 minutes, while writing an article with the same information will probably take us 1/2 hour. Which means that either you get all this info in video format, or you get 20% of this info as articles, and the remaining 80% is lost. So please answer yes for this question if you think you will watch at least 30% to 40% of the videos, and answer no, if you think you’ll probably not watch the videos at all.)
  • Any other suggestions are welcome

Please leave your responses as comments. Your responses will help us decide whether we should continue this, or use that time to work on some other aspect of PuneTech.

PuneTech Comment Policy

PuneTech is a for-the-community, by-the-community site, and comments by our readers play an important part of the content. However, to ensure that the discussion always stays healthy, constructive and safe, we occasionally have to delete some of the comments. This note lays out our comments policy to help the community understand what kinds of comments we delete, and why.

PuneTech comment policy – Short version

  if (the comment is not relevant to the article)
     We will delete it;
       /* take your irrelevant rambling elsewhere */

  else if (the comment is a personal attack)
     we will delete it;
       /* rude people not welcome here */

  else if (the comment has abusive language)
     we will delete it;
       /* we are trying to have a civil discussion here */

  else if (the comment exposes PuneTech to legal liability)
     we will delete it;
       /* we don't want to get sued
          that distracts from the purpose of this website
          more details below */

  else
     your comment is welcome;

In select cases, we might allow a comment in spite of violating one of the above rules, if it has other redeeming qualities. Also, if we delete a comment, and you really, really want your voice heard, we suggest a workaround that will allow the world to still see the comment.

PuneTech comment policy – Long version

Relevance

Our primary objective is to provide PuneTech readers with focused, relevant articles and discussions. Anything that distracts from this reduces the value of PuneTech for our readers. Hence, any comment that has nothing to do with the article (and trust me, we get a bunch of these), will be deleted (unless we find it very interesting in its own right). If you want a job, please post your resume on naukri.com – don’t post a comment here. If you find yourself compelled to beg for jobs on PuneTech, seriously consider changing careers.

Personal Attacks

We are trying to build a community here, not poison it. Something about the internet makes people more rude than they would be in real life. Please resist the temptation. We love a good argument, we are after all argumentative Indians. But please argue the issues. You might be surprised to discover that it can be done without attacking the character of the other person.

Abusive Language

If it is worth saying, it can be said in polite language. If you have abusive language in a boring comment, we’ll delete it. If you have abusive language in an interesting comment, we will, at our discretion, remove the offending words, or sentences. If you don’t want your comment mangled like this by us, use polite language.

Defamation

PuneTech is a non-commercial website that is run by us on a part-time basis. We make no money from PuneTech. Which means that we have neither the time, nor the money to get involved in legal issues. We cannot afford to retain lawyers to get accurate legal advice. In the absense of that, we have to make a guess based on our understanding of the law. And anything that we think exposes us to legal liability, will be deleted.

Here is our limited understanding of the law:

If something can hurt the reputation of another person or company, legally, we can publish it if and only if it is true.  This is tricky because we need to be sure of the truth before we feel safe. Just because it is on wikipedia, does not make it true. Just because Times of India published it, does not make it necessarily true. And we can be sued even if we are simply relaying info published by someone else.

Even if the damaging statements are contained in a comment made by a third-party commenter (i.e. somebody other than us) we are still obligated to remove the comment. Otherwise PuneTech can be held liable.

So it boils down to this: if we cannot verify the truth of a damaging claim in a comment, we will delete the comment.

Please note, just because it is true, does not necessarily mean that we will allow a comment. The earlier filters of relevance, rudeness, etc. still apply. If we are unsure about the “public good” of a true but damaging statement, we will delete the comment.

Other Objectionable Content

Other reasons why comments might fall afoul of the laws are: obscenity; hurting religious sentiments; promoting violence; against security of the state; or infringing of someone’s right to privacy. In most cases, these will get deleted for violating one of our earlier policies (e.g. irrelevance, personal attack, etc.) . In the rare case that the comment somehow manages to not violate any of the earlier policies, it can still get deleted for being against the law.

Workaround

When we delete one of your comments, we are not really preventing you from expressing yourself. Please feel free to go ahead and post it on your own blog. If for some reason, you are ashamed of putting your own comment on your own blog, go ahead and create a brand new blog on blogger.com just for holding this one comment. It’s easy, it’s free, and anybody can do it. Then post a link in the comments on PuneTech. If it is relevant to the post, we’ll probably allow the link to remain.

Suggestions

If you have any feedback for us, please leave a comment below, or send us an email. The comment is subject to the same policies (ha! ha!) unless we decide to change the policy based on your suggestion. In any case, we promise to read everything, even if we delete it.

Moderation

Comments on PuneTech are moderated. Which means that one of us might have to take a look and approve the comment before it appears on the site. Sometimes, it takes us a while to get around to doing this. Please be patient. Don’t post the comment multiple times. If you are unsure of whether your comment has reached our moderation queue, send us an email.

Further Reading

What I’ve learned from Hacker News by Paul Graham. A good introduction to the issues to be considered when deciding why and how to moderate comments on a site.

Free to blog but accountable you are. The Supreme Court of India weighs in on blogging and online expression. – Dhananjay Nene

Of blogs, bloggers and freedom of expression – Mutiny.in

Bloggers Legal Guide from EFF – Note this applies to US law, but still worth reading, as it does a great job of explaining the issues.

Acknowledgements

Thanks to Dhananjay Nene, Rohit Srivastwa, Amit Kumar Singh, Unmesh Mayekar, Manas Garg, Rohas Nagpal, and Debasis Nayak for discussions that helped us clarify our thinking and craft this policy. Note: this comment policy does not necessarily reflect the views of these people – it is just that they helped us while we were struggling to figure out what the comment policy should be.

Reblog this post [with Zemanta]

PuneTech’s 11 most popular articles/pages of the year

Pune TechnologyThe end of the year is time for resolutions, introspection and top-10 lists. We too got swept up in the wave, and here is our top-11 list of the most popular articles on PuneTech in the last year. PuneTech started on 6th March, 2008 and at that time, we were not sure of what directions it will take, and whether we’ll be able to sustain or grow it. It has exceeded our expectations in terms of impact and interesting new directions. But we have been a little disappointed with the amount of community-contributed content – most of the content is still put up by just one or two people – I was hoping that there would be at least 10 people contributing regularly by now. If you would like to contribute articles to PuneTech, please get in touch with us.

In any case, here are the most popular pages on PuneTech as measured by the number of people who viewed them:

  1. Building Eka, the world’s fastest privately funded supercomputer: This was one of the first articles I wrote, and has hit the #1 spot because it got picked up by slashdot. I’m fairly proud of the fact that my server survived the slashdotting without going down or even slowing down. The related page announcing this event, and the PuneTech wiki page for CRL managed to get other spots on this list due to spillover traffic, but since they are pretty much the same, I’ve not included it in the list.
  2. The PuneTech Blog main page: Not surprisingly, the main page – which essentially holds the PuneTech blog – is the second most visited page on PuneTech.  I would still prefer that people subscribe to PuneTech instead of visiting the main page – that way, there’s less chance of missing any articles.
  3. The PuneTech wiki: This page is unfortunately a distant third – I was hoping that over time this would become the “main page”, but we are still some way away from that. The problem partially is that it doesn’t get much prominence on the main page. Also, the content there needs some work. Amit Paranjape is working on a new improved categorized list of companies to spruce up the wiki, so expect more prominence for it in this year.
  4. About PuneTech: Have you read that page? If not, maybe you should look at it, to get an idea of the how any why of PuneTech.
  5. Categorized List of Companies on the wiki: While this is very far from where it should be, it is still a useful resource – and expect it to become better. Please note, you too can add your company to this list. It’s easy and free.
  6. The list of all company overview articles on the PuneTech blog: The PuneTech blog has categorized listing pages that you can browse by clicking on the “More Links” bar at the top.
  7. Badmash.tv: Pune-based Animation Studio: To be frank, I have no idea why this one is so high up on the list. In any case, animation is a hot and happening area for tech in Pune, and I am hoping that one of these days, someone will write an nice in-depth article on this area, covering the important technologies, companies and organizations in Pune in this area. Anyone up for the challenge?
  8. Vision for Pune – 2015: Specific projects – This page, and the companion article giving the broader, longer-term Vision for Pune was written by Anupam Saraph, CIO of Pune, at the insistence of PuneTech. I’m glad to report Anupam has found some collaborators for some of these projects due to the appearance of these articles and at least one pilot project is under way already.
  9. Data Leakage Prevention overview: While it might not be obvious from this list, overview articles like this one, which introduce our readers to a new technology area are some of our most popular articles – and they do form a bulk of the #11 to #20 on the most popular list. One of my favorite such articles is the overview of SCM, (which is not on the list possibly because it is “too young”). I’m hoping for more such articles this year. Look out for overviews of CAD, CAM, CAE, life sciences, animation, etc. Suggest your own areas for an overview like this. Or best, write one such article.
  10. Narendra Karmarkar’s recent research: Karmarkar revolutionized linear programming by his Interior Point Method. After this, he worked on a new architecture for supercomputing, based on concepts from projective geometry. Currently, he is based in Pune and is synthesizing these concepts with some new ideas he calls sculpturing free space (a non-linear analogue of what has has popularly been described as folding the perfect corner). This approach allows him to extend this work to the physical design of machines. This article on PuneTech and the corresponding PDF represent the first public disclosure of his new ideas.
  11. Category:Events: I had to extend this list to 11 instead of 10 because I could bear to leave out the events page. PuneTech has easily become the most comprehensive source of information about interesting technology events happening in the city. There are various different ways in which you can keep track. All tech events that we find interesting are added to the PuneTech calendar. You can subscribe to the RSS feed or email alerts. Additionally, we’ve found that people prefer to be reminded of events by SMS, hence we’ve started the (free) PuneTech SMS reminder service. And, of course, if you are aware of an event that is not listed, please let us know.

There is much more in store for 2009, so stay tuned.

Reblog this post [with Zemanta]