Dhananjay Nene, a software architect, a passionate programmer, an internet enthusiast, is one of the strong, in-depth, technical voices that graces tech events in Pune, and the online/offline tech community. In spite of an MBA from IIM-A, he has remained a techie, but he uses his dual background to good advantage – amongst managers, he becomes a techie, explaining to them complexities they don’t understand or appreciate, and amongst a group of techies he starts channeling managers, to hammer some business sense into them.
In the latest post on his blog (which you must subscribe to), he is pretending to be a manager who is resisting exhortations from his techies that the group should switch all development to Scala, the hot new language that is touted as a long-term replacement for Java. Note: these are not necessarily Dhananjay’s personal views – he is just trying to explain to techies all the issues that a good manager might worry about with respect to adoption of new technologies.
This is a must-read for all techies, and hence, is reproduced here with permission.
Why should I switch to Scala?
This post is a role-play and does not reflect my individual opinion about scala accurately. I am convinced about the capabilities and features of Scala along with the fact that it deserves the mantle of a long term replacement for Java. However language adoption goes beyond technical capabilities, and this post is a speculation on what a typical manager might be dealing with when attempting to decide whether to switch to Scala.
So I have been reading a lot about Scala lately and even opinions about how it will be a long term replacement for Java. I’ve also read some interesting writeups about Scala adoption such as On Scala’s Future and A Tipping Point for Scala. While I used to code a lot, my responsibilities today require me to interact with and address a lot of issues including those faced by our customers, our development teams and also engage with my peers and superiors on many other difficulties bedeviling our organisation. This gives me little time to try out Scala. I know I should be doing that, but sincerely I do not have the time. So I rely on the feedback of my team, the trade journals and other influential architects within and outside my organisation.
I have heard about many developers switching from Java to Python / Ruby. However I have heard of relatively only a smaller number of large Java shops which have done the shift – most of the switch stories I’ve heard reflect a smaller sized teams. I can feel the excitement Scala has generated amongst the development teams – the brevity, the functional programming model introduction, the exciting stuff being done concurrently et al. I have no doubt that, given so much excitement it must really be a good language.
To introduce my organisation – it is one of those shops which service many projects concurrently. Given the tremendous business and growth, I must confess we do not always have the luxury of being able to hire the most top notch talent. We do have a lot of projects we use Java for, and thats a language our customers are comfortable with. I’ve had some of the senior people check out Scala to gain some feedback into the language. But at this stage I must say I am inclined to evaluate the shift but not convinced enough to do so. I am sure that I could if convinced drive the change to Scala incrementally. However my fear stems from the fact that if things don’t turn out well, despite all the great advice I’ve received – its going to be my rear end on the line. So here’s some of my concerns regarding evaluating the shift to Scala and there are many of them, so some of you might be able to help me through this thought process.
- Functional Programming : I’m sure in many ways it rocks. But my guys tell me they are not sure how to use it in the typical bread and butter applications which read from database, do some processing and write back to the database. Does Functional Programming help me in this context ? Will my team scale into being able to write functions with no side effects assuming thats a desirable goal ? What if they tie themselves up in knots and my release to the customer is risked ? I can’t afford that. Is functional programming even desirable in such contexts ? So I am not sure if in these contexts I should just ditch functional programming and work with just normal imperative programming capabilities of Scala. I am so confused, and afraid.
- Different Syntax : While Scala runs on the JRE, its syntax is very different from Java. From what I could gather, it is much easier for a Java programmer to read (make sense of) simple Python code than to read Scala code. Is it true ? So even if I do get compatibility in terms of the runtime environment, would I be picking up a language that is syntactically so different a language that it would involve a substantial relearning curve ? I remember when we had to learn Java and Javascript. For better or for worse these were indeed relatively minor modifications of the C/C++ syntax, compared to what I sense as the syntactic shift between Java and Scala. Am I wrong ? If so, could you help point me to resources which help me understand that Scala code is not much different than Java ?
- Sample code : Guys, I need your help. I need to see some good sample code. Some code which reflects how a typical application is architected, designed and programmed in Scala. And I don’t need it for a complex multi threaded actor based processing – I just need to see simple J2EE server based departmental applications maybe a simple recruitment tracking or library maintenance application. If I find a good one, I’ll just take it and give it to my team and say – there, thats how we’re largely going to build it, and even if we make a few changes along the way we at least have a reasonable template that we can build from.
- Dumbed down environment : I remember my great adventures with C and vi and make. But my team today is very different. They want great IDEs. They must have syntax highlighting, autocompletion and nice refactoring capabilities. If I ask them to move, some of them might be excited about the change and be willing to overcome these short term hurdles. But there are some of them who will not be keen to do so and may be disinclined to support such a shift. And at the end of the day my ability to conduct this shift is a function of my ability to carry a large proportion of them along with me. Even when I considered a shift from svn to git, the IDE support was a big issue even though quite obviously git capabilities were really exciting. I couldn’t push along that change, and in this case we are talking of changing the language.
- Is this a good time to shift to Scala ? I remember the early adopters of Java from 1996 thru 2001. While they gained a lot of experience, JRE and J2EE really matured only post JRE 1.3. Scala seems to be coming out with so many enhancements so fast, I am not sure if it has stabilised. I am told there is a 2.8 coming out in a few months. So if I train my team and Scala continues to change rapidly will I have to keep on retraining my team regularly ? And what about the customers I take to production. Will the frequent upgrades mean I end up supporting multiple customers on multiple versions of Scala ? Maybe Scala is stable but it would be helpful for someone important enough to make a clear statement that there are no new major shifts anticipated anytime soon and that these version shifts are likely to be no faster than the JRE version upgrades (which were fast enough).
- Support from peers and superiors : I remember the day I decided to shift to Java. What made the move easy for me was the sheer fact that Java was a big paradigm leap away from the then dominant C++. Not only was it cross platform with binary compatibility thrown in for good measure, Sun ensured that it made all the right noises to appeal to the enterprise architects and all the business managers. I see the senior developers in my team clamouring for the shift to Scala, but my peer managers and my superiors don’t display even the fraction of the enthusiasm they displayed during the Java shift. The implication for me is that the risk cover I get when I order the shift is far lesser than what I had when I made the move to Java. Which means if things don’t quite work out well, I’m really going to be screwed.
- Business friendliness : I understand all the nice talk about the technical excellence of Scala. But I really need to translate all these great language features into a projected ROI that I can use to convince others about. So I would like to see actual case studies of applications that were moved to Scala and what impact it had on the time and cost so that I can use it to compute my ROI. And what scares me is that learning curve may risk the initial applications long enough to push my breakeven point of shifting to Scala well beyond a 12 month and perhaps even a 24 month period. I fear things might not be as difficult but in absence of known studies, I am likely to lean towards projecting a worst case scenario rather than an optimistic one.
So folks, I am asking for your help. And while a lot of you may think that people like us who balk at the thought of limited IDE support are wimps, please remember that 80% of us don’t fit into the top 20%. And if you would like Scala to be popular, you need us as much as we need you. And if you are not too sure, please remember Lisp and Smalltalk are great languages as well.
About the author – Dhananjay Nene
Dhananjay is a Software Engineer with around 17 years of experience in the field. He is passionate about software engineering, programming, design and architecture. He did his post graduation from Indian Institute of Management, Ahmedabad, and has been involved in Senior Management positions and has managed team sizes in excess of 120 persons. His tech blog, and twitter stream are a must read for anybody interested in programming languages or development methodologies. Those interested in the person behind the tech can check out his general blog, and personal twitter stream. For more details, check out Dhananjay’s PuneTech wiki profile.
![Reblog this post [with Zemanta]](http://img.zemanta.com/reblog_b.png?x-id=3662453a-ffd6-4852-85b3-c160ba6257f5)
I don’t know about the language syntax and the works about Scala, but I hear the JavaPosse podcast and know from there that Netbeans has excellent support for Scala. You may want to evaluate that.
I am very interested in the feedback you get on all the points you have raised, since I have been thinking about these things too.
Keep me posted when you summarize your findings.
I find this as much a post about dealing with incompetence and mediocrity as about dealing with new languages/technologies/tools. I am not a Scala fanboy, and am not trying to stand up for Scala in particular, but against some of the managers who try to pass off technical incompetence as real-world software-engineering considerations. Note: This is neither a personal attack against the author in particular, nor a diatribe against all managers in general.
Functional Programming:
Sadly, 99% of all managers and programmers have never been exposed to functional languages, and so don’t really understand either their features, or their generality.
Q: Does Functional Programming help me in this context?
A: Probably not in the specific context, unless you’re doing some nontrivial processing of the data read from the database. If you have to hit a nail, a hammer is generally a good tool. If you have never used a hammer for this purpose, investing some time to try it out will probably not be a bad idea.
Q: Will my team scale into being able to write functions with no side effects assuming that’s a desirable goal?
A: Functional programming isn’t really centered on purity. You can be impure (mutate data structures) and still be a bonafide functional programmer. Functional programming, among other things, is as much about type safety, higher-order programming, and pattern matching, to name a few features. As for your team scaling, Scala, doesn’t seem to force the functional features onto the programmer. You could always start small, learn, and grow. At the outset, it would be good to have a reason to use Scala in the first place. A reason that is more concrete than following some blog that claims that it’s the best thing since sliced bread.
Q: What if they tie themselves up in knots and my release to the customer is risked?
A: They can just as easily do it with any other language, even one they claim to fully understand and love.
Q: Is functional programming even desirable in such contexts?
A: Functional programming languages are general purpose languages. This question is just as valid as the question “Are imperative/logic/stack-based/object-oriented languages desirable in context X?”. I assume that this question is more than just rhetorical and that the managers would actually work and get an answer.
“I am so confused, and afraid.”
Education helps get rid of both. Being a manager shouldn’t mean that one stops being technically aware.
BTW, if the typical application is simply to read from databases, do some processing (I assume really trivial processing), and then write back, and Java works perfectly fine, then I’m not sure why look at something else.
Different Syntax:
A college graduate with at least a few years of experience should really be technically mature enough to be able to surmount small difficulties posed by new syntax. Again, I consider this a lack of competency masquerading as a technical consideration. I can appreciate the existence of a learning curve in understanding when to use a feature, the set of available libraries, the APIs, programming styles, but getting stuck at the level of syntax is mediocrity posing as a technical challenge.
>> ..it is much easier for a Java programmer to read (make sense of) simple Python code than to read Scala code.
If your manager claims to love Perl and raises this issue, fire him instantly. As for Python, it also has functional features, and can be fairly terse. Ask your manager who claims this to explain as to why this code (reproduced from http://www.ibm.com/developerworks/library/l-prog.html) is more readable to a programmer used to C++/Java, than what it would be in Scala.
bigmuls = lambda xs,ys: filter(lambda (x,y):x*y > 25, combine(xs,ys))
combine = lambda xs,ys: map(None, xs*len(ys), dupelms(ys,len(xs)))
dupelms = lambda lst,n: reduce(lambda s,t:s+t, map(lambda l,n=n: [l]*n, lst))
print bigmuls((1,2,3,4),(10,15,3,22))
I don’t think that the syntactic differences are so great between Scala and C++/Java, and hence do not consider this as a valid issue.
Sample code:
A simple Google search is only a few keystrokes away. http://www.ibm.com/developerworks/java/library/j-scala12228.html gives a good introduction to Scala in the context of developing web applications. For more general examples, please look at “Scala by example” (by Martin Odersky himself), and a longer http://www.artima.com/shop/programming_in_scala (also by Odersky (and others)).
Dumbed down environment:
Scala already seems to have an Eclipse plugin, but I have another problem here: Why is dumbing down tools a good thing (especially if you can still learn a few things and be as productive or more productive)? The point is, would you rather have programmers who know make/Ant enough to be able to read it/debug it, or be completely dependent on the existence of an IDE which spoonfeeds them all the time? I’m not a masochistic 25×80-green-monochromatic-text-terminal person, but really, your team members who *must* rely on these tools aren’t different, they are probably just not as competent or more productive, the availability of productivity tools notwithstanding.
Q: Is this a good time to shift to Scala?
A: Has there ever been a good time to shift to C++? It’s still evolving (with fairly fundamental changes that are still being written about in research papers). If your team needs to be “retrained” on new releases, then it’s incompetent. While backward feature/bug compatibility is a genuine issue, it’s as relevant to Java as it is to Scala, as it is to the next standard for XML/C++/C#/EcmaScript/???.
Support from peers and superiors:
I assume that a good manager would be able to
1. Understand enough technology to ascertain if benefits clearly outweigh risks. If the manager claims to be a non-technical person, then I expect her/him to have enough trust in a competent technical person to get this analysis right, and then to use it without prejudice.
2. Be lucid enough (and dumb down the technical stuff enough, if necessary) to convince nontechnical superiors
Business friendliness:
I have yet to hear a completely objective and quantitative ROI-based analysis of any particular platform/language/tool (Maybe I just haven’t looked in the right places, or hard enough, and would love to know more on this. I’m also not convinced that there is a way to have a real-life scenario comparing two options where you start with equally talented teams) — why is Scala any different. Did you really get Microsoft to give you a list of concrete metrics with which they defended the existence of the .NET platform?
Navneet,
There’s some fairly decent feedback on the original post itself here : http://blog.dhananjaynene.com/2009/08/why-should-i-switch-to-scala/#comments
You’ve to read very carefully between the lines to realise this post is the findings and not a question (since it is presented as a roleplay). But if I had to represent the information as findings (my personal subjective opinions), these would be :
a. Functional programming : Ditch FP – Intially just treat Scala as an OO language. Build (and use FP skills later)
b. Different Syntax : The syntax is different. Invest in learning it. No way around it. But should that even be an issue for good programmers ?
c. Sample Code : The community needs to invest more in writing and publishing sample code.
d. Dumbed Down Environment : From all I know, IDE seem to be getting better. Both eclipse and netbeans have scala support and its getting better even though it may not satisfy the laziest (I choose the word carefully) developers.
e. Good time to shift: Java shop and not single Scala project in the pipeline ? Reconsider, and try to at least start building the skill sets in a lower risk / criticality or internal development project.
f. Support from peers ? Go do some work and start communicating internally to start building support. Sun helped you through with Java – its not going to in this case. But you (the manager) still have to do what you gotto do.
g. Business friendliness : Start a small development project internally and start getting some feedback. Its your job to assume for the worst case but its also your job to take measured and managed forward steps – doesn’t Scala look like one which deserves the latter ?
KS,
Great set of points. But you’ve actually re-questioned very issue I was raising. Should the community expect such manager’s to come up the curve or should it step forward and help him without being judgemental.
If a language has to be made popular, it has to be made easy for a lot of folks including the manager above. Shaking the head, and saying all it needs is better intelligence, more than average effort etc. are not directly relevant. I’ve already described the manager to be busy in tons of issues and unlikely to have the time to spend on learning and coming to terms with Scala, and yet having the disarming honesty to appeal for help.
To expect that managers will have the time and inclination to always go out and learn more is like making a statement “the world needs to be fair” – accurate but irrelevant. The rationale to it lies in organisational structures and management control systems the way they are conventionally structured. To expect either of them to be modified to come knocking on the doors of Scala (or any other technology) without a clear driving force of what it means in terms of an ROI is simply wishful thinking. I’ve been through the phase of decrying how management works (never actually left it in my mind) and would submit that it sometimes is more useful to adapt rather than critique based on the local context.
But this post raises an interesting counterpoint which I quote below.
One of the great things management processes teach you is if there is an impediment in your goals – go figure out what it is and do what it takes (often bending over backwards) to get rid of it. There is no pride or arrogance or (and quite unfortunately sometimes I’m afraid) morals involved. These are sufferances which are not widely observed. Thats a trait that might not be entirely forgotten in this context.
What I do point out is that if Scala (or for that matter any other technologies) need to achieve mainstream popularity, it will not just be a question of “build and they shall come”. One will not have to only build great technical stuff, but also go out and sell it to the possible users often overcoming arrogance in the process. Thats the reference I make to when I refer to wimps/IDE in that quote and also to the fact that Lisp and Smalltalk are great languages too – but they are nowhere near what one would term as having reached mainstream popularity.
Here’s how I read the response. Do correct me if I’m wrong, and please do not take the first-person singular pronoun “I” personally:
“I’m an extremely busy manager in an industry that builds software, but have no time or inclination to understand technology that’s used to build it.
I’m too busy managing persons, logistics, deadlines, financial targets, escalations, conflicts to have any time left over for looking at technology.
I shouldn’t be expected to take the time and effort to understand technology to build a potentially better product — that’s far too idealistic and irrelevant to the way things work. On the contrary, it is the new technology and the people behind it who need me to get popular and attain widespread acceptance.
In fact, it’s the larger community that needs to accept my humility in admiting my difficult position and must do the work of building productivity tools, instructive manuals and tutorials, reducing my technology-related risk to zero, and give me all the ROI and risk analyses that my superiors may demand from me. Don’t bother knocking on my doors unless you have done all of this.
Of course if the technology works, I get the kudos and the larger pay packet for all the work that the community did. Life isn’t fair, so don’t expect me to change my ways.
Einstein can say what he wants, but technology must be made so simple as possible, and even simpler to appeal to the lowest common denominator.
WIMP may have given us the Mac, Simula may have given us Smalltalk/C++/Ruby, Lisp may have begot(?) a bastardized nephew in the form of XML (a quote attributed to Philip Wadler), Pascal p-code machines may have ended up as JVMs but none of these technologies/tools/languages/artifacts were rock-star-level in their appeal, so, in effect, they have failed.”
Aside: This schizophrenic post commenting is hard (separating I the me from I the manager). In this comment below, I refers to the manager
@KS in particular and the tech community in general.
Seems like attempting to educate is boomeranging on me by questioning my abilities or inclinations to educate myself. So lets quickly get some relevant facts on the table.
I don’t really really need to adopt Scala (or any new technology) right now. There is no business imperative to do that. I can choose to wait. And I can choose to see if things really work out well for it. Whats my business cost if I defer adoption for a little while more ? Frankly quite little. What I will do is to have a low priority thread to further educate myself but until I feel comfortable about the risk/return tradeoff for my customers in my context I shall keep it exactly that – low priority.
You paraphrased me correctly when measured in accuracy terms. But I feel brutally victimised when I read between the lines. Since there is a tone there which is very uncomfortable for me. And that tone makes me seem like a lazy bum or unconcerned about investing in technology – whereas I tend to believe I am focused more and most on where it is most important – my customers. Believe me, my customers aren’t telling me to adopt Scala yet and they are unlikely to do so for a long time.
Frankly, to make Scala great (assuming for a moment already – it isn’t) – I’m not required. Thats not a contribution I am inclined to make. I am not even per se excited by the goodness and greatness of Scala since at the end of the day I shall measure it based on the ROI it offers to me and my customers. So really, don’t bother pleasing me or even reading me if making Scala a wonderfully elegant language is what is your goal. I am a non player in that space.
But if Scala needs to achieve mainstream popularity, and especially if it needs to achieve it relatively sooner, I am a player. When I am sold, I can carry my team with it. Along with that I bring in the ability to convince my customers to assume the risk. And if I am successful, I bring in the ability of getting more developers, managers and customers excited about it. That is the key to mainstream popularity, and I can play the role of one of the early dominos to fall.
And if serving it all to me on the platter is seemingly what I am asking for, whats wrong with it ? Afterall, in return, if successful, I shall be getting you a whole new bunch of converts, proponents and cheerleaders. And if you think it is a one way street where me adopting it successfully gets me better pay packets and promotion, be sure to realise that if it fails, I shall be the one to take the hit as well.
The question isn’t really me or my attitude. I’ve been wasting my time if thats what you got out of this whole discussion. The question is if a good sexy piece of technology does need to get to mainstream popularity, am I a relevant participant given all my so called laziness, inattention and serve me attitude ? And if indeed I am a relevant participant, I’ve described what increases my probability of success. I believe I see some good in adopting Scala, and I think there is a good likelihood of doing so one day. But I can afford to wait, not because the technology is good or great or otherwise – simply because I will do the shift only when I am convinced for a proper risk/return characteristic for me and my customers. I will do the shift when the technology is usable as measured from my perspective.
very good article. learned a lot after reading. one question – any update with respect to lift 2.0? does it have good future? will it be able to make bread and butter (and have it eaten from within a golden bowl)?
I am asp.net developer and looking for a change.
and i am stuck at ruby or groovy or scala or foobar