The Yin-Yang of Software Development

By Tariq Siddiqui, principal architect, EMC Corporation

  • The Ying-Yang of Software Development

Innovation has a recipe

Creativity and innovation have long been the hallmarks of software development. However, developers are witnessing a substantial change, not only in terms of what kinds of software products they're producing, but also in terms of how they're designing those products.

With the advent of rapid-development tools, the evolution of frameworks and platforms, and the emergence of agile, lean software development methods, development cycles have accelerated significantly. Development time has shrunk from years to months, and in some cases, to weeks.

Creativity, aligned with efficient processes, forms today's software development yin-yang effect. It sets the stage for solutions that are innovative and are developed with super speed. Creativity and efficiency should always be coupled together; they work best as a team.


Creativity is about ideas and imagination. A creative idea, realized, ends up being an innovative product or service. But what does creativity mean specifically in the context of software development? Does it mean "writing creative code"?

That would be a good start. But actually, the most creative way to write code is to write no code at all. Instead, reuse, leverage, and integrate it. One needs team members who can envision creative solutions to a problem but who also realize that creative thinking, by itself, delivers nothing. A team must put a creative idea through efficient processes and mechanisms to turn it into a real solution.


Efficiency is about effortthe amount of effort needed to accomplish a task. In software development, achieving "operational efficiency" means optimizing staff and resources while streamlining processes to deliver the creative solution with minimal effort.

Different methods exist for measuring efficiency. At the project level, you may monitor key performance indicators such as cost adherence, schedule adherence, quality, scope, and delivery. At the organizational level, you may measure productivity, capacity management for the affected IT platforms, and pipeline and portfolio management. At an individual level, you may monitor resource utilization vs. allocation.

Core principles

Recently, my group at EMC was asked to build an "IT Metrics Dashboard" to monitor the availability, performance, and scalability of our company's mission-critical IT systems. My manager assembled the team very quickly because we had to prototype, develop, and deploy this product within a few weeks if we wanted to consider the project a success.

Our traditional linear model of "define, design, and develop" went out the window. It was replaced by "adapt, innovate, and create."

Every phase of the project had to move with super speed, and all phases had to align eventually. We had to put agile software development methodologies on steroids to make this dashboard happen.

But agile practices don't deliver anything by themselves. We still needed to define what we were going to deliver, how we were going to construct it, and when we were going to deliver it. And to achieve that super speed, we had to synchronize our:

  • Team synergy
  • Simplicity
  • Outside-the-box thinking
  • Reusing, integrating, and leveraging of code
  • Expectation management

Those five principles are the underpinnings of creativity and efficiency. Each element feeds the others. But even by themselves, they provide great value to a software development project.

Team synergy

"No one can whistle a symphony. It takes a whole orchestra to play it."Halford E. Luccock

Team synergy blossoms when the work produced by a team collectively exceeds the work produced by the individual team members.

We've all seen this phenomenon unfold: It happened during the 2004 Red Sox World Championship Series. It happened with the introduction of the Apple Mac in 1984. We saw it in the groundbreaking animation of Snow White and the Seven Dwarfs in 1937.

Synergistic projects happen on smaller scales, too. And then, excitement is in the air; things are electric; magic ensues. The result is exponential progress, not mere linear forward movement. I have had my share of such experiences in my 20 years of developing software.

If you're leading an effort such as this, it is crucial that you know where your team's expertise lies, and, equally importantly, which areas are not within their realm of expertise. Developing software is a collective effort. All team members need to complement each other's skills.

"Team chemistry" is often regarded as the defining attribute of success. But I believe it's team synergy that delivers success.

A few years ago, I was assigned to lead a high-visibility project that was, initially, to have been a six-month effort. The project was in trouble. It was already late by three months. I was told to fix itto finish it within six weeks. (This sounds strange, but it's not as strange as you might think.)

Well, that first week, I realized the development team did not have the right combination of skills for the project. They had great chemistry, but they were all new Java developers with more or less the same background. Therefore, they were all stumbling across the same nuances and subtleties of the technology.

I stepped up, filled in the holes, and worked like a superman, and still, the project exceeded deadline by two weekstwo weeks too many. My lesson from that painful experience was that complementary skills are an absolute must for team synergy and success.

And if you're hiring a new addition, then hire someone who is a sharp learner, who is versatile, who works hard, and who is excited about the work. Most tech interviews are "gotcha" interviews, in which the interviewer tries to trump the interviewee. That's so self-defeating.

You will never find someone who already knows everything that you or your group knows. (The reverse is also true: You will never know everything your interviewee knows.) As long as the person can learn, has a solid programming foundation, and loves this work, then I am okay with him or her.


"Everything should be made as simple as possible, but not simpler." Albert Einstein

"Simple software" sounds like an oxymoron, but it isn't. Software-development simplicity doesn't mean simpler code or less functionality. It means the software solution is simple to design, develop, deploy, support, and maintain.

So many architectural and design patterns exist; they make one's head spin. Don't get me wrong; these are great, handy references, but not everything needs to be a sophisticated multi-tier architecture.

Over the past two decades, I have seen my share of systems that provide little functionality but are heavyweights when it comes to required support, maintenance, and extendibility. My belief is that a system should be self-sustaining. It should require minimal maintenance and product support. My advice to others is to keep software solutions simple, without oversimplifying things.

Outside-the-box thinking

"There is no wisdom in conventional wisdom." Scott McNealy

As I mentioned, software development cycles have shrunk from years to months or weeks. My job as a software architect today is to construct systems with more and more efficiencyto provide creative solutions that can be implemented quickly.

A little more than a year ago, my team and I were asked to build a web application in approximately six weeks. Previous attempts to deliver this product had been unsuccessful due to the application's rather demanding performance requirements.

We had to be cognizant of those performance requirements and our own time constraints. So, instead of building everything from scratch, we researched and bought a third-party tool that could provide functionality that would have taken us months to build by ourselves.

Buying the license for that tool cost us just a few hundred dollars. We got a skeleton architecture up and running in no time, and we delivered the product within our allotted deadline. The project became a perfect example of how outside-the-box thinking can lead to a successful outcome.

Reusing, integrating, and leveraging

"Learn from the mistakes of others. You can't live long enough to make them all yourself." Eleanor Roosevelt

Never write a piece of software that has already been written. That sounds like a sweeping proclamation, but truthfully, someone, somewhere, probably has already developed the system you want to develop. Take maximum advantage of that existing work.

I am always looking for things that are available either internally or from the free and open source software (FOSS) movement, or are for sale at a minimal price.

SourceForge.net is the biggest FOSS repository. This web-based source code collection currently hosts 180,000 projects and 1.9 million registered users. SourceForge, along with its rivals RubyForge, Tigris.org, BountySource, JavaForge, and Savannah, offer a wealth of software to fill any need. (But be advised: Finding the exact code you need can be akin to looking for a needle in a haystack. You need to know exactly what you are looking for. The newly restructured SourceForge.net is a great start in making this effort easier.)

Managing expectations

"The first step in exceeding your customer's expectations is to know those expectations." Roy H. Williams

My job as an architect is to ensure the business's expectations are reasonable and that I am striving to align them with reality. I believe in setting the right expectations, then over-delivering on them: Don't over-commit and under-deliver.

I remember when we were asked if we really thought we could deliver that IT Metrics Dashboard in a few weeks. I tried my best to lower the expectations. Then, all of us did our best to over-deliver on them.

Final thoughts

Software development isn't rocket science, but a rocket is pretty old technology at this point, anyway. Rather, software development is both an art and a science. The scienceespecially the efficiency aspectscan be taught, and the artthe element of creativitycan be learned.

You might as well have a blast learning it.

No resources matching your request were found.