The “New Work” of the IT Architect: Innovation Labs and Incubators

Last week I enjoyed my semi-annual “whack on the side of the head” meeting with my Pioneers—our lead users who are veteran technology and business architects. This small band has been gathering twice a year since the early ’90s. What they have in common is a way they view themselves and the world. This worldview evolves over time, of course, but there seem to be some basic principles that are wired into their nervous systems/perceptual filters.

They were all pioneers in the early days of distributed object computing—so they view the world as a place in which intelligent robust services come together dynamically to empower and enable complex operations. They have designed and implemented a variety of very impressive systems over the years—from nuclear power plants to global banking systems, from complex logistics systems to derivatives trading systems. Their IT designs often feature large shared memory in which big abstract models can run—these abstractions are connected to the physical world and instrumented with real-time events. The designs often incorporate rules engines. Yet they love simplicity. Every element is self-contained and self-describing. 

These people aren’t “geeks.” They’re social architects as well. They spend as much time observing the natural world as they do the world of bits and bytes. You give them any business problem or technology challenge and they’ll come up with a surprisingly similar way of approaching it. I’d love to harness this brain trust. Maybe it’s time to launch “Architects R Us”—your intractable problems may have relatively simple solutions when viewed by Patty’s Pioneers. Let me know if you’re interested in tapping into this brain trust!

This fall’s Pioneers gathering was both intimate (several of our loyal band were absent) and bandwidth-content-rich. (A good bottle of wine helps!) Here are a few of the patterns that I took away (more next week).

The Programmers Have Taken over the Enterprise

What pioneers see when they look out over the typical large enterprise landscape is an environment in which computing is truly commoditized. There are production IT shops, outsourced coding, and hundreds of people toiling away working on uninteresting problems in uninspired ways. They note that although business leaders have become increasingly IT-savvy, they don’t have the knowledge or the vision to direct IT strategy. At the same time, IT managers—CIOs—have become producers and project managers. Their “strategy” has devolved into resource allocation and project management.

In today’s enterprise software world, there are too many people working on most projects. There is no architectural planning or discipline. Application platforms are adopted, worked on, and then rejected. Major projects are missing deadlines. Millions of dollars are being wasted on projects that crash and burn. This isn’t new, but it’s getting worse.

Pioneers believe that the more people you throw at a problem, the longer it takes. Armies of coders don’t develop good software—they just maintain and patch existing software. Pioneers favor very small teams of very smart people solving interesting problems. There’s no room for that in corporate IT anymore. Maybe at Google?

So where does human creativity naturally go, in the currently stultifying environment of corporate IT? To open source, of course (and to Web 2.0 hacks). So what’s happening in many corporate IT shops is that open source has run amuck. It’s not that it isn’t valuable and useful. But it’s often out of control. Developers with a project to deliver grab open source routines, add a few tweaks, embed it into their own code and check it in as their own. The result is an auditor’s nightmare. Who owns what? Where did this come from? How do we know it’s OK to use? The antidote to this renegade behavior are services like Black Duck and Palamida that will scrub your code to detect open source and manage your IP risks.

Where Do Senior Technology Architects Go?

In this currently bleak world of commoditized corporate IT, veteran technology architects are ambassadors without clout. They draw pretty pictures. They try to get everyone to agree to adopt simpler, better-designed approaches to problems. They are itching to roll up their sleeves and get back to work actually designing and delivering real-world solutions, but there’s no place for them to do so. So, they are:

• Starting their own businesses
• Turning into venture capitalists
• Going back to school to tackle new disciplines
• Occasionally being given new spin-offs to fix or to run
• Wishing they could “take over” the business they’re in. One architect with a staff of 100 very smart people moves from company to company delivering huge value and then moving on because he can’t actually run/influence the direction of the company—even though he has successfully re-architected it.
• Running Innovation Labs!

The “New Work” of the IT Architect: Innovation Labs and Incubators

Here’s the epiphany we all came to as we looked at what was happening in a few of these pioneers’ firms. A great fit is to tap one of these senior architects to head up your Innovation Lab. They can tackle hard problems quickly with new tools, yet still apply proven architectural disciplines and reuse tried and true design patterns. They can leverage the talent of your renegade programmers in a “safe space”—a place where experimentation is encouraged because you aren’t trying to develop code that meets all compliance regulations, just code that works and delivers value. They can also transfer their wisdom to the young, smart Turks that you’d like to be able to recruit and keep. This knowledge transfer happens not because they’re senior and the boss, but because they can show the young hotshots better, more efficient ways to do things, as well as ways to avoid unintended consequences.

It is encouraging to note that innovation labs appear to be sprouting up all over the place. In comparing notes, we discovered several varieties:

• Customer Innovation Labs. Made popular by Google and Yahoo! but since adopted by organizations as diverse as the BBC and Fidelity, these are labs whose purpose is to provide interfaces and tools for external developers and super users to use. The original purpose was for external customers to try new things out (as in, “this is in beta, use it at your own risk”). Today, these customer innovation labs provide APIs and tools that are designed to be mixed, matched, and extended by end users.
• Incubators. These are usually internal labs that are set up to spawn rapid prototypes, test new concepts, and develop new solutions. In the past, these were “internal only” affairs. An incubator is a place to experiment with agile development, with risky technologies, and maybe a test bed for projects that need to be fast-pathed. Today, the trend is to break down the wall between internal experimentation and external experimentation. Invite select customers in to roll up their sleeves and play along. Or open up certain projects to the world at large. Make your test bed a global one. 

• Labs for Internal “Lead Users.” One of our pioneers spotted the beginning of a new trend. It’s a lab that encourages your lead business users to strut their stuff and to continue to experiment. Every company has these renegades. They’re the people who use Excel in ways it was never designed to be used; who build their own departmental databases to slice and dice information in ways they need, and to organize data and knowledge in new ways. At some point, these renegades hit the wall. Either they get tired of maintaining a home-grown application that is now essentially a “production” application. Or, they trigger the organizational immune system and the corporate IT folks have to take over their project to make it compliant (e.g., secure, scalable, backed up, robust, and auditable). Why not, she argued, formalize this “lead user” activity and create a safe space in which business super users can go ahead and push the envelope? After all, they’re either creating throw-away apps that will die a natural death, or they’re creating working prototypes of things the business really needs. Having an architect oversee an internal innovation lab is a great idea. You’ll get maximum reuse and clear migration paths from prototype to production. 
• Online Gaming Platforms as Business Innovation Labs. There’s also a relatively new phenomenon afoot: Using an online gaming environment as a test bed for both new technologies and new business prototypes. Several pioneers pointed to the construction of pilot environments in Second Life, for example. In this case, you’re setting your innovation lab up in a parallel universe that is inhabited by real people who are assuming their fantasy world alter egos. But, as the pioneers pointed out, the social constructs are the same. People are social animals and their human behaviors—forming into cliques, foraging and creating, networking and accumulating, cheating and fighting—all seem to play themselves out in virtual worlds.

I’m sure there are many other kinds of innovation labs. What kind(s) have you found? Are these internal only, or do they invite participation by external users? If so, are they wide open or by invitation only? What kinds of people are running them? Are senior/veteran IT architects engaged? If so, are their contributions valued?