From Agile Scrums to Codeless Workshops - The Evolution of Enterprise Software Development

Will the transition from Agile Scrum development to Codeless Real-Time Development Workshops be as high impact as the move from Waterfall to Scrum?

The Journey from Waterfall to Codeless Software Development

In the Beginning

For companies that were accustomed to adopting a waterfall project approach for their software developments, the move to agile scrum methods has been dramatic, both in terms of its results and its impact on the size and skills of project teams.

Waterfall describes a traditional software development approach (that dates back well into the 1970’s) where different threads of development are performed by different development people or teams. 

When development teams adopt the waterfall method, projects are scope up front and then components are developed in parallel by different people – often employing specialist skills and tools.   The number of people involved in projects and the many threads of activity creates a complex project model that then requires expert project management tools, methods and skills to orchestrate (PRINCE2 became the answer to this eventually and led to a whole new set of skills requirements for companies to mince over).

Following development of the various streams of the waterfall, these are bound together and formed into a final ‘end-product’.  This approach has been fraught with project management issues. Delays in any of the parallel developments causes the entire project to over-run.  Often, ‘next steps’ of integration with third party tools and data, and activities such as testing and tuning, can’t be progressed until the last thread of development has been done. This can result in huge project over-runs and costs.  

The article ‘Why Your IT Project May Be Riskier Than You Think’ by HBR (November 2011), that followed a survey of 1,471 IT projects with an average spend of $167m, found that the average overrun was 27%, one in six of the projects studied was a black swan, with a cost overrun of 200%. And almost 70% of black swan projects also overrun their schedules.

Even when projects do get delivered on-time, the consequences of using one or many expert programmatic tools means that documentation is more complex, testing and tuning takes longer, and more bugs are likely to be embedded in the code. EVEN should everything go according to plan, the remoteness of intended system users and stakeholders to the back-room development process means that resulting systems are difficult to get right as a perfect fit to the user community.  It’s quite an art to write a paper specification for a software application and produce a perfect system right-first-time.  More like impossible in fact.

A major challenge all development teams face is application users and stakeholders rarely know PRECISELY what they need in terms of database management, reports, process workflows and user interfacing requirements out-of-the-box.  Normal domain expert business users find it hard to visualize end-applications and often struggle to formalize their end-product in a format that programmers can use. 

Accepting the high resourcing costs, need for costly specialist best-of-breed tools, use and constraints of coding and its impact on testing, tuning and adaptability, perhaps the biggest issue of traditional waterfall development tools and methods however lies in what gets left behind.  Programmed systems are inherently difficult to maintain and adapt. Companies don’t particularly want to be left with systems they can’t change over time, or that require a team similar in size to the team that built the application to maintain it into the future (mainly because of the dependencies created on a smorgasbord of best-of-breed tools and skills.

The step-up to AGILE SCRUM

Advances in Integrated Development Environment (IDE’s) since the turn of the century, aided by the timely emergence of standards around enterprise web server computing platforms, has produced a number of capable tool-kits to enable software development teams to work more closely together with fewer tools and fewer expert skills.  This reduction in tool-kit complexity has meant the project approach has been able to transition more towards a team-based culture where individuals working towards a shared goal can work together in a collegiate way.

AGILE tools are so-called because they provide development teams with more adaptive tools that produce results faster.  One aspect of the ‘agility’ comes from using bigger building blocks of code that can be shaped to fit a variety of needs but doesn’t require programmers to start with a blank page. 

The SCRUM model has evolved around the need for members of development teams to keep in close contact with one another and stay on the same page.  More frequent reviews of progress – perhaps on a weekly, even daily basis – ensures that the streams of development stay on-track and can be regularly check to ensure the quality of the end-product is as it needs to be and that project managers can check time and again that ‘what is being produced’ is ‘what the internal client wants’.

AGILE tooling does not provide developers with everything they need in one box.  To improve the integration and use of third party expert tools, software providers today normally provide an Applications Programming Interface (API) so the various tools are EASIER to integrate with one another.  While this is EASIER it remains a major overhead on developments as each API tends to be different and developments inherit any limitations presented by the ‘black box’ of functionality provided by the best-of-breed software they are integrating with.  This inability to interplay one application with another using the same properties means that developers find themselves constantly having to come up with work-arounds to produce the best end-product.

SCRUM has its limitations too. While it’s great for developer teams to work towards a common goal and take more of an active interest in making sure their contribution is producing the best ‘total’ outcome, developers continue to work in back-rooms and continue to lack an appreciation of the domain area they are building applications for.  Any developer knows the ‘last mile’ (i.e. managing the interaction with the user and stakeholder communities) is the most difficult part of an application to get right as different user groups will have very different expectations and needs from the applications they use. 

Like the traditional waterfall approach, AGILE SCRUM still relies on coding and that’s a problem.  Not only does it install more post–development overheads are more bugs are built into the end-product, and more testing, tuning, integration is necessary, it means that resulting applications continue to be expensive to maintain and adapt.

The New Age of Codeless Development

So-called ‘codeless’ development software describes Integrated Development Environments that displace the need for applications authors to code.  The leading enterprise platform for this approach is Encanvas.  All of the building blocks required to develop enterprise applications are built into a common IDE. ‘Design Elements’ - as they are known - adopt similar properties and have been designed to provide a sufficient level of tailoring to fit the majority of needs. 

At one time, it was thought that codeless platforms would be only good enough for piloting of projects or serve ‘little application’ needs of the enterprise where larger platforms were too costly to use.  The last decade has shown however that IDEs like Encanvas are capable of delivering enterprise, pan-regional and cross-industry applications to enterprise scale and levels of performance.

In the same way that a step-change in enterprise web platform standards at the turn of the century led to the evolution of SCRUM methods for software applications authoring, use of codeless tools has led to the formation of near-real-time applications authoring methods.

With codeless software, the removal of the programming activity in project (and the removal of best-of-breed tools and the need for APIs) has meant that users and stakeholders can be engaged in development workshops where applications designs are iterated in consort with the users and stakeholders who are able to see and understand what’s being authored because they see the ‘end-product’ being formed in-front of their eyes.

Codeless software has dramatically reduced the number of tools and the variety of skills needed to author applications, but its biggest impact is found in the leave-behind: resulting systems can be easily adapted as business requirements and user needs change.  Neither are there any upgrade costs associated with software; platforms like Encanvas enable hundreds of applications to be authored and do not charge for any new improvements to the IDE.  This means IT leaders seeking to reduce the number of technologies they support in their enterprise IT architecture can shrink their footprint from 100’s of tools to less than 10.

The bi-product of this change in technology tooling and methods makes a big impact on IT budgets. The expectation of IT leaders now is that IT operations should cost no more that 1% of operating profits. Deployments delivered through codeless workshop development methods demonstrate a factor of 10 increase in time-to-market and a similar dynamic in the lowering of onward IT costs.

Will the move from SCRUM to CODELESS be bigger or smaller than the move from WATERFALL to SCRUM?

I would argue the move from SCRUM to CODELESS has a greater bottom-line impact but a lesser change impact compared to the original transition from WATERFALL to SCRUM for the following reasons:

1. IT departments have already shrunk their internal IT teams and now use contractors for much of the heavy-lifting in software development and support. What the move to codeless does is internalize the support of IT systems – making IT more adaptive and better able to support the long-tail of demand for new or better enterprise applications - while displacing the use and cost of contractors. 
2. IT departments are already using IDEs and reducing their use of specialist tooling.  Platforms like Microsoft .NET and Ruby-on-Rails have done much to consolidate tooling for developers.  What Codeless platforms do is FURTHER consolidate tooling.
3. IT leaders are already finding ways to embed IT development skills ever closer to the domain expert communities that use applications.  What codeless software does is take development out of the back-room into the front-office workshop so that developments produce faster, better results that the users and stakeholders WANT to use because they were instrumental in the design of applications they use.
4. IT leaders are already rationalizing the numbers of technologies they support in the enterprise technology stack.  What codeless software does is further reduce the footprint of technologies employed across the enterprise to install more commonality in development methods and tools.
5. Arguably the biggest change factor then with codeless workshop development lies in the impact of an IT department being able to keep-up with the ideas and process changes of the enterprise.  This has surprisingly big organizational design implications.  For decades people in the business have been able to BLAME IT for failing to support their domain needs.  Consider what happens when suddenly it does:  No longer is IT the scape-goat.  Also consider the impact of embedding software development into improvement teams as some innovative companies like AUDI GROUP already do.  The analysts that scope and build the applications are PART of the improvement team, incentivized by the impact of the process changes they manifest rather than being paid for ‘writing code’. Departmental and operational managers will be inevitably more responsible and accountable for IT as part of their function.  They have no choice but to consider the performance of the IT they use as being part of their portfolio of responsibility.  The role of the CIO becomes one of guardianship of IT resources rather than the deliverer of every single IT initiative.

  Keywords:  Waterfall, Agile, Codeless Software Development, CAAD, Encanvas, Software Development Methods, ALM, Application Life-cycle Management

Enterprise mashups and de-risking software development projects

One of the main uses of Encanvas and other enterprise mashup platforms is in custom software development.

Traditionally, business software applications have been purchased as shrink-wrapped products that serve a specific process or functional business area. The fundamental problem of this approach is there are so many processes in a business.  Attempting to purchase a shrink-wrapped solution for every one of them is hugely expensive. Another problem is there's no guarantee that each of the shrink-wrapped packages is architected to use the same building block components of technology - so organizations have to contend with supporting a variety of different database engines, workflow engines and reporting tools (that all work differently).

The alternative to buying read-made software solutions of course is to build your own. The problem in doing this is the high cost and risk of project failure. Developing bespoke software applications has traditionally been performed by programmers and IT professionals in back-rooms working on a common plan that's been drafted as the 'best guess' of what the outcome should look like. Programmers find themselves working in parallel on different parts of the same system. No surprise, when the project team tries to bring all of these development strands together and present it to the users and stakeholders, it's not uncommon for lots of re-working to be needed. Primarily for this reason, organizations make do with buying shrink-wrapped software and coping with the high expense of operating a plethora of different software applications that still don't really fit their business need or how they want to work.

Encanvas, and other composite applications design products, overcome these issues by providing IT leaders with a common toolkit of ready-to-use technology components so they can develop applications across the enterprise as the need arises. What makes Encanvas particularly unique is that it doesn't require coding or scripting skills. Its design environment is completely 'point-and-click'. Even its data integration and mashup features and logic building features are created using drag and drop methods. Another feature is that unlike other Enterprise Mashup products, Encanvas is supplied with ready-made design elements for mapping, forms capture, social networking, reporting, file transfer, dashboards etc. - so there's no need to rely on third party components. This is important for developers because they can see developoment projects all the way through to delivery of a robust and scalable system in the knowledge they have full control over design components and can modify every aspect of the user interface design and application functionality.

By removing the time and complexity of coding in development, Encanvas reduces project risk by enabling business analysts to develop their applications in real-time with stakeholders in workshop environments. With everyone on the same page, it's possible to architect right-first-time deployable solutions.

Time to value on Encanvas is greatly reduced by it use of existing data resources (with its ability to mashup data from existing IT systems) and its ready-made building blocks for complex technology components like geo-spatial mapping.  It means that IT leaders can embed IT-centric business analysts into process improvement teams and provide these individuals with a single design, deployment and operational environment with the dexterity to cope with all of their enterprise information management challenges.