Message from CEO
3-Tier is Dead; Untangling spaghetti complexity calls for new paradigms
Steven ten Napel
CEO
As the pace of digital transformation is accelerating, ISVs and software businesses across diverse domains are struggling to keep up due to their legacy burden. Consider the following scenario, which is the norm in the ISV world:
- 2/3 of time & budget is spent on handling customer escalations, leaving little room for innovation
- Even though there is a consistent demand to deliver software over the cloud, as a service (SaaS), many ISVs are not only not ready, but are also reluctant to share with their customers that they will have to re-architect their legacy software, which will be a herculean effort
- ISVs are scared to touch the database schema of their legacy software, as it has thousands of tables with hundreds of foreign-key dependencies, sometimes even developed by people that are no longer with the company
Twisted, convoluted, and tangled world of spaghetti code
I can fully relate to this situation. Having spent a significant part of my career with Baan, which was the 2nd largest ERP software (only trailed SAP in the 90’s), with thousands of customers (including Boeing), I was used to the following:
- 3,000 tables and hundreds of Foreign Key (FK) relations
- 10,000 screens
- 1 major customer Boeing with 35,000 users across 80 sites, all concurrently using a single, massive database
Can you imagine how nightmarish it would be to maintain a massive software with this kind of complexity?
This is precisely what happened to enterprise software. It has become twisted, convoluted, and tangled and came to be known as spaghetti code, a synonym for unmanageable legacy applications struggling to handle massive modules with loads of code and data complexity.
In my view, even though there are multiple reasons for the complexity of enterprise software, the biggest factor is:
“Traditional 3-layer thinking and databases with hundreds of FK relations are the root cause of all software complexity”
CRUD is Crude
As most developers and users of enterprise software have discovered over the past 15-20 years, the traditional 3-tier architecture that has evolved as the de facto standard for enterprise software, has serious limitations in a fast-changing technology and business landscape:
- Especially in large enterprise-scale applications, the explosion in number of modules with interdependent data and business logic, ended up creating huge complexity.
- This interdependence means, for even small changes the entire application has to be modified and tested, which makes it impossible to be agile in a fast-changing world.
- The interdependence between various modules, makes the application very inflexible.
- As all functional components of a module are tightly coupled, independent scaling of components of an application is impossible.
- Often, due to bad design and development practices, clear separation of concerns is not implemented, making it difficult to modify the UI or change the business logic.
- The conventional frameworks used to develop the UI-layer are not responsive and cannot meet the demands of the new class of devices.
- CRUD is crude: Relational databases that are widely used by most enterprise applications, and the basic CRUD (Create-Read-Update-Delete) operations for data access and storage, were very well-suited to meet the needs of structured data. But, they are not at all suitable to model and store connected data, which leads to extreme complexity. On top of that due to the proliferation of data and foreign key dependencies, even smallest changes to schema have become heavyweight. The conventional relational databases are built for stability, and not for complexity or business agility.
In a nutshell, legacy software applications built with traditional 3-tier architectures are complex and interdependent, and can’t be scaled, upgraded and maintained easily, or to put it simply:
3-tier is dead
It’s clearly evident that traditional 3-tier architecture and relational databases can no longer meet today’s technology and business needs.Untangling the spaghetti complexity and modernizing legacy applications has three distinct facets:
- Architectural modernization
- Organizational transformation
- Delivery decoupling (or deployment modernization)
We strongly believe that architectural modernization should drive transformation of software organizations and enable deployment modernization for delivery decoupling.
In this newsletter, we will focus on modern architectural paradigms that will help overcome the spaghetti complexity of legacy applications.
Microservices
Microservices is an architectural paradigm, explicitly designed to overcome the complexity of interconnected, functional modules in large, complex applications.
“The microservice architectural style is an approach to developing a single application as a suite of small services, each running in its own process and communicating with lightweight mechanisms, often an HTTP resource API. These services are built around business capabilities and independently deployable by fully automated deployment machinery. There is a bare minimum of centralized management of these services, which may be written in different programming languages and use different data storage technologies.”– Martin Fowler
Compared to traditional 3-tier architecture, Microservices offers the following advantages:
- Loosely coupled services that are better aligned with business and domain functionality, and which can be independently deployed
- Each microservice will have its own database, avoiding the need for data sharing and tight coupling; when needed, data sharing between microservices is accomplished through asynchronous messaging and event sourcing avoiding the need for foreign keys and cumbersome SQL JOINs
- Decomposition into granular services makes it easier to upgrade, maintain, and modify services whenever there is a business need, and not the entire module or application
NoSQL
While traditional relational databases such as MySQL met the needs of structured data, they are not well-suited to handle unstructured data, multiple datatypes, and relations between data. Over the past few years, popular NoSQL databases such as MongoDB, Apache Cassandra, and Redis have emerged as powerful alternatives, and offer the following benefits over relational databases:
- NoSQL databases allow you to stuff data into a database without defining a formal storage structure ahead of time
- NoSQL databases don’t need rigid schema, making them lightweight
- NoSQL databases are elastic and scale better, as they are designed to run on distributed and clustered environments
Event Sourcing (ES) and CQRS to overcome the limitations of CRUD
In the traditional Create, Read, Update, and Delete (CRUD) model of data persistence, typical operations such as reading data from the store, making modifications, and updating the current state of the data with new values, are done through transactions that lock the data. In a modern context with huge data volumes, this approach has the following limitations:
- In collaborative environments with lots of concurrent users, data conflicts are likely as the update operations are performed on a single data item.
- In the absence of a data log, information on changes made i.e. history is lost.
- NoSQL databases are elastic and scale better, as they are designed to run on distributed and clustered environments
In the past few years, Command Query Response Segregation (CQRS) pattern, in combination with Event Sourcing (ES), has evolved as a powerful architectural paradigm to overcome the limitations of CRUD, and offers the following advantages:
|
|
Goal-oriented UI/UX frameworks
Perhaps the biggest paradigm shift is in our approach to UI/UX design. Most legacy applications were designed for a specified, pre-defined input/output pattern, and very often the presentation layer is tightly integrated with business logic, making it very difficult to change. Traditional UI/UX is also mostly CRUD-driven with mass input and output. Now the current generation of applications demand goal-oriented UI/UX based on roles, tasks, state, and must be responsive to support a wide range of devices ranging from desktops to handhelds to mobile and wearables. At the same time, in addition to the standard keyboard-based input, they must also support user interactions from multiple sources including speech, vision, gestures, and touch. Some of the popular UI/UX responsive frameworks used for creating modern, digital applications include Bootstrap, Foundation, Skeleton, CreateJS etc.
News you can use
The Cambridge Analytica affair reveals Facebook’s “Transparency Paradox”
Facebook is facing a transparency paradox, and is under tremendous pressure to be transparent and reveal how targeted advertising works, and how its News Feed and trending algorithms work. On one hand, there’s this strong pressure to be more transparent and share data with trusted third parties, and on the other hand, there’s equally strong pressure to increase security of the data that is revealed or shared and to protect users’ privacy.
GDPR is coming. Are you ready?
GDPR is applicable to any company in the world that does business in Europe, and if they gather personal data of EU residents. GDPR applies to personal data relating to EU residents regardless of where that data is processed, and also defines the scope of EU data protection legislation. GDPR also gives regulators serious teeth — compliance costs and fines can reach up to €20 million or four percent of global annual revenue for the preceding financial year, whichever is greater.
UK lawmakers warn against tech giants dominating AI industry
An influential body of UK lawmakers said regulators should stop major tech companies from dominating the field of artificial intelligence, and also warned on the potential of widespread unemployment due the technology.
India’s opportunity and role in shaping the Fourth Industrial Revolution
With the right mix of accelerators – including regulatory frameworks, educational ecosystems and government incentives – India can lead the Fourth Industrial Revolution while simultaneously enhancing the quality, equity and sustainability of its own growth and development outcomes.
Google Announces Dutch Data Center Expansion
Google announced a €500 million expansion to its data center in Eemshaven, the Netherlands, and stated that this investment will help Google meet growing European consumer demand.
New Strategy to make Denmark the New Digital Frontrunner
With The Strategy for Denmark’s Digital Growth, government aims to create the best foundation for Danish companies to exploit new technologies to generate growth and more wealth for all Danes.
$15m is available to solve a burning problem
To claim a share of the $15m X-Prize, winners will have to turn carbon dioxide extracted from power-plant flues into something useful—and do so profitably.
10 Breakthrough Technologies 2018
Dueling neural networks, Artificial embryos, AI in the cloud…. read MIT’s annual list of 10 technology advances that will shape the way we work and live now and for years to come.
Analyst Corner
Big Data and Business Analytics (BDA) Revenues in the Asia/Pacific will see significant growth in 2018
A new update to the Worldwide Semiannual Big Data and Analytics Spending Guide from IDC forecasts, revenues for big data and analytics (BDA) in the Asia/Pacific (excluding Japan) will reach $14.7 billion in 2018, an increase of 14.4% over 2017.
Worldwide Public Cloud Revenue to Grow 21.4 Percent in 2018: Gartner
Gartner forecasts, that the worldwide public cloud services market will grow by 21.4 percent in 2018 to total $186.4 billion, up from $153.5 billion in 2017. Gartner says that the fastest-growing segment of the market will be cloud system infrastructure services (Infrastructure as a service or IaaS).
The smart home will advance a subscription economy: Forrester
Forrester believes smart home companies will pursue new forms of bundled services that combine monthly subscriptions with smart home products included or discounted.
coMakeIT Blog Digest
Data breach is the new oil spill
2018 witnessed a major spill, this time it is not oil, but data. Just as oil was the vital and essential commodity that powered the manufacturing era, data is the indispensable fuel powering the connected, digital economy.
Collaborative Innovation in the Platform Economy
Collaborative innovation is both a business and technology needin the platform economy, but ISVs and digital businesses must retain strategic control over core IP, as it is a key, competitive differentiator.
Legacy, once a burden, is now a new opportunity to transform
Modernization of legacy software applications cannot be achieved with a uni-dimensional approach. Modernization has multiple dimensions, and one must choose the right strategy based on the current state of the software, competitive drivers, and broad business context.
Nurturing Talent through Trust and attaining Team Nirvana
Trust and Talent aligned well with Individuals and Teams can work wonders in building high-performance teams, and will lead to a true Team Nirvana.
Customer Buzz
Keylane expands its customer footprint across the Nordics
Keylane, which strengthened its solution suite with the acquisition of Schantz in 2017, is fast expanding its customer footprint across the Nordics. Recently, Møretrygd, a Norwegian non-life insurance company, has selected Keylane’s policy and claims management system. The Danish pension fund ISP (Pension Fund for the Danish Society of Engineers) has entered into a new agreement for upgrading to the latest version of Schantz Life, which is now a part of Keylane pension suite.
MPO appoints Buck Devashish as CIO
MPO strengthened its executive management team with the appointment of Buck Devashish as Chief Innovation Officer. Buck will be based in MPO’s Boston office, and will be responsible for Global Business Development, Sales and Marketing. Buck has an impressive track record of business development across multiple companies including Infosys, Yantra, GT Nexus and Infor.
Planon acquires CAFM software conjectFM
In an effort to strengthen its position in the fast-growing Central European market, Planon acquired CAFM (Computer Aided Facility Management) software conjectFM from Aconex, including all employees, customers and software products. Pierre Guelen, Founder and CEO of Planon, said, “this acquisition supports Planon’s ambitious global growth strategy and will accelerate the pace of innovation in this disruptive market.”
CCS helps UMG to GoLive with CED Connect
CCS helped UMG (Unirobe Meeùs Group), one of the largest intermediaries in the Dutch vehicle insurance market Go Live with CED Connect, an advanced platform for digital claims handling, developed by CED, a leading third-party service provider. This rollout was enabled through a collaborative effort between CCS, UMG and CED.
Chevin wins ‘Innovation in Mobile Comms’ award
Chevin’s new mobile app, FleetWave Forms, won an award in the Innovation in Mobile Commscategory, in the recently held FleetWorld Honours 2018. This award recognised FleetWave Forms’ unique customizable capabilities, and its ability to tackle more than just day-to-day driver duties.
News about coMakeIT
Thijs Petter joins as CTO
coMakeIT strengthened its executive management team through the addition of Thijs Petter as its new CTO. Thijs is an accomplished technology expert with an impressive track record, and is brought on board to help customers leverage exponential technologies, and drive innovation.
Launches Technology Innovation Center (TIC): coMakeIT announced the launch of its Technology Innovation Center (TIC), which will bridge the gap between the potential of emerging technologies and their practical usage in a complex business and regulatory environment. The TIC will continuously evaluate disruptive technologies and bring them into practice through execution of PoCs, partnerships with technology vendors, trainings, and participation in beta programs.
10-yr club: In this past quarter, two more colleagues have reached the 10-yr milestone with coMakeIT. Sashikanth, a key member of the executive leadership team, and Ganesh, an architect with Keylane team were recognized for their longevity, for their contributions to the growth of coMakeIT, and enabling the success of numerous customers over the years.
Inspiration session with MPO: MPO, a coMakeIT customer and fast-growing provider of innovative software solutions to the logistics industry, hosted an inspiration session on March 7th, at their Rotterdam offices on the theme: co-Innovation with agile teams.Many technology companies that were interested in exploring offshoring and collaborative innovation participated in the session. Paul van Dongen (co-founder and CTO), and Ronald van den Heuvel (VP of software) of MPO, shared their experiences and best practices in working with distributed development teams.
Wall-of-Art: This initiative is a big hit and very popular with all our employees. In the previous quarter, three rounds of competition were held for the wall-of-art, which saw very enthusiastic and high-quality participation. The art works and the winners for various rounds can be seen here.
TaRAnG Activities: Our employee engagement and cultural club TaRanG, conducted photography, karaoke, poetry, and signing competitions, which witnessed widespread and enthusiastic participation.
Women in Tech: coMakeIT marked the International Women’s Day by recognizing significant contributions by women in diverse roles. To celebrate diversity of their contributions, women in various roles were invited to share their experiences through a series of blogs.