Business Problem
A text modeling project requires significant amounts of labor to code, troubleshoot, and test before a model is placed into production. Once in production, the models work to create tables, and the tables are analyzed to produce value. Over time the model results drift, and the models may be adjusted to improve results. This process is similar to other software projects and processes.
With that first project running successfully, customers faced a new problem – the second project. Unlike other software projects, however, much of the code from a project code was unusable.
With significant amounts of time, labor, and testing resources invested, users did what any other rationalizing human would do – fall straight into the “sunk cost fallacy” trap by trying to reuse their previous work in a new project.
The sunk cost fallacy is a human cognitive bias where people refuse to cut their losses because they have “too much invested” in an effort. They refuse to cut ties and move on. Instead, they work to recover some portion of their past investments and reuse it towards the future. It’s understandable, we all do it. Cars, homes, marriages, careers, etc.
When project work was recovered, it often didn’t produce the same output as the original did. If text modeling rules were math, the rule “2+2=x” would result in “4” in the first project, and “4.2” in the second. An incorrect result. Troubleshooting the problems incurred more sunk cost.
System Analyses
Product A Use Process
These problems were bad enough. However, the Sales professionals who sold the product encountered a different problem. The product portfolio included two text modeling products, and both had similar, overlapping features. Product selection was not clear, and customers simply moved on to other vendors rather than entertain explanations or multiple pitches. The Sales organization lost opportunities due to the confusion.
After losing several engagements, the Sales organization strongly recommended that both products consolidate. Development agreed, and I was asked to investigate both products to determine how best to design and integrate them. The products had separate architectures and architectural changes would provide an opportunity to address several longstanding issues. User research efforts provided some ideas.
Product B Use Process
Product A/B Gap Analysis
Product A/B Task Environment Comparison
User Research
Customer issue logs, subject matter expert notes, industry white papers, and known application usability issues were evaluated to understand issues and develop a solution strategy.
Users had long complained of dogged performance while compiling projects. Subject matter experts were also frustrated with this issue, along with a number of other issues detailed in other project examples.
Industry white papers discussed other problems common to text modeling applications. Businesses experienced limited benefits due to the time and labor required to build useful, mature development platforms. Maturing the product platform is quite difficult when you cannot standardize and reuse previous work.
Businesses also failed to invest in maturing their products due to how they approached projects. When undertaking a project, businesses hired specialized staff to develop platform resources based on that specific project. When the projects were completed, these resources were released. Lacking technical expertise, development languished and the platform ceased growth and maturation.
Several customers of this product did invest resources, however, but the product design did not match the organizational work structures. This product was designed for a single seat user, but companies developed projects in collaborative teams.
To overcome the collaboration issue, the Development team created a special “collaboration” engine which mashed the work from single seat users into a “collaborative” monolithic taxonomy. When mashed together, project performance dropped significantly, and users experienced the “2 + 2” now equals “4.2” problem described above. The product was a macro-ergonomic failure.
What if analytical projects were built differently? Could changes in how projects were constructed improve usability and increase platform development and maturity? This question lead to other questions, asked during user interview sessions. During these sessions, users were asked to provide their opinions about:
- Common project assets built from discrete blocks
- Centralized asset repositories
- Projects built using a common asset block process
- How this architecture would change their process
Users explained their processes, and provided overwhelmingly support for this type of change. Many were frustrated about the current product and shared the workarounds they created to accomplish the same idea. These were hacks that users of expensive Enterprise software should not be resorting to.
The user feedback was consolidated and reported to Development stakeholders. Development was intrigued, and began investigating the technical feasibility of these ideas.
Design Concepts
The following examples illustrate how a new architecture which supported common asset blocks could refine product workflows. Using this structure, the product would have an Initialization Phase, followed by successive levels of maturity. Each maturity level reduces the amount of labor and resources required to build a project.
Initial Phase
The Initial Phase begins much like the old product; no immediate value is gained in this stage. Value was gained as the system was matured and built out. The purchase of pre-built modules could speed maturity and add new revenue sources. Contracted resources or collaborative teams are most useful in this phase.
Adolescent Phase
As projects are completed, the base component library is populated. Optional modules may also be purchased to populate the library more quickly. Collaborative teams may continue building, or shift to other priorities.
Mature Phase
Over time, the base libraries and other libraries mature, reducing project building to a drag and drop process requiring little modification or rework. Model management resource needs may decrease. Using the readily available components could provide organizations with the ability to perform lean activities such as rapid experimentation and quick hypothesis testing. As I presented these concepts, stakeholders realized additional benefits:
- Code tracking. Model management staff could now track model construction.
- Code versioning. Model code requires maintenance. Discrete blocs can be tested separately and iterated upon to improve performance. New block versions can be added, tested and refined while the model is in production. Once developed, the new block versions can be added or retracted per organizational needs.
- Performance tracking. Model performance “drifts” over time. Tracking the performance of discrete pieces across models across time helps with detecting and managing drift. Perhaps different versions perform differently over time. This adds a rapid experiementation component and other maintenance improvements.
- Pre-built discrete block or custom block development services. New consulting and revenue streams become available.
- Improved product performance. Models only need what’s necessary to accomplish the job. Discrete blocks should eliminate the spaghetti code reference problems. Each block does a simple job, and blocks are added and removed as necessary.
- Consistent results. If code blocks are discrete and perform simple independent functions, then the results should remain consistent; “2 + 2 = 4” wherever the block is used.
Outcome
As the scheduled stories were nearing implementation, a high-level reorganization occurred. The Development Manager who was favorable to this direction was replaced.
This new Development Manager was far more risk adverse, opting to continue to rebuild the legacy product within the current architecture.
Nothing changed and legacy product replication continued.
Related Examples
Team Collaboration Process
Development conceived this product being used by a single person. Research, however, revealed something different. This story describes the collaboration process companies use to extract value from this product.
Taxonomy Redesign
Taxonomies were a major component of this application and a source of many problems. Users complained of poor performance, loss of orientation, and coding errors attributed to taxonomies. This story summarizes multiple redesign efforts working to fix the problems.