Perhaps no other technology has captured the human imagination like AI (Artificial Intelligence), and over the past decade AI has enjoyed both rapid progress in its capabilities and explosive growth in terms of its adoption. Big technology companies are enjoying commercial success by incorporating AI into their products (e.g., Google, Microsoft) and others are using it to improve operational efficiency (e.g., Amazon, Netflix, ByteDance / Tik Tok). McKinsey is estimating that AI will add $13 trillion to the global economy in the next decade.
Yet, the industry as a whole is struggling with AI as illustrated by the data points below:
This article illustrates - Why the discrepancies exist? and what approaches can be adopted by companies to ensure that their AI efforts are successful?
How are companies approaching AI?
Companies have the opportunity to use AI to significantly enhance their Products and/or to improve their Operational Efficiency. In both cases having AI and Domain Expertise is critical to success. However, there is a fight for Machine Learning talent, and it is not uncommon for this talent to land $250,000 / year salaries. Many companies are hence struggling to recruit and retain the necessary talent.
These organizations are attempting to inject AI expertise into their organizations, and scale their AI efforts; and are doing so in one of two ways.
Build a generic Centralized AI Team. This approach often suffers from the lack of deep domain expertise.
Build multiple Domain Specific AI Teams. In this approach recruitment and retention is a bigger challenge.
Outsourcing / staff augmentation tends to be expensive and does not offer a long-term solution. Many companies' needs can be addressed with Commercial-off-the-shelf (COTS) tools, and AutoML, Low Code and No Code solutions will continue to have gaps.
Furthermore, companies when pursuing AI transformation / initiatives are often taking a Waterfall approach (A sequential approach to delivering projects). They generally try to invest in Big Data first, then try to build AI Models and finally address Deployment and Operations. Millions of dollars (often 10s millions dollars) are spent (or wasted) before anything is deployed (often with suboptimal results).
Monoliths vs. Microservices
Enterprises have been shifting from building Monoliths to SOA (Service Oriented Architecture) and Microservices over the past decade. The difference between microservices and monolithic architecture is that microservices compose a single application from many smaller, loosely coupled services as opposed to the monolithic approach of a large, tightly coupled application. The difference between microservices and SOA is more subtle, where SOA focuses more on the integration layer whereas microservices focuses more on services.
Microservices also have a symbiotic relationship with domain-driven design (DDD)—a design approach where the business domain is carefully modeled in software and evolved over time, independently of the plumbing that makes the system work.
Semantic Brain's BizML aligns with microservices architecture in that it enables Finance & Marketing professionals to carefully model business domain into smaller AI models. BizML process and templates generally encourage the users to break the overall problem into smaller problems and utilize ensemble models to make the final prediction. Much like microservices reuse of AI Models (deployed as microservices) is possible and encouraged.
Furthermore BizML in Enterprises accelerates the process of building, deploying and managing entire lifecycle of 100s microservices. This approach enables Enterprises to rapidly increase their Intelligence while facilitating the adoption innovative business models (e.g. use of crypto).
Note: Data Scientists in above scenarios will continue to play a key role. However, they will be in a better position to deliver better results and scale their services cost effectively.
Waterfall vs. Agile
Agile delivery is an iterative approach to software delivery in which teams build software incrementally at the beginning of a project rather than ship it at once upon completion. Agile development means taking iterative, incremental, and lean approaches to streamline and accelerate the delivery of projects (Source: GitLab)
While most in software delivery have already embraced Agile, AI / Data Science sector is just starting to make the transition (many AI projects follow R&D process) to more agile processes such as Microsoft TDSP. BizML is promising to accelerate the adoption of agile in AI by composing Microservices and enable ongoing Domain Expert interaction.
Note: BizML capabilities have been captured in the diagram on the right and have been colour coded as grey.
BizML Success & Value Proposition
Semantic Brain has used BizML Framework Process / Application to develop a few high performing (i.e. Accuracy, Precision, Recall, F1-Score, AUC) AI services. One of the best examples of BizML success is its performance wrt stock price prediction.
BizML was used to train and compose AI services that can be used to predict RUSSELL 1000 component (i.e. stock) price movement over 5 trading days. Its predictions over a 5 month window are captured in the scatter plot below. Upon observation and analysis it is clear that ~70% of the stocks that AI predicted to go up indeed moved up. It is also evident that the magnitude of change in the direction predicted is greater when compared to the opposite direction.
Subsequently the same AI was used to make Long and Short trade recommendations on RUSSELL 1000 components. A simple Long and Short trading strategy with 5% stop loss created a portfolio which had much higher returns and lower volatility when compared to S&P 500, RUSSELL 1000 and RUSSELL 2000.
Conclusion
Traditionally success in AI has required a combination of Deep Domain and Data Science Expertise. This in conjunction with Waterfall thinking has necessitated the need for significant investment, and has often led project failure.
BizML is a game changer. It enables Business Professionals to build high performing AI Microservices with less Data Scientist involvement. Hence, companies can still succeed implementing AI with smaller / outsourced Data Science teams while growing their business know -how . Other details are below.
| AI delivery with BizML | Contemporary AI delivery |
Approach | Business Professionals lead AI Initiatives and starts them by •Performing extensive Exploratory Data Analysis •Choosing appropriate Financial / Marketing Templates •Deriving and optimizing AI input Simpler environment with high levels of automation Agile with Microservices |
Benefits
1) Rapid delivery in smaller chunks, 2) much higher project success rates, 3) 5% to 20% increase in accuracy, 4) 50% reduction in development time, and 5) 70% reduction in compute
Operations
BizML is an opinionated solution where customers can focus more on solving Business Problems and less on Technology and Operations
Risks
Reduce 1) Data Science team building risk, 2) Business Professional / Data Scientist attrition risk, and 3) delivery risk
Reference
Introducing BizML (Business Machine Learning Optimizer) - https://www.semanticbrain.net/post/introducing-bizml-business-machine-learning-optimizer
Everyone's fighting to hire Machine Learning Engineers - https://www.businessinsider.com/how-to-land-job-as-machine-learning-engineer-250000-salary-2022-2