AI Model Orchestration with Wity.AI (Part 1)
Key Problem :
There are numerous AI models, and new ones are emerging every day.
With the rapid proliferation of AI models -
how do we create a utilisation framework for effectively and quickly harnessing multiple diverse models or any individual model over sequences of prompts, towards our various requirements and workflows?
How Wity addresses this:
1.) Model Selection:
Dynamically selecting appropriate model(s) based on given inputs, and desired outcomes.
Whether crafting a book, designing a system, or creating a tutorial, multiple models are dynamically selected throughout the sequence of prompting.
2.) Model Orchestration:
Abstracting the generation or enhancement, management & execution of prompts with selected models, & consolidation of outputs from diverse models.
See live example in the Usage Abstractions section below
2.1.) Prompting Strategies:
- Automatic Prompting (Chain of Prompts that are automatically generated or used Unsupervised based on input context & desired outcomes)
- Automatic Chain of Thought Agents/ Prompt-Chains
Automated agents that guide one or multiple models through a logical chain of thoughts or steps. - Automatic Contextual Prompting
- Automatic Chain of Thought Agents/ Prompt-Chains
- Interactive Prompting (Prompts that are generated in-loop with User Interaction)
- Adaptive User-Driven Prompting
- Context enhanced User Prompts
2.2.) Consolidation of Model Outputs:
Consolidation and integration of outputs from different AI models over time into a unified comput-able & query-able structure like a response graph or tree.
- Unified Representation as Knowledge Graph/ Tree of Thoughts:
- Enhanced Queryability & Prompting
Context & Consistency based Self Adaptive Prompting. - Knowledge Interoperability
Interoperability of Unified Knowledge Graph with other markup formats like OPML facilitating further processing, integration, & sharing across different platforms
- Enhanced Queryability & Prompting
3.) Composition of Content
Flexible transformation of Knowledge-graph to Multi-Modal Content.
- Mixing, Generating & Rendering Multi-modal Content
Wity can render knowledge graphs into rich, multi-modal HTML content, incorporating images, videos, audio, text, graphics, and more, arranged within a unified HTML layout.
Below is an Example Output -
Click to Open (No Login Required)
Usage Abstractions
1.) Wity Agents :
An Agent in Wity is an executable process template to automate a particular workflow - through model selection, orchestration & consolidation of output
- Custom Agents can be created using a json-based DSL. Also called dialectics template in wity. There is a UI in the app that simplifies this.
- Agents are triggered from user inputs or programmatic events.
Below is a Live Agent that you can Run, that employs automating prompting strategy to generate & consolidate magazine content as a knowledge graph, based on your input -
Click to Run this Wity Agent (No Login Required)
2.) Composers :
A Composer in Wity is a set of markups & rules to automate rendering, generation & mixing of multi-modal content from an input json structure or internally from wity's knowledge graph structure.
- Custom Composers can be created using the UI. To know more or enable more advanced usage - you can reach out to support@wity.ai.
FAQs:
1. Can new models be flexibly added into the orchestration system?
New models can be flexibly added to the orchestration system at runtime without disrupting existing workflows.
2. Can we use our own custom models in the orchestration system?
Yes, there is provision to add platform-hosted custom models, to utilize your own AI capabilities. Please contact support@wity.ai for more info on this.
3. Can we connect our OpenAI account or other platforms to the orchestration system?
Yes, You can connect API keys from platforms like OpenAI, Claude, Replicate, and Hugging Face, enabling seamless integration with these services. You can find this inside usage preferences tab in your account section in the app.
