Last updated on October 9th, 2023 at 05:31 pm
INTRODUCTION
OpenAI’s ChatGPT has introduced plugins that allow the language model to access current information, perform computations, and use third-party services, while prioritizing safety. The implementation of plugins is being gradually rolled out to study their real-world use, impact, safety, and alignment challenges.
Developers invited by OpenAI can use documentation to create plugins for ChatGPT, and the first plugins have been created by companies such as Expedia, Slack, and Wolfram.
OpenAI’s goal is to create a community that shapes the future of human-AI interaction by enabling a wide range of use cases.
Chat Plugins developed by OpenAI
OpenAI has expanded the capabilities of its ChatGPT language model through the incorporation of various plugins, such as a web browser and code interpreter. Furthermore, OpenAI has made the knowledge base retrieval plugin’s code open-source, enabling developers to integrate it into their own projects.
These plugins enhance ChatGPT’s functionality and increase its potential, which may have implications for other companies building on GPT-3. Below is a brief summary of each plugin.
Browser Plugin:
A browsing plugin has been developed for language models like ChatGPT, allowing them to access fresh information from the internet and expand their range of discussion. The plugin is designed with safety in mind, and only uses GET requests via the Bing search API.It respects websites’ robots.txt files and cites sources to give credit to content creators. The plugin is an additive way to retrieve information and excludes transactional operations to reduce safety risks.
Code Interpreter Plugin:
It is an experimental ChatGPT model that uses Python and enables uploads and downloads. The model has a Python interpreter in a secure execution environment and allows persistent sessions for code run during a chat conversation.
It can be used for various tasks such as solving mathematical problems, data analysis and visualization, and file format conversion. Safety is a primary concern, so the AI-generated code is sandboxed and resource limits are set on each session. The code sandbox’s functionality is limited by disabling internet access.
The Retrieval plugin for ChatGPT:
It allows users to search personal or organizational information sources for relevant document snippets using natural language queries. The open-source plugin can be deployed by developers, who can choose a vector database for indexing and searching documents. Developers should ensure that only authorized content is added to the plugin.
Third-party plugins can be enabled by users in conversations on chat.openai.com, and are described by a manifest file with metadata and documentation. Plugins currently call backend APIs, but there are plans to explore client-side APIs.
Access to these plugins will be gradually enabled for ChatGPT users, starting with ChatGPT Plus subscribers and a small number of developers from their waitlist.
Third Party ChatGPT Plugins
Here is a brief summary of Third Party ChatGPT Plugins (Chat Plugins)
1. Expedia: Turn your travel plans into reality by finding transportation, accommodations, and things to do.
2. FiscalNote: Provides access to real-time data sets for legal, political, and regulatory information.
3. Instacart: It allows you to easily order from your preferred nearby grocery stores.
4. KAYAK: Helps you search for flights, hotels, and rental cars, and provides recommendations based on your budget.
5. Klarna Shopping: Lets you search and compare prices from thousands of online stores.
6. Milo Family AI: Gives parents tools to turn everyday moments into special memories, with daily prompts and suggestions from the Hey Milo voice assistant.
7. OpenTable: Provides restaurant recommendations and an easy way to make reservations.
8. Shop: Lets you search for millions of products from top brands.
9. Speak: Helps you learn how to say anything in another language with an AI-powered language tutor.
10. Wolfram: Gives you access to computational tools, math resources, curated knowledge, and real-time data through Wolfram Alpha and Wolfram Language.
11. Zapier: Helps you connect and automate workflows across over 5K apps such as Google Sheets, Trello, Gmail, HubSpot, Salesforce, and more.
How To Build Chat Plugins?
The end-to-end flow of building a plugin involves creating a manifest file, registering it in the ChatGPT UI, and manually activating it in the UI.
Developers need to expose one or more API endpoints, a standardized manifest file, and an OpenAPI specification to define the plugin’s functionality. ChatGPT consumes these files and makes calls to the developer-defined APIs.
Once a user asks a relevant question, the model may invoke an API call from the plugin, and the model will incorporate the API results into its response to the user.
Currently, the system sends the user’s country and state in the Plugin conversation header, but users will have to opt-in via a consent screen for further data sources.
At present, Chat Plugins are in a limited alpha stage and may not be accessible to all users. To know more about building plugins that can allow ChatGPT to make intelligent calls to third-party applications’ APIs, please check out.
Advantages and Limitations of ChatGPT Plugins (Chat Plugins)
Plugins can help overcome limitations of language models by providing access to up-to-date or personalized information not included in the training data, as well as enabling the model to perform safe and constrained actions.
However, connecting language models to external tools introduces new risks, which OpenAI is working to mitigate through implementing safeguards.
Despite the risks, plugins have the potential to address current limitations and enable new use cases for language models, and OpenAI plans to expand the plugin platform beyond ChatGPT to other OpenAI models in the future.
OpenAI has prioritized safety in the development of its plugin platform and has taken various measures to ensure it. Red-teaming exercises with external collaborators have been conducted to identify potential concerns that could arise from the use of plugins without proper safeguards.
Harmful actions that plugins could perform include injecting prompts, sending fraudulent emails, bypassing safety restrictions, and misusing information.
The team is using these findings to develop safety measures that will limit risky plugin behaviors and increase transparency in their usage. To ensure safe deployment of plugins, access is being granted gradually.
Researchers are invited to use OpenAI’s Researcher Access Program to study safety risks and mitigations related to plugins, and developers and researchers can submit evaluations on plugin safety and capabilities through OpenAI’s Evals framework.
CONCLUSION
OpenAI’s plugin support for ChatGPT is a true game-changer, linking the powerful language model to third-party applications and enabling it to achieve even greater feats. With the help of these plugins, ChatGPT can interact with APIs defined by developers, unlocking a world of possibilities and making it capable of performing an array of actions that extend far beyond the realm of natural language processing.
Among the many exciting capabilities of these plugins, ChatGPT can retrieve real-time information, including sports scores, stock prices, and the latest news. It can also access knowledge-base information, such as company documents and personal notes, and even perform actions on behalf of the user, like booking a flight or ordering food. The possibilities are endless, and the future of ChatGPT with these plugins is boundless.
With the promise of rolling out access to more users, we can expect Chat GPT to become even more versatile and competitive in the coming months. source
Navigate the AI landscape and find new inspiration with these carefully selected articles that resonate with your interests: ChatGPT, GPT-3, InstructGPT, ChatGPT Whisper API, GPT-3 Vs InstructGPT, GPT-4, ChatGPT Function Calling
