Advertisement

In recent years, language models have taken the field of natural language processing by storm, but even the most advanced ones have their limitations. They struggle with basic tasks such as arithmetic and fact-checking. However, a new AI language model named Toolformer, developed by researchers at Meta, promises to change that.

Machine Learning

Toolformer is different from other language models, such as ChatGPT, since it can essentially teach itself to use external tools such as search engines, calculators, and calendars. This is possible thanks to the use of APIs, short for Application Programming Interfaces. Toolformer can use APIs to access information from a variety of external software tools, including language translators and factual references.

During training, Toolformer was given a small set of human-written examples that demonstrated how each API is utilized, and it was then allowed to annotate a large language modeling dataset. It did this in a “self-supervised” way. The model learned to predict each text-based API call as if they were any other form of text. When in operation, it can insert the calls as required and even self-sufficiently decide which tool to use (and how to use it).

Toolformer’s API-calling ability enables it to work around limitations in other language models, such as arithmetic. For example, if a user wants an LLM-based assistant to add a date to their calendar, Toolformer could handle that task by using an API link to a calendar app.

Toolformer is built on a pre-trained GPT-J model with 6.7 billion parameters. Various experiments conducted on a variety of tool-oriented concluded that Toolformer is a far better performer than the much larger GPT-3 model. It is worth noting that the GPT-3 model has more than 175 billion parameters.

While integrating external tools into language models is not a new concept, most existing approaches have relied on large amounts of human annotations or have been limited to specific task-specific settings. Toolformer, on the other hand, can learn to use a range of tools in a generalized way that does not require specialized training for specific tasks. However, there will always be a small risk that such language models might accidentally invoke the wrong tools or cause harm to user data. Nevertheless, Toolformer is a promising step in the right direction for the field of natural language processing.

RELATED:

(Via)

Comments

RELATED ARTICLESMORE FROM AUTHOR