Today, Google made the announcement that it will be launching PaLM 2, its most recent large language model (LLM), at its annual I/O developer conference. PaLM 2 will be used to power Google’s improved Bard chat tool, which is the company’s competitor to OpenAI’s ChatGPT.
PaLM 2 will also operate as the foundation model for the majority of the new AI features that the company is releasing today. PaLM 2 is currently accessible to developers through Google’s PaLM API, Firebase, and Colab.
In a manner similar to that of OpenAI, Google did not publish a significant amount of technical data regarding how it trained this next-generation model, including counts of the model’s parameters (for what it’s worth, PaLM 1 is a 540-billion parameter model).
PaLM 2 was constructed using Google’s most recent JAX and TPU v4 architecture, which is the only piece of technical information that Google has disclosed regarding this product.
“What we found in our work is that it’s not really the sort of size of model — that the larger is not always better,” During a press event held in advance of today’s announcement, DeepMind Vice President Zoubin Ghahramani made the following statement.
“That’s why we’ve provided a family of models of different sizes. We think that actually parameter count is not really a useful way of thinking about the capabilities of models and capabilities are really to be judged by people using the models and finding out whether they’re useful in the tests that they try to achieve with these models.”
Instead, the corporation made the decision to concentrate on its strengths and competencies. According to Google, the new model is superior in the areas of mathematics, logic, and common sense thinking.
In point of fact, as Ghahramani mentioned, the business that developed the model educated it using a vast quantity of mathematical and scientific writings, as well as mathematical expressions.
It’s no secret that huge language models, with their focus on language, have had trouble dealing with arithmetic concerns without relying on third-party plugins.
In fact, it’s a problem that’s been going on for years. On the other hand, Google contends that PaLM 2 is able to quickly answer mathematical conundrums, reason through issues, and even produce diagrams.
In addition to this, PaLM 2 also provides enhanced assistance while writing and debugging code. The training for the model was done on twenty different programming languages, including well-known ones such as JavaScript and Python, as well as less common ones such as Prolog, Verilog, and Fortran.
PaLM 2 serves as the foundation for Google’s coding and debugging paradigm known as Codey, which the company is also unveiling today as part of its code generation and completion service, among other things.
PaLM 2 was trained on a corpus that contains over one hundred different languages, which, according to Google, allows it to “excel at multilingual tasks,” including the use of more subtle phrasing than earlier models. This was another point that Google highlighted today.
Google refers to PaLM as a family of models, which includes not only Codey but also Med-PaLM 2, the company’s model that is centered on medical knowledge. This family of models is known as PaLM. In addition, there is a version of PaLM called Sec-PaLM that is centered on security use cases.
There is also a smaller PaLM 2 model that can run on smartphones, which has the potential to open up PaLM to use cases that are more focused on privacy. However, Google has not committed to a timetable for the implementation of this feature.
This model, according to Google, is capable of processing 20 tokens per second, which isn’t very quick but maybe just fast enough for specific use cases (Google would not disclose which phone it tested this model on, however).
It is not a secret that Google has launched these AI features using a very planned manner, which is something that the corporation acknowledges.
On the other hand, the normal response that Google’s representatives provide when asked about this topic is that the business aims to develop these technologies in a responsible manner while keeping safety in mind.
Not only that, but that is also what the corporation has stated regarding PaLM. Because we weren’t able to evaluate it in advance of today’s announcement, it goes without saying that we have no idea how well it works or how it deals with edge scenarios.