Earlier this week, Google unveiled its new Gemini artificial intelligence (AI) model, and it’s protected to say the software completely wowed the tech world. That was partially on account of a formidable “hands on” video demo (under) that Google shared, but it’s now emerged that each one was not because it appeared.
According to Bloomberg, Google modified interactions with Gemini in quite a few methods with a purpose to create the demonstration. It raises questions over the chatbot’s skills, in addition to how a lot Google has been capable of meet up with rival OpenAI and its personal ChatGPT product.
For example, the video’s YouTube description explains that “for the needs of this demo, latency has been decreased and Gemini outputs have been shortened for brevity.” In different phrases, it in all probability takes so much longer for Gemini to reply to queries than the demo recommended.
And even these queries have come below scrutiny. It seems that the demo “wasn’t carried out in actual time or in voice,” says the Bloomberg report. As an alternative, the true demo was constructed from “nonetheless picture frames from the footage, and prompting by way of textual content.”
Which means that Gemini wasn’t responding to real-world prompts shortly in actual time – it was merely figuring out what was being proven in nonetheless photographs. To painting it as a easy, flowing dialog (as Google did) feels a bit deceptive.
A protracted method to go
That’s not all. Google claimed that Gemini might outdo the rival GPT-4 mannequin in virtually each check the 2 instruments took. But wanting on the numbers, Gemini is barely forward by a number of share factors in lots of benchmarks – regardless of GPT-4 being out for nearly a yr. That implies Gemini has solely simply caught as much as OpenAI’s product, and issues would possibly look very totally different subsequent yr or when GPT-5 in the end comes out.
It doesn’t take a lot to seek out different indicators of discontent with Gemini Professional, which is the model at the moment powering Google Bard. Customers on X (previously Twitter) have proven that it’s vulnerable to most of the acquainted “hallucinations” that different chatbots have skilled. For example, one user asked Gemini to inform them a six-letter phrase in French. As an alternative, Gemini confidently produced a five-letter phrase, considerably confirming the rumors from earlier than Gemini launched that Google’s AI struggled with non-English languages.
Different customers have expressed frustration with Gemini’s inability to create accurate code and its reluctance to summarise sensitive news topics. Even easy duties – reminiscent of naming the most recent Oscar winners – resulted in flat-out mistaken responses.
This all means that, for now, Gemini could fall in need of the lofty expectations created by Google’s slick demo, and is a well timed reminder to not belief all the pieces you see in a demo video. It additionally implies that Google nonetheless has an extended method to go to meet up with OpenAI, regardless of the large assets on the firm’s disposal.
Discussion about this post