The problem with testing AI chatbot subscriptions like Google’s Gemini Advanced and OpenAI’s ChatGPT Plus is their generality. The same tool is used for disparate applications; the same software service that developers in San Francisco are using to build their latest app might also be used by parents in Kansas to plan a Paw Patrol birthday. Even though companies often tout esoteric benchmarks to prove their chatbot’s superiority, it can be hard to discern how a chatbot’s technical prowess translates into a better experience for you, the user.
Google is the latest company to offer one of its best AI chatbots as a subscription product. In early February, the company began offering access to Gemini Advanced for $20 a month. In doing so, Google was following the precedent set by OpenAI, which sells access to its GPT-4-powered chatbot for $20 a month. Additionally, Microsoft sells subscriptions to its top tool, Copilot Pro (which is also powered by ChatGPT-4), for the same price. But, do you really need to factor another pricey subscription into your budget? After hours of testing these subscription chatbots and prodding at their limitations, my two core takeaways remain the same in 2024 as they were last year when these services first arrived.
First, most people are fine with the free option. If you have a specialized need for the tool, like coding, or want to experiment with powerful AI models and features currently available, then Gemini Advanced or ChatGPT Plus might be worth $20 a month. For the average chatbot user, who may utilize AI to craft emails at work and Rick and Morty fan fiction at home, the basic versions of ChatGPT and Gemini are free, competent, and wildly more powerful than anything available in the recent past.
My second key takeaway? Don’t immediately trust the output. It’s been said a million times, and I’m here to say it again: Chatbots love to lie. For example, in previous tests ChatGPT’s image analysis feature confidently mislabeled my daily multivitamin as a prescription pill for erectile dysfunction, a potentially dangerous mix-up.
Are you still interested in subscribing to an AI chatbot tool, but not sure which one is the best fit for you? Here’s some helpful context about how Gemini Advanced and ChatGPT Plus compare—and what sets each subscription apart.
What’s Included With the AI Chatbot Subscriptions
Gemini Advanced from Google: As a package deal, Gemini Advanced offers the most to users on top of an impressive chatbot. Yes, you receive access to Google’s best AI model, Gemini Ultra 1.0, with the $20 per month AI Premium plan, but you also get everything offered with the company’s Google One subscription included in that price, which includes 2 terabytes of cloud storage. The company is expected to add a Gemini integration for Gmail and Docs as part of the subscription. Google just announced another new Gemini model, Gemini Pro 1.5, that can process more data than the current iteration, but this is not yet available to the public.
ChatGPT Plus from OpenAI: If you’ve experimented with AI chatbots in the past, odds are you’re familiar with using ChatGPT, which makes the transition to ChatGPT Plus with GPT-4 and Dall-E 3 quite simple. While OpenAI’s subscription does not include ancillary perks like cloud storage, it does have one exclusive, innovative feature: the GPT store. Here you can build and share custom versions of ChatGPT that have been optimized for different situations.
Copilot Pro from Microsoft: Similar to ChatGPT Plus, you get unfettered access to GPT-4 and Dall-E 3 when you subscribe to Copilot Pro. Built on top of OpenAI’s technology, Copilot Pro’s core differentiator is its integration with Microsoft’s suite of productivity software. The AI tools can be used directly inside of Excel, Outlook, and PowerPoint, if you’re also an active Microsoft 365 subscriber.
Comparing Outputs From Gemini Advanced and ChatGPT Plus
Even though we have experience testing a variety of chatbots at WIRED and putting fresh AI features to the test, keep in mind that these comparisons are designed to give you an overview of how the tools work. My tests are not all-encompassing, bro. (For example, I have too much respect for coders to pretend that I could gauge the worthiness of an AI tool for software development.) Also, since Microsoft’s offering uses the same generative AI models as OpenAI’s service, you can expect similar results from both tools. For this reason, I just compared the results of ChatGPT Plus and Gemini Advanced.
To start off, chatbots are often positioned as a productivity tool for white-collar workers. So I tried to see how well ChatGPT Plus and Gemini Advanced would be at a basic meeting summary. After uploading a transcript from an interview with a video game developer, I asked the chatbots, “Could you please summarize this meeting transcript into five bullet points?”
