Every contender vies for your $20 monthly to unlock their top-tier AI chatbot. The victor? Whoever aligns best with your feature preferences.
Testing AI chatbot subscriptions like Google’s Gemini Advanced and OpenAI’s ChatGPT Plus presents a common dilemma: their broad applicability. The same software utilized by developers in Silicon Valley may also be embraced by parents in the heartland planning a Paw Patrol-themed bash. Despite companies flaunting intricate benchmarks to showcase their chatbots' superiority, deciphering how technical prowess translates into a superior user experience remains challenging.
Google recently joined the subscription fray, offering access to Gemini Advanced for $20 monthly in early February. Following OpenAI's lead, Microsoft also provides access to Copilot Pro, fueled by ChatGPT-4, at the same price point. But, must you add yet another costly subscription to your budget? After extensive testing and probing at their limitations, my key insights from 2024 echo those from last year when these services first emerged.
Testing AI chatbot subscriptions like Google’s Gemini Advanced and OpenAI’s ChatGPT Plus presents a fundamental issue: their broad applicability. The same tool employed by developers in bustling tech hubs like San Francisco might also find use among parents in quieter locales like Kansas planning a Paw Patrol-themed birthday bash. Despite companies often boasting about obscure benchmarks to assert their chatbots' superiority, understanding how technical prowess translates into a superior user experience remains a challenge.
Google recently joined the subscription fray, rolling out access to Gemini Advanced for $20 monthly in early February. This move followed OpenAI's lead, offering its GPT-4-powered chatbot for the same price. Microsoft also joins the competition, offering subscriptions to Copilot Pro, also fueled by ChatGPT-4, at a comparable rate. But, do you truly need to add another pricey subscription to your budget? After extensive testing and exploring their limitations, my core insights from 2024 echo those from last year when these services first emerged.
Read more: What is ChatGPT AI? Here's everything you need to know
Firstly, for most individuals, the free option suffices. If you have a specialized need for the tool, such as coding, or want to experiment with powerful AI models and features, then Gemini Advanced or ChatGPT Plus might justify the $20 monthly fee. However, for the average chatbot user, whose AI interactions may range from crafting work emails to drafting Rick and Morty fan fiction, the basic versions of ChatGPT and Gemini are not only free but also competent and significantly more robust than previous offerings.
My second key insight? Don’t blindly trust the output. It’s a known fact, reiterated time and again: Chatbots have a penchant for misinformation. For instance, in prior tests, ChatGPT’s image analysis feature confidently misidentified my daily multivitamin as a prescription pill for erectile dysfunction, a potentially hazardous error.
If you're still considering subscribing to an AI chatbot tool but unsure which one suits you best, here’s some insightful context on how Gemini Advanced and ChatGPT Plus stack up against each other—and what distinguishes each subscription.
What You Get: Unpacking the Offerings of AI Chatbot Subscriptions
Gemini Advanced by Google: Bundling together the best features, Gemini Advanced not only provides access to Google's top-tier AI model, Gemini Ultra 1.0, but also throws in extras from the company's Google One subscription, including 2 terabytes of cloud storage, all for $20 monthly. Moreover, Google plans to introduce Gemini integration for Gmail and Docs as part of the package. While a new Gemini model, Gemini Pro 1.5, boasting enhanced data processing capabilities, has been announced, it's not yet accessible to the public.
ChatGPT Plus by OpenAI: If you're already familiar with ChatGPT, transitioning to ChatGPT Plus with GPT-4 and Dall-E 3 is seamless. While OpenAI's subscription lacks additional perks like cloud storage, it does offer a unique feature: the GPT store. Here, users can create and exchange custom versions of ChatGPT tailored for various scenarios.
Copilot Pro by Microsoft: Like ChatGPT Plus, Copilot Pro grants unrestricted access to GPT-4 and Dall-E 3. Leveraging OpenAI's technology, Copilot Pro sets itself apart with its integration into Microsoft's suite of productivity tools. Users can directly utilize AI functionalities within Excel, Outlook, and PowerPoint, especially beneficial for active Microsoft 365 subscribers.
Analyzing Results: Evaluating Outputs Between Gemini Advanced and ChatGPT Plus
While we've had our fair share of chatbot testing at WIRED and have put the latest AI features through their paces, it's important to note that these comparisons aim to provide a general overview of how the tools operate. My tests don't cover every possible scenario, and I have too much respect for coders to claim I could accurately assess the suitability of an AI tool for software development. Additionally, since Microsoft's offering utilizes the same generative AI models as OpenAI's service, you can anticipate similar outcomes from both tools. Hence, I focused solely on comparing the results of ChatGPT Plus and Gemini Advanced.
Starting off, chatbots are often touted as productivity aids for white-collar workers. So, I decided to evaluate how well ChatGPT Plus and Gemini Advanced could handle a basic meeting summary task. After uploading a transcript from an interview with a video game developer, I asked the chatbots to summarize the meeting into five bullet points.
This test yielded the most promising and similar results among all. Both chatbots excelled at capturing key moments and condensing them effectively. Four out of the five highlights generated by the chatbots mirrored the pivotal parts of the conversation, with only negligible differences in the fifth bullet point.
Another common workplace application for chatbots is improving email correspondence. Wondering how adept these tools are at refining harsh emails into a more professional tone, I drafted a stern message for my editor (who's actually a gem, and I'd never be unkind to) and requested both chatbots to make it workplace-appropriate.
Gemini Advanced aced the rephrasing test, providing not only a suitably reworded email but also offering multiple tips for crafting less harsh messages in the future. While ChatGPT Plus managed to adjust the email's tone, its writing came across as stiff and overly reliant on formal structures.
Exploring non-work uses for the chatbots, I submitted a cute photo of my partner and me hiking in Yosemite National Park to mimic one of Google's prompts for the Gemini Advanced demonstration. I asked for a catchy Instagram caption with multiple emojis and no hashtags.
Unfortunately, both chatbots struggled with this task. ChatGPT Plus's attempt produced an unconventional interpretation of "happy trail," while Gemini Advanced refrained from providing any text related to the image due to visible real people, implementing guardrails to prevent potential harassment.
Considering both chatbots include image generation features, I tested this aspect by requesting a blank party invitation for my imaginary 6-year-old's birthday party with a Peppa Pig theme and their favorite colors, pink and gold.
ChatGPT Plus managed to design a visually appealing invitation, albeit with some bizarre renditions of Peppa. However, neither chatbot could generate entirely legible text for the invite, with Gemini Advanced's results being particularly scribble-like.
Lastly, I wanted to gauge the chatbots' ability to engage in role-playing. I asked both tools to act as an ancient space wizard searching for a Dungeons & Dragons group on Earth but facing mild reluctance from people.
Surprisingly, both chatbots excelled at this task, delivering entertaining and engaging responses. ChatGPT Plus stayed in character admirably, peppering its dialogue with witty remarks, while Gemini Advanced, although breaking character occasionally, impressed with its lyrical and engaging writing style when weaving fantastical elements.
Exploring the Privacy Balances: Understanding the Trade-Offs
While engaging in conversations with a chatbot might seem like a private exchange, it's never entirely confidential. It's crucial to refrain from sharing sensitive or personal information with publicly available chatbots.
The primary concern stems from the fact that service providers can utilize your conversations to enhance their machine intelligence algorithms. Although OpenAI does offer the option for users to opt out of having their ChatGPT conversations contribute to algorithm training, these permissions are enabled by default, requiring users to actively choose to disable chat history storage. However, even with this opt-out feature, conversations are not entirely ephemeral. OpenAI's FAQ states that all conversations are retained for 30 days before permanent deletion, primarily to monitor for abuse.
While this may not be ideal, it's a step ahead of the default settings offered by Google's Gemini. In Gemini, if a conversation is randomly chosen for human review, it remains stored on Google's servers, even if the user decides to delete it. Although the conversation is disassociated from the user's account during review, it could still be retained by the company for up to three years. However, users have the option to disable Gemini Apps activity, preventing their new conversations from being reviewed by humans or used for AI model training. With this feature turned off, Google retains the data for a maximum of three days.
Pause: Is This Exclusive to American Users?
Not quite, but there's a caveat. Although ChatGPT Plus and Gemini Advanced are accessible in numerous international markets, the language you prefer for interacting with the chatbot does play a role. Both companies prioritize English.
While ChatGPT does offer support for languages beyond English, its success is somewhat limited. Initially, Gemini Advanced was tailored exclusively for English queries, but Google is gradually introducing support for other languages like Japanese and Korean for its chatbot.
Can Silicon Valley Ever Move Past the AI Chatbot Obsession?
Perhaps someday, but not in 2024.
Indeed, many AI tools are currently riding on a wave of exaggerated hype. Consumer expectations regarding AI-driven hyperproductivity may not align with the current capabilities of even the most advanced chatbots. However, these tools do offer tangible benefits for power users and early adopters, who are willing to invest in monthly subscriptions for access to cutting-edge technology.
Even if chatbots eventually fade from the limelight—whether due to dwindling public interest or legal battles over copyright—the underlying technology is poised to become integral to the next generation of web browsers, search engines, and operating systems. While AI developers may harbor aspirations of creating a supremely powerful, semi-sentient algorithm that revolutionizes society, for now, a $20 investment can significantly enhance your email writing skills at work.
){kind=link}