OpenAI has just announced a significant update to ChatGPT, introducing support for voice conversations and image recognition. This AI-powered chatbot is expanding its capabilities to comprehend and provide information related to images shared by users across various platforms.
Additionally, it’s gaining the ability to engage in back-and-forth conversations using OpenAI’s Whisper speech recognition tool, coupled with a new text-to-speech (TTS) technology, promising lifelike audio experiences on the ChatGPT smartphone app.
OpenAI disclosed that this image recognition feature will be accessible on all platforms, while voice conversations will be initially offered on iOS and Android through an opt-in setting. These enhancements are intended for ChatGPT Plus and Enterprise subscribers, with no information yet regarding their availability for free-tier users in the future.
Enabling voice conversations on ChatGPT is a straightforward process via the Settings > New Features menu, allowing users to choose from five distinct voices, a feature developed in collaboration with professional voice actors.
The ChatGPT app will efficiently convert spoken queries into text that the chatbot can understand, and responses will be rendered using the new TTS technology for a human-like auditory experience.
It’s noteworthy that OpenAI’s TTS technology won’t be exclusive to ChatGPT; Spotify has also harnessed it for an AI-based voice translation tool, designed for podcast creators.
This tool can automatically translate English-language podcasts into French, German, and Spanish. Initially tested with select podcast hosts, translated episodes will eventually become accessible to all Spotify users worldwide.
OpenAI’s image recognition tool relies on the company’s multimodal GPT-3.5 and GPT-4 models, enabling it to analyze both images and text present in photos, screenshots, and documents. Users can either capture new images or share existing ones with ChatGPT to glean insights from the chatbot.
Furthermore, ChatGPT allows users to share multiple images for discussion, with an added functionality of a built-in drawing tool.
This tool enables users to highlight specific areas in the images they want to focus on. For example, marking a dislodged bicycle chain in a shared photo can prompt ChatGPT to offer guidance on how to rectify the issue.
