Nvidia has introduced a revolutionary AI model, Fugatto, designed to transform how we create and manipulate audio. Short for Foundational Generative Audio Transformer Opus 1, Fugatto sets itself apart by offering unparalleled control over the generated output, enabling users to craft music, voices, and sound effects tailored to precise requirements.
Unlike existing audio-focused AI tools like Beatoven or Suno, Fugatto allows users to shape their creations with exceptional granularity. From generating entirely new sounds to modifying existing audio, the possibilities are expansive.
Features and Capabilities
In a recent blog post, Nvidia revealed that Fugatto can:
- Compose original music snippets or edit existing ones by adding or removing instruments.
- Modify vocal characteristics, such as altering accents, emotional tones, or even languages.
- Create imaginative sounds, such as a trumpet that “barks” or a saxophone that “meows.”
The model supports both text and audio inputs, enabling users to refine their requests with specific instructions. For example, through a process Nvidia calls ComposableART, users can generate a voice speaking French with a controlled level of sadness and accent intensity. Additionally, Fugatto’s temporal interpolation feature allows for dynamic soundscapes, like a rainstorm evolving into distant thunder.
Cutting-Edge Technology
Fugatto is powered by 2.5 billion parameters and was trained on Nvidia’s DGX systems, building on the company’s expertise in speech modeling, vocoding, and audio comprehension. The team behind Fugatto brought together experts from around the world—including Brazil, China, India, Jordan, and South Korea—to enhance its multilingual and multi-accent capabilities.
Impressively, Fugatto can even produce sounds it wasn’t explicitly trained on, showcasing its adaptability. Nvidia explained that users can describe virtually any audio scenario, and the model will generate it.
While Fugatto’s potential is immense, Nvidia has yet to announce plans to make the model accessible to the public or enterprises.
This development underscores Nvidia’s commitment to pushing the boundaries of generative AI, signaling an exciting future for audio innovation.
