Login
iscover
Machine learning and artificial intelligence developer Trellis Data has announced a breakthrough in large language model decoders that enables outputs to be generated more than three times faster.
To continue reading the rest of this article, please log in.
Create free account to get unlimited news articles and more!
Trellis Data’s Dynamic Depth Decoding (D3) technique uses speculative decoding to reportedly achieve an average speed increase of 44 per cent, compared with the previous fastest decoder.
The company claimed the improvement is the fastest decoder available in the world currently
Large language models (LLM) are the architecture behind AI systems and D3 enables them to generate text approximately 3.16 times faster than standard decoding without compromising on accuracy.
The World Economic Forum estimates the computational power required to sustain global growth in AI is doubling every 100 days. With AI systems creating more and more value for organisations around the world, methods that create efficiencies in cost and energy requirements and mitigate environmental impact have become highly sought after.
According to Trellis Data research submitted for global peer review, the improvement in speed delivered by D3 translates to an estimated 68.4 per cent reduction in computational power requirements, which, in turn, lowers the costs and carbon emissions associated with running LLMs.
“There are three components critical to the success of any AI system: trust, speed and performance. Speed is a key area because it has a direct correlation with cost and environmental impact,” Trellis Data CEO Michael Gately said.
“D3 enables us to address one of the key bottlenecks of speed – the decoder – offering customers a reduction in the cost of running AI servers and a lower carbon footprint.
“Trellis Data is at the forefront of the rapidly developing AI field and we’re excited to make D3 available on our platform. D3 will provide customers – using our speech management and knowledge management capabilities – immediate benefits, including an enhanced user experience and greater ability to scale.”
D3 is broadly applicable to generative AI use cases, including chatbots, text summarisation, translation, transcription, and content generation. Additionally, the reduced computational costs and carbon emissions deliver a competitive advantage for customers, particularly as LLM size and usage continues to grow.
Trellis Data uses a broad spectrum of AI technologies, including generative AI, transcription, translation, and computer vision, specifically designed to save time and reduce costs. The development of D3 is another key milestone in Trellis Data’s strategy to address the important concerns relating to the adoption of new AI technologies as they deliver secure, disconnected AI applications that seamlessly integrate with existing business systems.
Headquartered in Canberra, Trellis Data has a growing team of over 40 staff with annual recurring revenue growth set to exceed 100 per cent. The company also has offices in Adelaide, Sydney and Arlington, with imminent plans to expand into the Middle East and Southeast Asia.
