{"id":67,"date":"2024-06-28T19:55:00","date_gmt":"2024-06-28T19:55:00","guid":{"rendered":"https:\/\/www.salesforce.com\/?post_type=sf_press_release&p=67"},"modified":"2025-09-24T14:25:35","modified_gmt":"2025-09-24T06:25:35","slug":"salesforce-announces-the-worlds-first-llm-benchmark-for-crm","status":"publish","type":"sf_press_release","link":"https:\/\/www.salesforce.com\/ap\/news\/press-releases\/2024\/06\/28\/salesforce-announces-the-worlds-first-llm-benchmark-for-crm\/","title":{"rendered":"Salesforce announces the world’s first LLM benchmark for CRM"},"content":{"rendered":"\n

New benchmark and leaderboard give businesses the guidance they need to make smart decisions when evaluating generative AI models for their CRM systems <\/em><\/p>\n\n\n\n

SINGAPORE, 28 Jun 2024 <\/strong>\u2014 Salesforce has announced the world\u2019s first LLM benchmark<\/a> for CRM to help businesses evaluate the rapidly growing number of large language models (LLMs) for use in their customer relationship management (CRM) systems. <\/p>\n\n\n\n

The new benchmark is a comprehensive evaluation framework that measures the performance of LLMs against four key measures: accuracy, cost, speed, and trust and safety. It\u2019s been specifically designed to evaluate common sales and service use cases, including prospecting, lead nurturing, as well as sales opportunity and service case summaries. The benchmark also includes a public leaderboard to help professionals decide which LLM is best for their CRM needs. Salesforce will continue to incorporate new use case scenarios into the benchmark and enhance its evaluation of LLMs, which will soon include fine-tuned LLMs. <\/p>\n\n\n\n

\u201cAs AI continues to evolve, enterprise leaders are saying it\u2019s important to find the right mix of performance, accuracy, responsibility, and cost to unlock the full potential of generative AI to drive business growth,\u201d said Silvio Savarese, EVP & Chief Scientist, <\/strong>Salesforce AI Research<\/strong><\/a>. \u201cSalesforce\u2019s new LLM Benchmark for CRM is a significant step forward in the way businesses assess their AI strategy within the industry. It not only provides clarity on next-generation AI deployment but also can accelerate time to value for CRM-specific use cases. Our commitment is to continuously evolve this benchmark to keep pace with technological advancements, ensuring it remains relevant and valuable.\u201d <\/p>\n\n\n\n

Why it matters:<\/strong> Existing LLM benchmarks have been limited to academic and consumer use cases, with very little business relevance. They also lack adequate expert human evaluations and fail to address accuracy, speed, cost, and trust considerations. These deficiencies have left CRM customers lacking a reliable way to gauge the effectiveness of generative AI-powered CRM solutions. Without a clear sense of how LLMs perform across those metrics for specific use cases, businesses are left to make decisions in the dark. <\/p>\n\n\n

\n
Open Image Modal<\/span>\"\"<\/a><\/figure>\n
<\/div>