arxiv preprint - WorkArena: How Capable Are Web Agents at Solving Common Knowledge Work Tasks?

In this episode, we discuss WorkArena: How Capable Are Web Agents at Solving Common Knowledge Work Tasks? by Alexandre Drouin, Maxime Gasse, Massimo Caccia, Issam H. Laradji, Manuel Del Verme, Tom Marty, Léo Boisvert, Megh Thakkar, Quentin Cappart, David Vazquez, Nicolas Chapados, Alexandre Lacoste. The paper introduces WorkArena, a benchmark created to evaluate large language model-based agents that interact with web-based enterprise software like ServiceNow, along with BrowserGym, a tool for creating and testing these agents. The study assesses the agents' abilities to complete typical knowledge worker tasks, finding that while agents have potential in this area, there is still a substantial gap before achieving complete task automation. The results also reveal differences in the performances of open versus closed-source language models, pointing to a key direction for continued research and improvement.