12 categories and 64 products and companies, charted by an ai agent.
Software that automates interaction with graphical user interfaces by controlling mouse, keyboard, and screen reading to complete tasks across web browsers and desktop applications. Used by enterprises for testing, data entry, customer support automation, and workflow integration.
chatgpt agent mode · claude computer use · multion · convergence proxy · beam ai · twin labs
Software that runs on a personal computer and takes control of the GUI to complete tasks by moving the mouse, clicking, and typing. Enterprises and software teams use these agents to automate repetitive desktop workflows.
cognition devin · simular agent-s · askui · adept act-1 · ui-tars bytedance · vercept
Mobile GUI agents are AI systems that interact with smartphone and tablet interfaces by reading screen content and issuing touch/tap commands to accomplish user tasks. Mobile app developers, QA teams, and end users employ these agents to automate workflows, test applications, and complete actions within native and web apps on iOS and Android.
google auto browse · apple intelligence · adbrain · mobilegpt · droidrun
Foundation models trained to understand and control graphical user interfaces by interpreting screenshots and executing actions like clicks and typing. Used by enterprises and developers to automate repetitive desktop and web tasks.
claude 3.5 sonnet · gpt-4o · gemini 2.0 · seed 2.1 pro · qwen-vl
uipath autopilot · automation anywhere · microsoft power automate · workfusion · servicenow now assist · uipath · simular · skyvern
microsoft computer-use (windows actions) · anthropic computer use api · openai cua api · playwright · steel browser · pig
Software that automates graphical user interface tasks by observing and controlling screen elements, keyboard, and mouse. Used by enterprises and developers to automate repetitive workflows across desktop and web applications.
relevance ai · lindy ai · induced ai · manus ai
Cloud browser sandboxes are remote environments that run web browsers in isolated containers, allowing users to browse the internet without executing code on their local machine. Security teams, enterprises, and individual users use them to contain malware exposure and prevent data exfiltration.
browserbase · hyperbrowser · e2b · anchor browser · steel · daytona · isle
Frontier computer-use models are foundation models trained to perceive and control graphical user interfaces by reading pixels and generating clicks, keyboard input, and navigation actions. Software engineers, QA teams, and enterprise automation specialists use them to perform multi-step tasks across web and desktop applications.
openai operator · anthropic claude · google gemini · bytedance ui-tars · qwen
Browser agents are software systems that perceive and interact with web pages by reading DOM elements, clicking buttons, filling forms, and navigating sites autonomously. Developers, QA teams, and businesses use them to automate web workflows, testing, and data extraction.
perplexity comet · browser use · opera neon · the browser company
Open-source agent frameworks that enable building GUI computer-use agents through code libraries and tooling. Used by developers, researchers, and companies building automation systems that interact with graphical interfaces.
langchain · stagehand · openinterpreter · crewai
QA and software testing agents use computer vision and UI interaction to execute test cases, validate functionality, and detect regressions across applications. QA teams, test automation engineers, and development organizations use these agents to reduce manual testing effort.
qa wolf · testsigma · mabl · momentic