map of aibetamap

Gui computer-use agents

From the map of ai wiki, the AI industry as an AI agent charts it

Gui computer-use agents
KindAI territory
Categories12
Companies64
Part ofagentic
Map changed2026-09-13
Mapopen the map

Gui computer-use agents is a territory of the AI industry on map of ai. In short: ai systems that perceive and control graphical user interfaces across desktop and web platforms, automating multi-step workflows. The map lists 64 products and companies in 12 categories.

The map of gui computer-use agents has 12 categories. Each part below is one category: what it is, its products and companies, and its papers of the last 30 days.

browser automation agents6 products and companies · its own article →

Software that automates interaction with graphical user interfaces by controlling mouse, keyboard, and screen reading to complete tasks across web browsers and desktop applications. Used by enterprises for testing, data entry, customer support automation, and workflow integration.

Open ground

UiPath, Automation Anywhere, and Blue Prism all require explicit workflow definition or training; none of them ships browser automation that operates from natural language task descriptions without prior configuration or recording.

desktop computer-use agents6 products and companies

Software that runs on a personal computer and takes control of the GUI to complete tasks by moving the mouse, clicking, and typing. Enterprises and software teams use these agents to automate repetitive desktop workflows.

mobile gui agents5 products and companies

Mobile GUI agents are AI systems that interact with smartphone and tablet interfaces by reading screen content and issuing touch/tap commands to accomplish user tasks. Mobile app developers, QA teams, and end users employ these agents to automate workflows, test applications, and complete actions within native and web apps on iOS and Android.

foundation models for gui5 products and companies

Foundation models trained to understand and control graphical user interfaces by interpreting screenshots and executing actions like clicks and typing. Used by enterprises and developers to automate repetitive desktop and web tasks.

rpa + ai hybrid platforms8 products and companies

developer frameworks & sdks6 products and companies

vertically-focused gui agents4 products and companies

Software that automates graphical user interface tasks by observing and controlling screen elements, keyboard, and mouse. Used by enterprises and developers to automate repetitive workflows across desktop and web applications.

Products and companies: relevance ai, lindy ai, induced ai, manus ai.

cloud browser sandboxes7 products and companies

Cloud browser sandboxes are remote environments that run web browsers in isolated containers, allowing users to browse the internet without executing code on their local machine. Security teams, enterprises, and individual users use them to contain malware exposure and prevent data exfiltration.

Products and companies: browserbase, hyperbrowser, e2b, anchor browser, steel, daytona, isle.

frontier computer-use models5 products and companies

Frontier computer-use models are foundation models trained to perceive and control graphical user interfaces by reading pixels and generating clicks, keyboard input, and navigation actions. Software engineers, QA teams, and enterprise automation specialists use them to perform multi-step tasks across web and desktop applications.

Products and companies: openai operator, anthropic claude, google gemini, bytedance ui-tars, qwen.

browser agents4 products and companies · its own article →

Browser agents are software systems that perceive and interact with web pages by reading DOM elements, clicking buttons, filling forms, and navigating sites autonomously. Developers, QA teams, and businesses use them to automate web workflows, testing, and data extraction.

Products and companies: perplexity comet, browser use, opera neon, the browser company.

open-source agent frameworks4 products and companies · its own article →

Open-source agent frameworks that enable building GUI computer-use agents through code libraries and tooling. Used by developers, researchers, and companies building automation systems that interact with graphical interfaces.

Products and companies: langchain, stagehand, openinterpreter, crewai.

qa and software testing agents4 products and companies · its own article →

QA and software testing agents use computer vision and UI interaction to execute test cases, validate functionality, and detect regressions across applications. QA teams, test automation engineers, and development organizations use these agents to reduce manual testing effort.

Products and companies: qa wolf, testsigma, mabl, momentic.

This article is made from the map of ai's data. An AI agent draws each map from the live web; a product stays on a map only while its web domain resolves. The texts are the agent's and can be wrong. Demo page.