Smooth Operator Agent Tools

If you are looking for the MCP server: find it here.

If you are looking for the Python Library, go on.. ;)

Smooth Operator Agent Tools - Python Library

This is the official Python library implementation for Smooth Operator Agent Tools, a state-of-the-art toolkit for programmers developing Computer Use Agents on Windows systems.

Overview

The Smooth Operator Agent Tools are a powerful toolkit that handles the complex tasks of interacting with the Windows Automation Tree and Playwright browser control, while providing advanced AI functions such as identifying UI elements through screenshots and textual descriptions.

This Python library provides a convenient wrapper around the Smooth Operator Tools Server API, allowing you to easily integrate these capabilities into your Python applications.

All features can be tested and explored through a convenient Windows user interface before implementing them in code. Try them out at Smooth Operator Tools UI.

Installation

pip install smooth-operator-agent-tools

Prerequisites

Google Chrome

The Smooth Operator Agent Tools library requires Google Chrome (or a compatible Chromium-based browser) to be installed on the system for browser automation features to work.

Server Installation

The Smooth Operator client library includes a server component that needs to be installed in your application data directory. The server files are packaged with the library and will be automatically extracted on first use.

First-Time Execution

When you first use the library, it will automatically:

Create the directory %APPDATA%\SmoothOperator\AgentToolsServer (or the equivalent on your OS)
Extract the server files from the package
Start the server process

Note that for Chrome automation features to work, you need to ensure Node.js and Playwright are installed as described in the Prerequisites section.

For Application Installers

If you're building an application installer that includes this library, you should include steps to install Node.js and Playwright during your application's installation process for better user experience. See the Prerequisites section for the required installation steps.

Usage

from smooth_operator_agent_tools import SmoothOperatorClient

# Initialize the client with your API key, get it for free at https://screengrasp.com/api.html
client = SmoothOperatorClient(api_key="YOUR_API_KEY")

# Start the Server - this takes a moment
client.start_server()

# Take a screenshot
screenshot = client.screenshot.take()

# Get system overview
overview = client.system.get_overview()

# Perform a mouse click
client.mouse.click(500, 300)

# Find and click a UI element by description
client.mouse.click_by_description("Submit button")

# Type text
client.keyboard.type("Hello, world!")

# Control Chrome browser
client.chrome.open_chrome("https://www.example.com")
client.chrome.get_dom()

# You can also use the to_json_string() method on many objects
# to get a JSON string that can easily be used in a prompt to a LLM
# to utilize AI even more for automated decision making

Features

Screenshot and Analysis: Capture screenshots and analyze UI elements
Mouse Control: Precise mouse operations using coordinates or AI-powered element detection
Keyboard Input: Type text and send key combinations
Chrome Browser Control: Navigate, interact with elements, and execute JavaScript
Windows Automation: Interact with Windows applications and UI elements
System Operations: Open applications and manage system state

Documentation

For detailed API documentation, visit:

Usage Guide: Detailed examples and explanations for common use cases.
Example Project: Download, follow step by step instructions and have your first automation running in mintes.
Documentation: Detailed documentation of all the API endpoints of the server that is doing the work internally.

License

This project is licensed under the MIT License - see the LICENSE file for details.

This server cannot be installed

security - not tested

license - permissive license

quality - not tested

How are these scores calculated?

local-only server

The server can only run on the client's local machine because it depends on local resources.

Windows automation MCP offering

AI Vision (e.g. Click by Description)
Windows UI Automation Tree tools
Chrome Automation via Playwright
Mouse control
Keyboard control
a lot more (>40 tools)

Also comes with Python/TypeScript/C# client libs and a Windows Desktop tool to try all the tools.

Smooth Operator Agent Tools - Python Library

Related MCP Servers

MCP Selenium
angiejones
-
security
A
license
-
quality
Enables browser automation using the Selenium WebDriver through MCP, supporting browser management, element location, and both basic and advanced user interactions.
Last updated -
867
225
JavaScript
MIT License
Playwright Server
williamvd4
A
security
A
license
A
quality
A MCP server that provides browser automation tools, allowing users to navigate websites, take screenshots, click elements, fill forms, and execute JavaScript through Playwright.
Last updated -
8
Python
Apache 2.0
PyMCPAutoGUI
kitfactory
-
security
A
license
-
quality
An MCP server that bridges AI agents with GUI automation capabilities, allowing them to control mouse, keyboard, windows, and take screenshots to interact with desktop applications.
Last updated -
8
Python
MIT License
PyAutoGUI MCP Server
hetaoBackend
-
security
A
license
-
quality
Provides automated GUI testing and control capabilities through an MCP server that enables mouse movements, keyboard input, screen captures, and image recognition across Windows, macOS, and Linux.
Last updated -
26
Python
MIT License

View all related MCP servers

Smooth Operator Agent Tools

If you are looking for the MCP server: find it here.