Skip to main content
Glama

Enhanced Web Scraper MCP Server

by JMRMEDEV
GitHub
JavaScript
OverviewInspectNewEndpointsSchemaRelated ServersReviewsScore
Need Help?View Source CodeReport Issue

Enhanced Web Scraper MCP Server

A professional Model Context Protocol (MCP) server for web scraping, React app testing, and React Native web app inspection using Playwright. Fully backward compatible with regular websites and standard React applications.

🚀 Recent Improvements

  • Enhanced Error Handling - Comprehensive input validation and error reporting

  • Optimized Performance - Reduced code duplication and improved efficiency

  • Standardized Timeouts - Configurable timeout constants for reliability

  • Professional Code Structure - ES6+ best practices and maintainable architecture

🔄 Backward Compatibility

This enhanced server maintains 100% compatibility with:

  • ✅ Regular websites (HTML, CSS, JavaScript)

  • ✅ Standard React applications (Create React App, Next.js, etc.)

  • ✅ Traditional web scraping workflows

  • ✅ Existing CSS selectors and interactions

Plus new enhanced support for:

  • 🆕 React Native web applications

  • 🆕 Expo web projects

  • 🆕 Mobile viewport emulation

  • 🆕 Advanced React component inspection

Features

🌐 Universal Web Scraping

  • Multi-browser support (Chromium, Firefox, WebKit)

  • Regular CSS selectors work as expected

  • Automatic fallback to React Native selectors when needed

  • Mobile viewport emulation (optional)

  • Device-specific emulation (iPhone, Pixel, etc.)

📱 Enhanced React Support

  • Regular React apps - Works with standard React applications

  • React Native web - Enhanced support for RN web components

  • React hydration detection - Smart waiting for React apps to load

  • Component tree analysis - Deep React component inspection

  • TestID and accessibility support - React Native testing patterns

  • Mobile gesture simulation - Touch interactions for mobile UX

🔧 Development Tools

  • Expo development integration - Metro bundler health checks

  • Framework detection - Identifies React, Expo, and React Native web

  • Performance monitoring - Load times and metrics

  • Debug capabilities - Enhanced error reporting

Available Tools

1. scrape_page - Universal Web Scraping

Works with any website - regular HTML, React apps, or React Native web.

Regular website example:

{ url: "https://example.com", selector: ".article-title", // Standard CSS selector screenshot: true }

React Native web example:

{ url: "http://localhost:8081", selector: "login-button", // Will try testID, aria-label fallbacks mobileViewport: true, device: "iPhone 12" }

2. test_react_app - Universal React Testing

Works with any React application - standard React or React Native web.

Standard React app example:

{ url: "http://localhost:3000", waitForHydration: false, // Optional for regular React apps actions: [ { type: "click", selector: "#submit-button" }, { type: "fill", selector: "input[name='email']", value: "test@example.com" } ] }

React Native web example:

{ url: "http://localhost:8081", device: "iPhone 12", waitForHydration: true, // Recommended for RN web actions: [ { type: "tap", selector: "login-button" }, { type: "swipe", selector: "scroll-view", value: "up" } ] }

3. get_page_info - Enhanced Page Analysis

Provides comprehensive information for any web page with React-specific insights.

{ url: "https://any-website.com", // Works with any URL includePerformance: true }

4. extract_content - Clean Content Extraction

Extract clean, readable content from web pages without HTML/CSS clutter. Perfect for documentation, articles, and structured content consumption.

{ url: "https://docs.example.com/api-guide", includeLinks: true, // Extract and categorize hyperlinks format: "markdown" // Output format: 'markdown' or 'text' }

Output Example:

# API Documentation ## Authentication You need to obtain an API key [1] from the developer portal [2]. ### Rate Limits See the rate limiting guide [3] for details. --- ## Links Found: [1] https://example.com/api-keys (internal) [2] https://developer.example.com (external) [3] https://example.com/docs/rate-limits (internal)

Features:

  • Clean Structure - Preserves headings, paragraphs, lists, code blocks

  • Link Extraction - Categorizes links as internal, external, anchor, or download

  • Content Filtering - Removes navigation, ads, sidebars automatically

  • Multiple Formats - Markdown or plain text output

5. wait_for_element - Smart Element Waiting

Intelligent element waiting with automatic selector strategy fallbacks.

{ url: "https://example.com", selector: ".loading-spinner", // CSS selector with RN fallbacks timeout: 10000 }

React Native Web Specific Tools

6. inspect_react_app - React Component Analysis

Deep inspection of React applications (works best with React Native web).

7. wait_for_react_state - React State Management

Wait for React-specific conditions like hydration, navigation, data loading.

8. execute_in_react_context - JavaScript Execution

Execute JavaScript in React context for advanced inspection.

9. check_expo_dev_server - Expo Development Tools

Check Expo/Metro bundler status for development workflows.

Selector Strategy Priority

The server uses intelligent selector strategies:

  1. Primary: Direct CSS selector (e.g., #button, .class, input[name='email'])

  2. Fallback 1: TestID attribute ([data-testid="button"])

  3. Fallback 2: Accessibility label ([aria-label="Button"])

  4. Fallback 3: AccessibilityLabel ([accessibilityLabel="Button"])

This ensures regular CSS selectors work normally while providing React Native web compatibility.

Usage Examples

Regular Website Scraping

// Works exactly like before { url: "https://news.ycombinator.com", selector: ".storylink", screenshot: false }

Standard React App Testing

// Standard React app (Create React App, Next.js, etc.) { url: "http://localhost:3000", actions: [ { type: "click", selector: "button.login" }, { type: "fill", selector: "#username", value: "testuser" } ] }

React Native Web App Testing

// React Native web with enhanced features { url: "http://localhost:8081", device: "iPhone 12", waitForHydration: true, actions: [ { type: "tap", selector: "login-button" }, // Uses testID { type: "swipe", selector: "scroll-view", value: "up" } ] }

Clean Content Extraction

// Extract clean content from documentation { url: "https://docs.react.dev/learn", includeLinks: true, format: "markdown" }

Installation

npm install npx playwright install

Usage with Amazon Q Developer

# Works with any website q chat "Scrape the headlines from https://news.ycombinator.com" # Works with React apps q chat "Test the login flow on my React app at localhost:3000" # Enhanced React Native web support q chat "Inspect the React Native web app at localhost:8081" # Extract clean content for reading q chat "Extract the main content from https://docs.react.dev/learn"

Troubleshooting

Error Handling

  • Input Validation - Server validates required parameters and provides clear error messages

  • Timeout Configuration - Default timeouts are optimized but can be adjusted per request

  • Browser Cleanup - Automatic resource cleanup prevents memory leaks

Regular Websites

  • Use standard CSS selectors (.class, #id, tag[attribute])

  • Set mobileViewport: false (default) for desktop sites

  • Set waitForHydration: false (default) for non-React sites

React Applications

  • Set waitForHydration: true for better reliability

  • Use semantic selectors when possible

  • Check browser console for React errors

React Native Web

  • Use testID attributes in your components

  • Enable mobileViewport or specify device

  • Set waitForHydration: true

  • Use inspect_react_app to see available elements

License

MIT

Deploy Server
-
security - not tested
F
license - not found
-
quality - not tested

How are these scores calculated?

local-only server

The server can only run on the client's local machine because it depends on local resources.

Tools

View all tools

Enables web scraping, React app testing, and React Native web app inspection using Playwright with multi-browser support. Provides backward compatibility with regular websites while offering enhanced features for React applications including mobile viewport emulation and component analysis.

  1. 🚀 Recent Improvements
    1. 🔄 Backward Compatibility
      1. Features
        1. 🌐 Universal Web Scraping
        2. 📱 Enhanced React Support
        3. 🔧 Development Tools
      2. Available Tools
        1. 1. scrape_page - Universal Web Scraping
        2. 2. test_react_app - Universal React Testing
        3. 3. get_page_info - Enhanced Page Analysis
        4. 4. extract_content - Clean Content Extraction
        5. 5. wait_for_element - Smart Element Waiting
      3. React Native Web Specific Tools
        1. 6. inspect_react_app - React Component Analysis
        2. 7. wait_for_react_state - React State Management
        3. 8. execute_in_react_context - JavaScript Execution
        4. 9. check_expo_dev_server - Expo Development Tools
      4. Selector Strategy Priority
        1. Usage Examples
          1. Regular Website Scraping
          2. Standard React App Testing
          3. React Native Web App Testing
          4. Clean Content Extraction
        2. Installation
          1. Usage with Amazon Q Developer
            1. Troubleshooting
              1. Error Handling
              2. Regular Websites
              3. React Applications
              4. React Native Web
            2. License

              New MCP Servers

              MCP directory API

              We provide all the information about MCP servers via our MCP API.

              curl -X GET 'https://glama.ai/api/mcp/v1/servers/JMRMEDEV/amazon-q-web-scraper-mcp'

              If you have feedback or need assistance with the MCP directory API, please join our Discord server