# Create a new API key Source: https://docs.weblinq.dev/api-reference/api-keys/create-a-new-api-key https://api.weblinq.dev/doc post /v1/api-keys/create Create a new API key for the authenticated user. System defaults: wq_ prefix, 1000 requests per 24 hours rate limit, no expiration, free plan metadata. # Delete API key Source: https://docs.weblinq.dev/api-reference/api-keys/delete-api-key https://api.weblinq.dev/doc delete /v1/api-keys/{id} Delete an existing API key. Note: API keys cannot be updated - create a new one if needed. # Get API key details Source: https://docs.weblinq.dev/api-reference/api-keys/get-api-key-details https://api.weblinq.dev/doc get /v1/api-keys/{id} Get details of a specific API key # List user API keys Source: https://docs.weblinq.dev/api-reference/api-keys/list-user-api-keys https://api.weblinq.dev/doc get /v1/api-keys/list Get all API keys for the authenticated user # Error Handling Source: https://docs.weblinq.dev/api-reference/errors Learn how to handle errors with the WebLinq API The WebLinq API uses conventional HTTP response codes to indicate the success or failure of an API request: * `2xx` indicates success * `4xx` indicates an error that failed given the information provided * `5xx` indicates an error with our servers ## Error Response Formats WebLinq uses two different error response formats depending on the type of error: ### Validation Errors (422 Unprocessable Entity) When request validation fails, you'll receive a detailed error response with specific field-level issues: ```json Validation Error Format { "success": false, "error": { "issues": [ { "code": "invalid_type", "path": ["url"], "message": "Expected string, received number" }, { "code": "too_small", "path": ["waitTime"], "message": "Number must be greater than or equal to 0" } ], "name": "ZodError" } } ``` ### Other Errors (401, 404, 500, etc.) For authentication, authorization, not found, and server errors, you'll receive a simpler format: ```json Simple Error Format { "success": false, "error": { "message": "Authentication required", "code": "authentication_required" } } ``` ## Common HTTP Status Codes ### 200 OK Request succeeded. The response will contain the requested data. ### 401 Unauthorized Authentication required. You need to provide a valid API key. **Example Response:** ```json { "success": false, "error": { "message": "Authentication required", "code": "authentication_required" } } ``` ### 404 Not Found The requested resource doesn't exist. **Example Response:** ```json { "success": false, "error": { "message": "API key not found", "code": "not_found" } } ``` ### 422 Unprocessable Entity Request validation failed. Check the `issues` array for specific field errors. **Example Response:** ```json { "success": false, "error": { "issues": [ { "code": "invalid_url", "path": ["url"], "message": "Must be a valid URL" } ], "name": "ZodError" } } ``` ### 429 Too Many Requests Rate limit exceeded. Wait before making more requests. **Example Response:** ```json { "success": false, "error": { "message": "Rate limit exceeded. Please try again later", "code": "rate_limit_exceeded" } } ``` ### 500 Internal Server Error Something went wrong on our end. Try again later. **Example Response:** ```json { "success": false, "error": { "message": "An unexpected error occurred", "code": "internal_server_error" } } ``` ## Error Handling Best Practices ### 1. Always Check the `success` Field ```javascript const response = await fetch('/api/web/screenshot', { method: 'POST', headers: { Authorization: 'Bearer your-api-key', 'Content-Type': 'application/json', }, body: JSON.stringify({ url: 'https://example.com' }), }); const data = await response.json(); if (!data.success) { // Handle error console.error('API Error:', data.error); return; } // Use successful data console.log('Screenshot taken:', data.data); ``` ### 2. Handle Validation Errors Specifically ```javascript if (response.status === 422) { // Validation error - check individual field issues data.error.issues.forEach((issue) => { console.error(`Field '${issue.path.join('.')}': ${issue.message}`); }); } else { // Other error - display general message console.error(`Error (${data.error.code}): ${data.error.message}`); } ``` ## Need Help? If you encounter persistent errors, please contact our support team with: * The HTTP status code * The complete error response * Your request payload (without sensitive data) * The timestamp when the error occurred Contact our support team for assistance # Delete a file Source: https://docs.weblinq.dev/api-reference/files/delete-a-file https://api.weblinq.dev/doc post /v1/files/delete Delete a file from database and optionally from R2 storage # List user files Source: https://docs.weblinq.dev/api-reference/files/list-user-files https://api.weblinq.dev/doc get /v1/files/list List all files (screenshots, PDFs) associated with the authenticated user # API Overview Source: https://docs.weblinq.dev/api-reference/overview Complete reference for the WebLinq API endpoints The WebLinq API provides powerful endpoints for web automation and content extraction. All endpoints use REST principles and require authentication. Need to get started quickly? Check our [quickstart guide](/quickstart). ## Base URL ```bash https://api.weblinq.dev/v1 ``` Always use HTTPS and include the `/v1` prefix for all API requests. ## Authentication Include your API key in the Authorization header: ```bash cURL curl -X POST "https://api.weblinq.dev/v1/web/screenshot" \ -H "Authorization: Bearer YOUR_API_KEY" \ -H "Content-Type: application/json" \ -d '{"url": "https://example.com"}' ``` ```javascript JavaScript const response = await fetch('https://api.weblinq.dev/v1/web/screenshot', { method: 'POST', headers: { Authorization: 'Bearer YOUR_API_KEY', 'Content-Type': 'application/json', }, body: JSON.stringify({ url: 'https://example.com' }), }); ``` ```python Python import requests response = requests.post( 'https://api.weblinq.dev/v1/web/screenshot', headers={'Authorization': 'Bearer YOUR_API_KEY'}, json={'url': 'https://example.com'} ) ``` ## Available Endpoints ### Web Operations Capture webpage screenshots Convert pages to clean Markdown AI-powered structured data extraction Create PDFs from web pages Retrieve raw HTML content Extract specific elements with selectors Get all links from a page Search the web programmatically ### API Keys Generate a new API key View all your API keys View details of a specific key Revoke an API key ### User Get current user details ### Files View stored files Remove stored files ## Response Format All responses follow a standard format: ```json Success Response { "success": true, "data": { // Your response data here }, "creditsCost": 1 } ``` ```json Error Response { "success": false, "error": { "message": "Error description", "code": "error_code" } } ``` ## Rate Limits Rate limits are enforced per API key: * **Free plan**: 100 requests/hour * **Pro plan**: 1,000 requests/hour Rate limit headers are included in responses: * `X-RateLimit-Limit`: Total requests allowed * `X-RateLimit-Remaining`: Remaining requests * `X-RateLimit-Reset`: Reset timestamp ## Need Help? Email us for technical support Learn about error codes # Rate Limits Source: https://docs.weblinq.dev/api-reference/rate-limits Understanding and handling API rate limits WebLinq implements rate limiting to ensure fair usage across all users. Rate limits are enforced per API key. ## Current Limits * 100 requests per hour * 1,000 requests per day * No concurrent requests * 1,000 requests per hour * 10,000 requests per day * 5 concurrent requests Need higher limits? [Contact us](mailto:support@weblinq.dev) about enterprise plans. ## Rate Limit Headers All responses include rate limit information: ```http X-RateLimit-Limit: 100 # Total requests allowed per hour X-RateLimit-Remaining: 85 # Remaining requests in current window X-RateLimit-Reset: 1625097600 # Timestamp when limit resets ``` ## Handling Rate Limits When you exceed your rate limit, you'll receive a `429 Too Many Requests` response: ```json { "success": false, "error": { "message": "Rate limit exceeded. Please retry after 60 seconds.", "code": "rate_limit_exceeded" } } ``` ## Best Practices ### Implement Retry Logic Use exponential backoff when handling rate limits: ```javascript async function makeRequest(url, maxRetries = 3) { for (let i = 0; i < maxRetries; i++) { try { const response = await fetch(url, { headers: { Authorization: 'Bearer YOUR_API_KEY' }, }); if (response.status === 429) { const retryAfter = response.headers.get('Retry-After') || 60; await sleep(retryAfter * 1000); continue; } return await response.json(); } catch (error) { if (i === maxRetries - 1) throw error; await sleep(Math.pow(2, i) * 1000); } } } function sleep(ms) { return new Promise((resolve) => setTimeout(resolve, ms)); } ``` ### Monitor Usage Track your API usage to stay within limits: ```javascript function trackRateLimit(response) { const limit = response.headers.get('X-RateLimit-Limit'); const remaining = response.headers.get('X-RateLimit-Remaining'); const reset = response.headers.get('X-RateLimit-Reset'); console.log(`Rate limit: ${remaining}/${limit} remaining. Resets at ${new Date(reset * 1000)}`); } ``` ### Space Out Requests For bulk operations, add delays between requests: ```javascript async function processBulkUrls(urls) { const results = []; for (const url of urls) { const result = await makeRequest(url); results.push(result); // Wait 1 second between requests for free plan (100/hour = ~36 seconds per request safe rate) await sleep(1000); } return results; } ``` ## Need Help? Contact us if you're consistently hitting rate limits or need higher quotas # Success Responses Source: https://docs.weblinq.dev/api-reference/success Understanding successful WebLinq API responses All successful WebLinq API responses follow a consistent format for predictable handling across all endpoints. ## Standard Format Every successful response (HTTP status `2xx`) uses this structure: ```json { "success": true, "data": { // Endpoint-specific response data }, "creditsCost": 1 } ``` ### Response Fields | Field | Type | Description | | ------------- | ------- | ------------------------------------------------------ | | `success` | boolean | Always `true` for successful responses | | `data` | object | Contains the actual response data (varies by endpoint) | | `creditsCost` | number | Credits consumed by this operation | ## Common Response Types ### Screenshot Response The `image` field is present only when `base64` is `true` in the request. ```json { "success": true, "data": { "image": "iVBORw0KGgoAAAANSUhEUgAAA...", // Base-64, present only when requested "metadata": { "width": 1920, "height": 1080, "format": "png", "size": 245760, "url": "https://example.com", "timestamp": "2024-01-15T10:30:45.123Z" }, "permanentUrl": "https://files.weblinq.dev/abc123.png", "fileId": "file_abc123" }, "creditsCost": 1 } ``` ### Markdown Response ```json { "success": true, "data": { "markdown": "# Example Site\n\nThis is the converted content...", "metadata": { "title": "Example Site", "description": "A sample website", "url": "https://example.com", "timestamp": "2024-01-15T10:30:45.123Z", "wordCount": 245 } }, "creditsCost": 1 } ``` ### JSON Extraction Response ```json { "success": true, "data": { "extracted": { "title": "Product Name", "price": "$99.99", "description": "Product description..." }, "metadata": { "url": "https://example.com", "timestamp": "2024-01-15T10:30:45.123Z", "model": "llama-3.2-3b-instruct", "responseType": "json" } }, "creditsCost": 2 } ``` ## Handling Responses ### Basic Response Handling ```javascript const response = await fetch('https://api.weblinq.dev/v1/web/markdown', { method: 'POST', headers: { Authorization: 'Bearer YOUR_API_KEY', 'Content-Type': 'application/json', }, body: JSON.stringify({ url: 'https://example.com' }), }); const result = await response.json(); if (result.success) { console.log('Markdown content:', result.data.markdown); console.log('Credits used:', result.creditsCost); } else { console.error('API error:', result.error); } ``` ### TypeScript Types ```typescript interface ApiSuccessResponse { success: true; data: T; creditsCost: number; } interface MarkdownResponse { markdown: string; metadata: { title?: string; description?: string; url: string; timestamp: string; wordCount: number; }; } // Usage const response: ApiSuccessResponse = await apiCall(); ``` ## Response Metadata Most endpoints include metadata with additional information about the operation: * **url**: The source URL that was processed * **timestamp**: When the operation was completed * **format/type**: The type of content or format used * **size/wordCount**: Size metrics for the content ## File Storage Screenshot and PDF endpoints may include file storage information: * **permanentUrl**: Permanent URL for accessing the file * **fileId**: Unique identifier for file management Files are automatically stored in R2 cloud storage when available and can be accessed via the `permanentUrl` or managed through the [Files API](/api-reference/files/list-user-files). # Update user plan Source: https://docs.weblinq.dev/api-reference/system/update-user-plan https://api.weblinq.dev/doc post /v1/system/update-user-plan Update a user's subscription plan (free/pro) - Admin only # Bootstrap credits for existing users Source: https://docs.weblinq.dev/api-reference/user/bootstrap-credits-for-existing-users https://api.weblinq.dev/doc post /v1/user/bootstrap-credits Assign initial credits to existing users who do not have credits yet. # Clear all cached data for the user Source: https://docs.weblinq.dev/api-reference/user/clear-all-cached-data-for-the-user https://api.weblinq.dev/doc get /v1/user/clear-cache Deletes all cached entries for web operations (V1 and V2) for the authenticated user. This includes cached results from screenshot, markdown, PDF, scraping, and other web operation endpoints. # Get current user information Source: https://docs.weblinq.dev/api-reference/user/get-current-user-information https://api.weblinq.dev/doc get /v1/user/me Returns the current user information. Works with both session-based authentication (browser) and API token-based authentication (server-to-server). Returns user details if authenticated, or guest info if not authenticated. # Get user credit information Source: https://docs.weblinq.dev/api-reference/user/get-user-credit-information https://api.weblinq.dev/doc get /v1/user/credits Returns the current user credit balance, plan, and last refill date. # Initialize user resources Source: https://docs.weblinq.dev/api-reference/user/initialize-user-resources https://api.weblinq.dev/doc post /v1/user/initialize Initialize WebDurableObject and other user resources for development environments. # Verify email with token and redirect Source: https://docs.weblinq.dev/api-reference/user/verify-email-with-token-and-redirect https://api.weblinq.dev/doc get /v1/user/verify-email-token Custom email verification handler that processes the token and redirects to frontend dashboard with new_user parameter. # Verify if email exists Source: https://docs.weblinq.dev/api-reference/user/verify-if-email-exists https://api.weblinq.dev/doc post /v1/user/verify-email Check if a user account exists for the given email address. # Extract YouTube captions Source: https://docs.weblinq.dev/api-reference/web-v2/extract-youtube-captions https://api.weblinq.dev/doc post /v2/web/yt-captions Extract captions/subtitles from a YouTube video using video ID # Search the web (v2) Source: https://docs.weblinq.dev/api-reference/web-v2/search-the-web-v2 https://api.weblinq.dev/doc post /v2/web/search Perform a web search and return results using the v2 API # AI extract structured data from a web page Source: https://docs.weblinq.dev/api-reference/web/ai-extract-structured-data-from-a-web-page https://api.weblinq.dev/doc post /v1/web/ai-extract AI extract JSON data or text content using AI-powered parsing # Capture screenshot of a web page Source: https://docs.weblinq.dev/api-reference/web/capture-screenshot-of-a-web-page https://api.weblinq.dev/doc post /v1/web/screenshot Capture a screenshot of the specified URL with optional configuration # Extract links from a web page Source: https://docs.weblinq.dev/api-reference/web/extract-links-from-a-web-page https://api.weblinq.dev/doc post /v1/web/links Get all links from the specified URL with optional filtering # Extract markdown from a web page Source: https://docs.weblinq.dev/api-reference/web/extract-markdown-from-a-web-page https://api.weblinq.dev/doc post /v1/web/markdown Convert web page content to markdown format # Generate PDF from a web page Source: https://docs.weblinq.dev/api-reference/web/generate-pdf-from-a-web-page https://api.weblinq.dev/doc post /v1/web/pdf Convert a web page to PDF format # Get raw HTML content from a web page Source: https://docs.weblinq.dev/api-reference/web/get-raw-html-content-from-a-web-page https://api.weblinq.dev/doc post /v1/web/content Retrieve the raw HTML content of the specified URL # Scrape specific elements from a web page Source: https://docs.weblinq.dev/api-reference/web/scrape-specific-elements-from-a-web-page https://api.weblinq.dev/doc post /v1/web/scrape Extract specific HTML elements using CSS selectors # Search the web Source: https://docs.weblinq.dev/api-reference/web/search-the-web https://api.weblinq.dev/doc post /v1/web/search Perform a web search and return results # Authentication Source: https://docs.weblinq.dev/getting-started/authentication Secure your API requests with API key authentication WebLinq uses API key authentication to secure all requests. Every API call must include a valid API key to identify your account and track usage. All endpoints require authentication except health checks. This ensures your usage is tracked and your data remains secure. ## Get your API key [Sign up at WebLinq](https://weblinq.dev/sign-up) - it's free to get started. 1. Go to [API Keys](https://weblinq.dev/dashboard/api-keys) in your dashboard 2. Click "Create API Key" and give it a descriptive name 3. Copy your API key immediately API keys are only shown once. Store them securely in environment variables. Verify your API key works: ```bash cURL curl -H "Authorization: Bearer YOUR_API_KEY" \ "https://api.weblinq.dev/v1/user/me" ``` ```javascript JavaScript const response = await fetch('https://api.weblinq.dev/v1/user/me', { headers: { Authorization: 'Bearer YOUR_API_KEY', }, }); const data = await response.json(); console.log(data.success ? 'Authenticated!' : 'Failed'); ``` ```python Python import requests response = requests.get( 'https://api.weblinq.dev/v1/user/me', headers={'Authorization': 'Bearer YOUR_API_KEY'} ) print('Authenticated!' if response.ok else 'Failed') ``` ## Authentication methods ### Bearer token Include your API key in the `Authorization` header: ```bash cURL curl -X POST "https://api.weblinq.dev/v1/web/markdown" \ -H "Authorization: Bearer YOUR_API_KEY" \ -H "Content-Type: application/json" \ -d '{"url": "https://example.com"}' ``` ```javascript JavaScript const response = await fetch('https://api.weblinq.dev/v1/web/markdown', { method: 'POST', headers: { Authorization: 'Bearer YOUR_API_KEY', 'Content-Type': 'application/json', }, body: JSON.stringify({ url: 'https://example.com' }), }); ``` ```python Python import requests response = requests.post( 'https://api.weblinq.dev/v1/web/markdown', headers={'Authorization': 'Bearer YOUR_API_KEY'}, json={'url': 'https://example.com'} ) ``` Use `Authorization: Bearer` when possible - it's the standard approach and works with most HTTP libraries. ## Error responses ```json Unauthorized (401) { "success": false, "error": { "code": "AUTHENTICATION_REQUIRED", "message": "Authentication required", "requestId": "req_1234567890abcdef" }, "timestamp": "2024-01-15T10:30:00Z" } ``` ```json Rate Limited (429) { "success": false, "error": { "code": "RATE_LIMIT_EXCEEDED", "message": "Rate limit exceeded", "requestId": "req_1234567890abcdef" }, "timestamp": "2024-01-15T10:30:00Z" } ``` ## Rate limits API keys have different limits based on your plan: | Plan | Requests/Hour | Concurrent | Features | | -------------- | ------------- | ---------- | ----------------- | | **Free** | 1,000 | 5 | All endpoints | | **Pro** | 10,000 | 20 | Priority support | | **Enterprise** | Custom | Custom | Dedicated support | ### Rate limit headers Every response includes rate limit information: ```http X-RateLimit-Limit: 1000 X-RateLimit-Remaining: 999 X-RateLimit-Reset: 1705312200 ``` ## Security best practices ### Environment variables Always store API keys in environment variables: ```bash # .env file WEBLINQ_API_KEY=your_api_key_here ``` ```javascript const apiKey = process.env.WEBLINQ_API_KEY; ``` ```bash export WEBLINQ_API_KEY=your_api_key_here ``` ```python import os api_key = os.getenv('WEBLINQ_API_KEY') ``` ```dockerfile # Pass as environment variable docker run -e WEBLINQ_API_KEY=your_api_key_here your-app ``` ### Security checklist * Create separate keys for development and production * Rotate keys every 90 days * Use descriptive names to track key usage * Delete unused keys immediately * Store keys in environment variables or secrets managers - Never commit keys to version control - Don't embed keys in client-side code - Avoid using keys in URLs or query parameters * Monitor API usage in your dashboard * Set up alerts for unusual activity * Review key usage patterns regularly * Track failed authentication attempts **Security reminder:** Never share API keys, commit them to Git, or use them in client-side code. ## Need help? Email us for technical support Explore all available endpoints # Quickstart Guide Source: https://docs.weblinq.dev/getting-started/quickstart Get started with WebLinq in under 5 minutes - choose between MCP integration or direct API access The **Model Context Protocol (MCP)** provides seamless integration with AI applications like Claude Desktop, Cursor, and VS Code. This is the easiest way to use WebLinq tools. 1. [Sign up for WebLinq](https://weblinq.dev/sign-up) - it's free to start 2. Go to [API Keys](https://weblinq.dev/dashboard/api-keys) tab in your dashboard 3. Generate a new API key. Copy your API key immediately - you won't see it again after generation. WebLinq works with any MCP-compatible application. Choose your preferred client and add the configuration: Edit your `claude_desktop_config.json` file: ```json { "mcpServers": { "weblinq": { "command": "npx", "args": [ "-y", "mcp-remote@latest", "https://mcp.weblinq.dev/mcp", "--header", "Authorization:Bearer your_api_key_here" ] } } } ``` Add to your MCP settings in `.cursor/mcp.json`: ```json { "mcpServers": { "weblinq": { "command": "npx", "args": [ "-y", "mcp-remote@latest", "https://mcp.weblinq.dev/mcp", "--header", "Authorization:Bearer your_api_key_here" ] } } } ``` Configure in LM Studio's MCP settings (`mcp.json`): ```json { "mcpServers": { "weblinq": { "command": "npx", "args": [ "-y", "mcp-remote@latest", "https://mcp.weblinq.dev/mcp", "--header", "Authorization:Bearer your_api_key_here" ] } } } ``` **Perfect for**: Research with local models, extracting documentation, analyzing websites. Import configuration or manually add to BoltAI settings: ```json { "mcpServers": { "weblinq": { "command": "npx", "args": [ "-y", "mcp-remote@latest", "https://mcp.weblinq.dev/mcp", "--header", "Authorization:Bearer your_api_key_here" ] } } } ``` **Perfect for**: Quick web research, screenshot capture, content extraction across apps. Enable WebLinq as a plugin in TypingMind settings: ```json { "mcpServers": { "weblinq": { "command": "npx", "args": [ "-y", "mcp-remote@latest", "https://mcp.weblinq.dev/mcp", "--header", "Authorization:Bearer your_api_key_here" ] } } } ``` **Perfect for**: Content creation, market research, competitive analysis. Connect via command line interface for automation: ```bash npx mcpomni-connect \ --server https://mcp.weblinq.dev/mcp \ --header "Authorization:Bearer your_api_key_here" \ --model openai/gpt-4 ``` **Perfect for**: Automation scripts, batch processing, command-line workflows. Test WebLinq MCP tools directly in Postman for debugging: ```http POST https://mcp.weblinq.dev/mcp Authorization: Bearer your_api_key_here Content-Type: application/json { "jsonrpc": "2.0", "id": 1, "method": "tools/call", "params": { "name": "screenshot", "arguments": { "url": "https://example.com" } } } ``` **Perfect for**: API testing, debugging tool calls, validating responses. Replace `your_api_key_here` with your actual API key from the dashboard. Start using WebLinq tools naturally in conversation with your MCP client: ```text Claude Desktop - Website Cloning "Take a screenshot of https://stripe.com/pricing and extract the pricing table data. Then help me create a similar pricing page for my SaaS product." "Analyze the design patterns on https://linear.app and generate HTML/CSS code to recreate their dashboard layout." "Extract all the feature descriptions from https://notion.so and help me write compelling copy for my productivity app." ``` ```text Cursor/VS Code - Development "Screenshot https://github.com/vercel/next.js and extract the README content. Help me create a similar project structure and documentation." "Scrape the API documentation from https://docs.stripe.com/api and generate TypeScript interfaces for the payment endpoints." "Extract the component library from https://ui.shadcn.com and help me implement similar components in my React project." ``` ```text LM Studio - Research & Analysis "Search for recent developments in AI agents and summarize the top 5 trends with sources." "Extract content from 3 competitor websites and create a feature comparison matrix." "Take screenshots of mobile app landing pages and analyze their conversion optimization strategies." ``` ```text TypingMind - Content Creation "Extract the blog structure from https://blog.openai.com and help me create an editorial calendar with similar topics for my AI startup." "Screenshot successful product hunt launches and analyze their presentation patterns to improve my upcoming launch." "Scrape testimonials from SaaS websites and help me craft compelling social proof for my landing page." ``` ```text CLI/Automation - Batch Operations "Extract pricing data from 10 competitor websites and generate a competitive analysis report." "Take screenshots of all pages in my sitemap and create a visual site audit." "Monitor 5 industry blogs daily and extract new article summaries into a digest." ``` **Available tools:** * **screenshot** - Capture high-quality webpage screenshots * **extract\_markdown** - Convert any webpage to clean markdown * **ai\_extract** - Extract structured data using AI (pricing tables, features, etc.) * **web\_search** - Search the internet for real-time information * **scrape\_elements** - Extract specific elements using CSS selectors * **generate\_pdf** - Create PDF documents from web pages * **extract\_links** - Get all internal and external links * **extract\_youtube\_captions** - Get video transcripts and subtitles ### MCP Troubleshooting Common authentication problems and solutions: * Ensure the `--header` argument includes `Authorization:Bearer your_api_key_here` * Replace `your_api_key_here` with your actual API key * Check your configuration file syntax is valid JSON * Restart your MCP client after configuration changes WebLinq uses the modern **Streamable HTTP Transport**: - **Endpoint**: `https://mcp.weblinq.dev/mcp` - **Protocol**: MCP version 2025-03-26 - **Bridge Tool**: `mcp-remote` handles the connection - **Legacy Support**: `/sse` endpoint available for compatibility Troubleshooting tips: **Tools not appearing**: Check your config file syntax and restart your MCP client **API key errors**: Test with `curl -H "Authorization: Bearer YOUR_KEY" https://api.weblinq.dev/v1/web/markdown` **Connection timeouts**: Check internet connection and firewall settings Use WebLinq's REST API directly for custom integrations and applications. 1. [Sign up for WebLinq](https://weblinq.dev/sign-up) - it's free to start 2. Go to [API Keys](https://weblinq.dev/dashboard/api-keys) tab in your dashboard 3. Generate a new API key. Copy your API key immediately - you won't see it again after creation. Let's extract content from a webpage using the markdown endpoint: ```bash cURL curl -X POST "https://api.weblinq.dev/v1/web/markdown" \ -H "Authorization: Bearer YOUR_API_KEY" \ -H "Content-Type: application/json" \ -d '{"url": "https://example.com"}' ``` ```javascript JavaScript const response = await fetch('https://api.weblinq.dev/v1/web/markdown', { method: 'POST', headers: { Authorization: 'Bearer YOUR_API_KEY', 'Content-Type': 'application/json', }, body: JSON.stringify({ url: 'https://example.com', }), }); const data = await response.json(); console.log(data.data.markdown); ``` ```python Python import requests response = requests.post( 'https://api.weblinq.dev/v1/web/markdown', headers={'Authorization': 'Bearer YOUR_API_KEY'}, json={'url': 'https://example.com'} ) data = response.json() print(data['data']['markdown']) ``` Replace `YOUR_API_KEY` with the actual API key from your dashboard. You'll receive a JSON response with the extracted content: ```json { "success": true, "data": { "markdown": "# Example Domain\n\nThis domain is for use in illustrative examples...", "metadata": { "title": "Example Domain", "description": "This domain is for use in illustrative examples...", "url": "https://example.com", "timestamp": "2024-01-15T10:30:00Z", "wordCount": 42 } }, "creditsCost": 1 } ``` The `data.markdown` field contains the extracted content, while `metadata` provides additional page information. Now explore these popular endpoints: Capture page screenshots AI extract data Search the internet Create PDF documents ## General Tips Check these headers in API responses: * `X-RateLimit-Remaining`: Requests left in current window * `X-RateLimit-Reset`: When the limit resets Implement retry logic with exponential backoff for 429 responses. Always check the `success` field in responses: ```javascript if (!data.success) { console.error('API Error:', data.error.message); return; } // Process successful response console.log(data.data); ``` * Store API keys in environment variables * Never commit keys to version control * Use different keys for development and production * Rotate keys regularly ## What's Next? Browse complete API reference with all available endpoints Deep dive into Model Context Protocol fundamentals Best practices for secure WebLinq integration Real-world examples and implementation patterns ## Need help? Email us for technical support and questions Join the Model Context Protocol community # E-commerce Use Cases Source: https://docs.weblinq.dev/guides/e-commerce Extract and monitor e-commerce data using WebLinq APIs (schemas-ready). ## Product Data (Structured vs Text) ```javascript Product – JSON Schema const apiKey = process.env.WEBLINQ_API_KEY; const base = 'https://api.weblinq.dev/v1'; export async function extractProduct(url) { // minimal but useful schema const schema = { type: 'object', properties: { name: { type: 'string' }, price: { type: 'number' }, currency: { type: 'string' }, availability: { type: 'string' }, }, required: ['name', 'price', 'availability'], }; const { data: { extracted }, } = await post('/web/ai-extract', { url, response_format: { type: 'json_schema', json_schema: schema }, }); return extracted; } function post(p, b) { return fetch(`${base}${p}`, { method: 'POST', headers: { Authorization: `Bearer ${apiKey}`, 'Content-Type': 'application/json', }, body: JSON.stringify(b), }).then((r) => r.json()); } ``` ```javascript Product – Text Insights const apiKey = process.env.WEBLINQ_API_KEY; const base = 'https://api.weblinq.dev/v1'; export async function productInsights(url) { const { data: { text }, } = await post('/web/ai-extract', { url, responseType: 'text', // ✅ matches schema prompt: `Analyse this product page and list key selling points, drawbacks and target audience.`, }); return text; } function post(p, b) { /* same helper as above */ } ``` ```python Python – Schema vs Text import requests, os, json API = 'https://api.weblinq.dev/v1' KEY = os.environ['WEBLINQ_API_KEY'] def extract_both(url): schema = { "type":"object", "properties":{"name":{"type":"string"},"price":{"type":"number"}}, "required":["name","price"] } struct = req('/web/ai-extract', { "url": url, "response_format":{ "type":"json_schema","json_schema":schema } })['data']['extracted'] text = req('/web/ai-extract', { "url": url, "responseType":"text", "prompt":"Brief pros & cons." })['data']['text'] return { "structured":struct, "analysis":text } def req(path,payload): return requests.post(f'{API}{path}', headers={'Authorization':f'Bearer {KEY}','Content-Type':'application/json'}, json=payload).json() ``` *** ## Competitor Store Snapshot ```javascript Store Snapshot const apiKey = process.env.WEBLINQ_API_KEY; const base = 'https://api.weblinq.dev/v1'; export async function snapshotStore(storeUrl) { // ① screenshot const shot = post('/web/screenshot', { url: storeUrl, screenshotOptions: { fullPage: true }, }); // ② internal links const links = post('/web/links', { url: storeUrl, includeExternal: false }); // ③ quick store schema const schema = { type: 'object', properties: { storeName: { type: 'string' }, totalProducts: { type: 'number' }, }, }; const meta = post('/web/ai-extract', { url: storeUrl, response_format: { type: 'json_schema', json_schema: schema }, prompt: 'Store overview', }); const [screenshot, linkData, storeInfo] = await Promise.all([ shot, links, meta, ]); const productLinks = (linkData.data.links || []) .filter((l) => /\/(product|item|p)\//.test(l.url)) .slice(0, 20); return { url: storeUrl, screenshot: screenshot.data.permanentUrl, storeInfo: storeInfo.data.extracted, productLinks, }; } function post(p, b) { return fetch(`${base}${p}`, { method: 'POST', headers: { Authorization: `Bearer ${apiKey}`, 'Content-Type': 'application/json', }, body: JSON.stringify(b), }).then((r) => r.json()); } ``` *** ## Quick Price Report → PDF ```javascript Price Report (PDF) const apiKey = process.env.WEBLINQ_API_KEY; const base = 'https://api.weblinq.dev/v1'; export async function priceReport(urls) { const schema = { type: 'object', properties: { product: { type: 'string' }, price: { type: 'number' }, currency: { type: 'string' }, }, required: ['product', 'price', 'currency'], }; const rows = await Promise.all( urls.map(async (url) => { const { data: { extracted }, } = await post('/web/ai-extract', { url, response_format: { type: 'json_schema', json_schema: schema }, }); return { url, ...extracted }; }), ); const html = `

Price Report

${rows .map( (r) => ` `, ) .join('')}
${r.product}${r.currency}${r.price}${r.url}
`; const dataUri = 'data:text/html;base64,' + Buffer.from(html).toString('base64'); const { data: { permanentUrl }, } = await post('/web/pdf', { url: dataUri }); return { rows, pdf: permanentUrl }; } function post(p, b) { /* helper like above */ } ``` *** ## Product Discovery (Multi-Shop) ```javascript Product Discovery const apiKey = process.env.WEBLINQ_API_KEY; const base = 'https://api.weblinq.dev/v1'; export async function findProducts(query, limit = 10) { const { data: { results }, } = await post('/web/search', { query: `${query} site:amazon.com OR site:ebay.com OR site:walmart.com`, limit, }); const schema = { type: 'object', properties: { name: { type: 'string' }, price: { type: 'number' }, currency: { type: 'string' }, }, }; const products = await Promise.all( results.map((r) => post('/web/ai-extract', { url: r.url, response_format: { type: 'json_schema', json_schema: schema }, }).then((p) => ({ platform: new URL(r.url).hostname, ...p.data.extracted, })), ), ); return products.filter((p) => p.price).sort((a, b) => a.price - b.price); } function post(p, b) { /* helper */ } ``` *** ## Inventory Check (Stock Status) ```javascript Stock Checker const apiKey = process.env.WEBLINQ_API_KEY; const base = 'https://api.weblinq.dev/v1'; export async function checkStock(urls) { const schema = { type: 'object', properties: { stockStatus: { type: 'string' }, quantity: { type: 'number' }, }, }; const items = await Promise.all( urls.map(async (url) => { const { data: { extracted }, } = await post('/web/ai-extract', { url, response_format: { type: 'json_schema', json_schema: schema }, prompt: 'Current stock status', }); return { url, ...extracted }; }), ); return { items, outOfStock: items.filter((i) => i.stockStatus === 'out_of_stock').length, }; } function post(p, b) { /* helper */ } ``` *** ## Key Features Define exact data structures to extract consistent, structured product information every time. Combine search, links, ai-extract, screenshot, and PDF APIs for comprehensive e-commerce intelligence. Monitor competitor pricing, inventory, and promotions across multiple platforms automatically. Build historical datasets by regularly extracting product data and generating trend reports. * **Consistent data structure** needed across multiple products - **Integration with databases** or structured systems - **Automated processing** and comparison workflows - **Data validation** and quality assurance requirements * **Flexible analysis** and insights generation * **Natural language summaries** for human consumption * **Creative content generation** (reviews, descriptions) * **Complex reasoning** that doesn't fit rigid structures Always respect website terms of service and implement appropriate rate limiting when monitoring e-commerce sites. # Workflow Automation Source: https://docs.weblinq.dev/guides/examples Build powerful workflows by combining WebLinq APIs WebLinq endpoints are designed to snap together. Mix-and-match them to build rich automations. *** ## Complete Research Workflow Use this workflow to research a topic efficiently. It searches the web for relevant pages and then extracts key findings using structured or text-based prompts. You can choose between structured JSON results or quick text summaries. ```javascript Research (Structured) // Searches the web and extracts structured key findings const apiKey = process.env.WEBLINQ_API_KEY; const base = 'https://api.weblinq.dev/v1'; export async function researchTopic(query) { // 1️⃣ search const { data: { results }, } = await post('/web/search', { query, limit: 5 }); // 2️⃣ minimal schema const schema = { type: 'object', properties: { title: { type: 'string' }, keyFindings: { type: 'array', items: { type: 'string' } }, conclusions: { type: 'array', items: { type: 'string' } }, }, required: ['title', 'keyFindings'], }; // 3️⃣ structured extract const extracted = await Promise.all( results.map((r) => post('/web/ai-extract', { url: r.url, response_format: { type: 'json_schema', json_schema: schema }, prompt: `Key findings on "${query}"`, }), ), ); return results.map((r, i) => ({ url: r.url, ...extracted[i].data.extracted, })); } function post(p, body) { return fetch(`${base}${p}`, { method: 'POST', headers: { Authorization: `Bearer ${apiKey}`, 'Content-Type': 'application/json', }, body: JSON.stringify(body), }).then((r) => r.json()); } ``` ```javascript Research (Text Insights) // Quick text summaries of search results for a given query const apiKey = process.env.WEBLINQ_API_KEY; const base = 'https://api.weblinq.dev/v1'; export async function researchInsights(query) { const { data: { results }, } = await post('/web/search', { query, limit: 4 }); const insights = await Promise.all( results.map((r) => post('/web/ai-extract', { url: r.url, responseType: 'text', prompt: `Summarise important insights about "${query}".`, }).then((res) => ({ url: r.url, analysis: res.data.text })), ), ); return { query, insights }; } function post(p, body) { // same helper as above } ``` ```python Python Mini-Workflow # Python version of the structured research workflow import requests, os API = 'https://api.weblinq.dev/v1' KEY = os.environ['WEBLINQ_API_KEY'] def quick_research(q): search = req('/web/search', {'query': q, 'limit': 3})['data']['results'] schema = { "type":"object", "properties":{ "title":{"type":"string"}, "keyFindings":{"type":"array","items":{"type":"string"}} }, "required":["title","keyFindings"] } return [ req('/web/ai-extract',{ "url": r['url'], "response_format":{ "type":"json_schema", "json_schema":schema }, "prompt":f'Key findings on "{q}"' })['data']['extracted'] for r in search ] def req(path, payload): return requests.post( f'{API}{path}', headers={'Authorization':f'Bearer {KEY}','Content-Type':'application/json'}, json=payload ).json() ``` *** ## E-commerce Monitoring Workflow Track product listings on e-commerce sites. This workflow captures a screenshot of the page, extracts visible product links, and parses out product names and prices using a custom schema. ```javascript Product Monitor // Tracks products, captures screenshot, extracts data const apiKey = process.env.WEBLINQ_API_KEY; const base = 'https://api.weblinq.dev/v1'; export async function monitor(url) { const shot = await post('/web/screenshot', { url, screenshotOptions: { fullPage: true }, }); const { data: { links }, } = await post('/web/links', { url }); const productLinks = links .filter((l) => /\/product\//.test(l.url)) .slice(0, 10); const schema = { type: 'object', properties: { products: { type: 'array', items: { type: 'object', properties: { name: { type: 'string' }, price: { type: 'number' } }, }, }, }, }; const products = await post('/web/ai-extract', { url, response_format: { type: 'json_schema', json_schema: schema }, prompt: 'List products and prices you can see.', }); return { url, screenshot: shot.data.permanentUrl, productLinks, products: products.data.extracted, }; } function post(p, b) { // helper } ``` *** ## Content Creation Pipeline Turn webpages into content assets like summaries, PDFs, and Markdown. This is useful for archiving pages or preparing content for blog posts and reports. ```javascript Report Generator // Converts a list of URLs into PDFs, Markdown, and structured summaries const apiKey = process.env.WEBLINQ_API_KEY; const base = 'https://api.weblinq.dev/v1'; export async function generateReport(urls) { const schema = { type: 'object', properties: { title: { type: 'string' }, summary: { type: 'string' } }, }; return Promise.all( urls.map(async (url) => { const [md, pdf, info] = await Promise.all([ post('/web/markdown', { url }), post('/web/pdf', { url }), post('/web/ai-extract', { url, response_format: { type: 'json_schema', json_schema: schema }, prompt: 'Short summary', }), ]); return { url, title: info.data.extracted.title, summary: info.data.extracted.summary, pdf: pdf.data.permanentUrl, }; }), ); } function post(p, b) { // helper } ``` *** ## Hybrid Extraction (CSS + Schema) Combine raw scraping (CSS selectors) with structured schema-based extraction for more flexible data collection. ```javascript Hybrid Extractor // Scrapes with CSS selectors and adds structured schema extraction const apiKey = process.env.WEBLINQ_API_KEY; const base = 'https://api.weblinq.dev/v1'; export async function extractBusiness(url) { const scraped = await post('/web/scrape', { url, elements: [ { selector: '.contact-info', attributes: ['text'] }, { selector: '.business-hours', attributes: ['text'] }, ], }); const schema = { type: 'object', properties: { businessName: { type: 'string' }, phone: { type: 'string' } }, }; const { data: { extracted }, } = await post('/web/ai-extract', { url, response_format: { type: 'json_schema', json_schema: schema }, }); return { url, scraped: scraped.data.elements, structured: extracted }; } function post(p, b) { // helper } ``` *** ## Content Intelligence Design a content strategy based on what your competitors are doing. This automation searches for guides, analyzes them, and generates a content plan. ```javascript Content Strategy // Builds a content plan based on competitor analysis const apiKey = process.env.WEBLINQ_API_KEY; const base = 'https://api.weblinq.dev/v1'; export async function buildStrategy(topic) { // 1️⃣ search const { data: { results }, } = await post('/web/search', { query: `${topic} guide`, limit: 5 }); // 2️⃣ analyse competitors const schema = { type: 'object', properties: { wordCount: { type: 'number' }, keyTakeaways: { type: 'array', items: { type: 'string' } }, }, }; const analysis = await Promise.all( results.map((r) => post('/web/ai-extract', { url: r.url, response_format: { type: 'json_schema', json_schema: schema }, prompt: `Analyse strengths for "${topic}".`, }).then((a) => ({ url: r.url, ...a.data.extracted })), ), ); // 3️⃣ content plan (text mode) const { data: { text: strategy }, } = await post('/web/ai-extract', { url: 'data:text/plain;base64,' + Buffer.from(JSON.stringify(analysis)).toString('base64'), responseType: 'text', prompt: `Create a content plan for "${topic}" using this JSON.`, }); return { topic, analysis, strategy }; } function post(p, b) { // helper } ``` # News Extraction Source: https://docs.weblinq.dev/guides/news-extraction Extract and monitor news content using WebLinq API Extract clean, structured news content using WebLinq API workflows and JSON schemas. ## Comprehensive News Analysis Extract structured news data with precise JSON schemas and flexible text analysis: `````javascript Structured News Schema async function extractNewsArticle(articleUrl) { const apiKey = process.env.WEBLINQ_API_KEY; // Define comprehensive news article schema const newsSchema = { type: "object", properties: { headline: { type: "string" }, subheadline: { type: "string" }, author: { type: "object", properties: { name: { type: "string" }, email: { type: "string" }, twitter: { type: "string" } } }, publishDate: { type: "string", format: "date-time" }, lastModified: { type: "string", format: "date-time" }, category: { type: "string" }, tags: { type: "array", items: { type: "string" } }, summary: { type: "string", maxLength: 300 }, keyPoints: { type: "array", items: { type: "string" } }, quotes: { type: "array", items: { type: "object", properties: { text: { type: "string" }, speaker: { type: "string" }, role: { type: "string" } } } }, readingTime: { type: "number" }, wordCount: { type: "number" }, credibilityScore: { type: "number", minimum: 1, maximum: 10 }, sources: { type: "array", items: { type: "string" } } }, required: ["headline", "publishDate", "summary", "keyPoints"] }; const response = await fetch('https://api.weblinq.dev/v1/web/ai-extract', { method: 'POST', headers: { 'Authorization': `Bearer ${apiKey}`, 'Content-Type': 'application/json' }, body: JSON.stringify({ url: articleUrl, response_format: { type: "json_schema", json_schema: newsSchema }, prompt: "Extract comprehensive news article information including metadata, key points, and quotes" }) }); const data = await response.json(); if (!data.success) { throw new Error(data.error?.message || 'Failed to extract news data'); } return data.data.extracted; } ````javascript Editorial Analysis (Text) async function analyzeNewsEditorial(articleUrl) { const apiKey = process.env.WEBLINQ_API_KEY; // Use text response for nuanced editorial analysis const response = await fetch('https://api.weblinq.dev/v1/web/ai-extract', { method: 'POST', headers: { 'Authorization': `Bearer ${apiKey}`, 'Content-Type': 'application/json' }, body: JSON.stringify({ url: articleUrl, prompt: `Provide an in-depth editorial analysis of this news article: 1. **Bias Assessment**: Identify any potential bias in language, source selection, or framing 2. **Fact vs Opinion**: Distinguish between factual reporting and editorial opinion 3. **Completeness**: What important context or perspectives might be missing? 4. **Impact Analysis**: How might this story affect different stakeholders? 5. **Credibility Factors**: What makes this reporting trustworthy or questionable? 6. **Follow-up Questions**: What questions should readers ask or investigate further? Write this as a media literacy analysis that helps readers think critically about the information presented.`, response_format: { type: "text" } }) }); const data = await response.json(); if (!data.success) { throw new Error(data.error?.message || 'Failed to analyze article'); } return data.data.extracted; } ```javascript News Summary (Text) async function generateNewsSummary(articleUrl, audienceType = 'general') { const apiKey = process.env.WEBLINQ_API_KEY; const audiencePrompts = { general: "Write a clear, accessible summary for the general public", technical: "Write a detailed summary for industry professionals and experts", executive: "Write a concise executive briefing focusing on business implications", student: "Write an educational summary with background context for students" }; const response = await fetch('https://api.weblinq.dev/v1/web/ai-extract', { method: 'POST', headers: { 'Authorization': `Bearer ${apiKey}`, 'Content-Type': 'application/json' }, body: JSON.stringify({ url: articleUrl, prompt: `${audiencePrompts[audienceType]}. Include: - Main story in 2-3 sentences - Why this matters now - Key people/organizations involved - Potential implications or next steps - Background context if needed Keep it engaging and informative.`, response_format: { type: "text" } }) }); const data = await response.json(); if (!data.success) { throw new Error(data.error?.message || 'Failed to generate summary'); } return data.data.extracted; } ```python Python Dual Approach import requests def comprehensive_news_analysis(article_url): api_key = "your_api_key_here" # Structured data extraction news_schema = { "type": "object", "properties": { "headline": {"type": "string"}, "author": {"type": "string"}, "publishDate": {"type": "string"}, "category": {"type": "string"}, "summary": {"type": "string", "maxLength": 300}, "keyPoints": {"type": "array", "items": {"type": "string"}}, "quotes": { "type": "array", "items": { "type": "object", "properties": { "text": {"type": "string"}, "speaker": {"type": "string"} } } }, "readingTime": {"type": "number"}, "credibilityScore": {"type": "number", "minimum": 1, "maximum": 10} }, "required": ["headline", "publishDate", "summary"] } # Get structured metadata structured_response = requests.post( 'https://api.weblinq.dev/v1/web/ai-extract', headers={'Authorization': f'Bearer {api_key}', 'Content-Type': 'application/json'}, json={ 'url': article_url, 'response_format': {'type': 'json_schema', 'json_schema': news_schema}, 'prompt': 'Extract comprehensive news article information' } ) # Get contextual analysis analysis_response = requests.post( 'https://api.weblinq.dev/v1/web/ai-extract', headers={'Authorization': f'Bearer {api_key}', 'Content-Type': 'application/json'}, json={ 'url': article_url, 'prompt': '''Analyze this news story for: - Historical context and background - Potential long-term implications - Different stakeholder perspectives - Related stories or trends - Questions this raises for further investigation Provide thoughtful analysis that goes beyond just summarizing the facts.''', 'response_format': {'type': 'text'} } ) return { 'metadata': structured_response.json()['data']['extracted'], 'analysis': analysis_response.json()['data']['extracted'] } ````` ## News Monitoring Workflow Automated news monitoring combining search, extraction, and archival: ````javascript News Monitoring System async function monitorNewsTopics(topics, maxArticlesPerTopic = 5) { const apiKey = process.env.WEBLINQ_API_KEY; const baseUrl = 'https://api.weblinq.dev/v1'; const allArticles = []; for (const topic of topics) { // Step 1: Search for recent news on the topic const searchResponse = await fetch(`${baseUrl}/web/search`, { method: 'POST', headers: { 'Authorization': `Bearer ${apiKey}`, 'Content-Type': 'application/json' }, body: JSON.stringify({ query: `${topic} news site:cnn.com OR site:bbc.com OR site:reuters.com OR site:apnews.com`, limit: maxArticlesPerTopic }) }); const searchData = await searchResponse.json(); if (!searchData.success) { console.error(`Search failed for topic: ${topic}`); continue; } // Step 2: Extract clean content from each article for (const result of searchData.data.results) { try { // Get clean markdown content const markdownResponse = await fetch(`${baseUrl}/web/markdown`, { method: 'POST', headers: { 'Authorization': `Bearer ${apiKey}`, 'Content-Type': 'application/json' }, body: JSON.stringify({ url: result.url }) }); // Extract structured data const newsSchema = { type: "object", properties: { headline: { type: "string" }, author: { type: "string" }, publishDate: { type: "string" }, category: { type: "string" }, sentiment: { type: "string", enum: ["positive", "negative", "neutral"] }, keyPoints: { type: "array", items: { type: "string" } }, entities: { type: "object", properties: { people: { type: "array", items: { type: "string" } }, organizations: { type: "array", items: { type: "string" } }, locations: { type: "array", items: { type: "string" } } } }, credibilityScore: { type: "number", minimum: 1, maximum: 10 } } }; const extractResponse = await fetch(`${baseUrl}/web/ai-extract`, { method: 'POST', headers: { 'Authorization': `Bearer ${apiKey}`, 'Content-Type': 'application/json' }, body: JSON.stringify({ url: result.url, response_format: { type: "json_schema", json_schema: newsSchema }, prompt: `Analyze this news article about "${topic}" and extract key information` }) }); // Take screenshot for visual record const screenshotResponse = await fetch(`${baseUrl}/web/screenshot`, { method: 'POST', headers: { 'Authorization': `Bearer ${apiKey}`, 'Content-Type': 'application/json' }, body: JSON.stringify({ url: result.url, screenshotOptions: { fullPage: false, type: 'png' } }) }); const [markdown, extracted, screenshot] = await Promise.all([ markdownResponse.json(), extractResponse.json(), screenshotResponse.json() ]); if (extracted.success) { allArticles.push({ topic, url: result.url, source: new URL(result.url).hostname, searchTitle: result.title, content: markdown.success ? markdown.data.markdown : null, structured: extracted.data.extracted, screenshot: screenshot.success ? screenshot.data.permanentUrl : null, extractedAt: new Date().toISOString() }); } } catch (error) { console.error(`Error processing ${result.url}:`, error); } // Rate limiting await new Promise(resolve => setTimeout(resolve, 1500)); } } return { topics, totalArticles: allArticles.length, articles: allArticles, byTopic: topics.reduce((acc, topic) => { acc[topic] = allArticles.filter(article => article.topic === topic); return acc; }, {}), bySentiment: { positive: allArticles.filter(a => a.structured?.sentiment === 'positive'), negative: allArticles.filter(a => a.structured?.sentiment === 'negative'), neutral: allArticles.filter(a => a.structured?.sentiment === 'neutral') } }; } ```python Python News Extraction import requests def extract_news_article(article_url): api_key = "your_api_key_here" news_schema = { "type": "object", "properties": { "headline": {"type": "string"}, "author": {"type": "string"}, "publishDate": {"type": "string"}, "category": {"type": "string"}, "summary": {"type": "string", "maxLength": 300}, "keyPoints": {"type": "array", "items": {"type": "string"}}, "quotes": { "type": "array", "items": { "type": "object", "properties": { "text": {"type": "string"}, "speaker": {"type": "string"} } } }, "readingTime": {"type": "number"}, "credibilityScore": {"type": "number", "minimum": 1, "maximum": 10} }, "required": ["headline", "publishDate", "summary"] } response = requests.post( 'https://api.weblinq.dev/v1/web/ai-extract', headers={ 'Authorization': f'Bearer {api_key}', 'Content-Type': 'application/json' }, json={ 'url': article_url, 'response_format': { 'type': 'json_schema', 'json_schema': news_schema }, 'prompt': 'Extract comprehensive news article information' } ) data = response.json() if not data['success']: raise Exception(data.get('error', {}).get('message', 'Failed to extract')) return data['data']['extracted'] ```` ## Breaking News Alerts Real-time news monitoring with instant alerts: ```javascript Breaking News Monitor async function monitorBreakingNews(keywords, checkInterval = 300000) { // 5 minutes const apiKey = process.env.WEBLINQ_API_KEY; const baseUrl = 'https://api.weblinq.dev/v1'; let lastCheckTime = new Date(); const alertSchema = { type: "object", properties: { urgency: { type: "string", enum: ["low", "medium", "high", "critical"] }, headline: { type: "string" }, summary: { type: "string", maxLength: 200 }, impact: { type: "object", properties: { geographic: { type: "string" }, sectors: { type: "array", items: { type: "string" } }, stakeholders: { type: "array", items: { type: "string" } } } }, timeline: { type: "object", properties: { when: { type: "string" }, expectedDuration: { type: "string" } } }, actionRequired: { type: "boolean" } } }; async function checkForBreakingNews() { const searchQuery = keywords.map(keyword => `"${keyword}" breaking news OR "${keyword}" urgent OR "${keyword}" alert` ).join(' OR '); try { const searchResponse = await fetch(`${baseUrl}/web/search`, { method: 'POST', headers: { 'Authorization': `Bearer ${apiKey}`, 'Content-Type': 'application/json' }, body: JSON.stringify({ query: searchQuery + ` after:${lastCheckTime.toISOString().split('T')[0]}`, limit: 10 }) }); const searchData = await searchResponse.json(); if (!searchData.success) return; for (const result of searchData.data.results) { // Extract alert information const alertResponse = await fetch(`${baseUrl}/web/ai-extract`, { method: 'POST', headers: { 'Authorization': `Bearer ${apiKey}`, 'Content-Type': 'application/json' }, body: JSON.stringify({ url: result.url, response_format: { type: "json_schema", json_schema: alertSchema }, prompt: "Analyze if this is breaking news and extract urgency, impact, and action requirements" }) }); const alertData = await alertResponse.json(); if (alertData.success && alertData.data.extracted.urgency !== 'low') { await sendAlert({ ...alertData.data.extracted, url: result.url, source: new URL(result.url).hostname, detectedAt: new Date().toISOString() }); } await new Promise(resolve => setTimeout(resolve, 1000)); } lastCheckTime = new Date(); } catch (error) { console.error('Error checking breaking news:', error); } } async function sendAlert(alert) { console.log(`🚨 BREAKING NEWS ALERT - ${alert.urgency.toUpperCase()}`); console.log(`Headline: ${alert.headline}`); console.log(`Summary: ${alert.summary}`); console.log(`Source: ${alert.source}`); console.log(`URL: ${alert.url}`); if (alert.actionRequired) { console.log('⚠️ ACTION REQUIRED'); } // Here you would integrate with your notification system: // - Send email alerts // - Push notifications // - Slack/Discord webhooks // - SMS alerts for critical news } // Start monitoring console.log(`Starting breaking news monitoring for: ${keywords.join(', ')}`); // Initial check await checkForBreakingNews(); // Set up interval setInterval(checkForBreakingNews, checkInterval); } // Usage await monitorBreakingNews(['earthquake', 'market crash', 'cyber attack'], 300000); ``` ## News Aggregation Report Generate comprehensive news reports with PDF summaries: ````javascript News Report Generator async function generateNewsReport(topics, timeframe = '24h') { const apiKey = process.env.WEBLINQ_API_KEY; const baseUrl = 'https://api.weblinq.dev/v1'; const reportData = []; for (const topic of topics) { // Search for news on this topic const searchResponse = await fetch(`${baseUrl}/web/search`, { method: 'POST', headers: { 'Authorization': `Bearer ${apiKey}`, 'Content-Type': 'application/json' }, body: JSON.stringify({ query: `${topic} news`, limit: 8 }) }); const searchData = await searchResponse.json(); if (!searchData.success) continue; // Extract key information from top articles const topicArticles = []; for (const result of searchData.data.results.slice(0, 5)) { const summarySchema = { type: "object", properties: { headline: { type: "string" }, keyPoints: { type: "array", items: { type: "string" } }, impact: { type: "string" }, trend: { type: "string", enum: ["rising", "stable", "declining"] }, sentiment: { type: "string", enum: ["positive", "negative", "neutral"] } } }; const extractResponse = await fetch(`${baseUrl}/web/ai-extract`, { method: 'POST', headers: { 'Authorization': `Bearer ${apiKey}`, 'Content-Type': 'application/json' }, body: JSON.stringify({ url: result.url, response_format: { type: "json_schema", json_schema: summarySchema }, prompt: `Summarize this news article about "${topic}"` }) }); const extractData = await extractResponse.json(); if (extractData.success) { topicArticles.push({ url: result.url, source: new URL(result.url).hostname, ...extractData.data.extracted }); } await new Promise(resolve => setTimeout(resolve, 1000)); } reportData.push({ topic, articles: topicArticles, summary: generateTopicSummary(topicArticles) }); } // Create HTML report const reportHtml = generateReportHTML(reportData, timeframe); // Convert to PDF const dataUri = 'data:text/html;base64,' + Buffer.from(reportHtml).toString('base64'); const pdfResponse = await fetch(`${baseUrl}/web/pdf`, { method: 'POST', headers: { 'Authorization': `Bearer ${apiKey}`, 'Content-Type': 'application/json' }, body: JSON.stringify({ url: dataUri }) }); const pdf = await pdfResponse.json(); return { topics, reportData, pdfUrl: pdf.success ? pdf.data.permanentUrl : null, generatedAt: new Date().toISOString(), stats: { totalArticles: reportData.reduce((sum, topic) => sum + topic.articles.length, 0), sentimentBreakdown: calculateSentimentBreakdown(reportData), trendAnalysis: calculateTrendAnalysis(reportData) } }; } function generateReportHTML(reportData, timeframe) { return ` News Report - ${new Date().toLocaleDateString()}

News Report - ${timeframe}

Generated: ${new Date().toLocaleString()}

${reportData.map(topic => `

${topic.topic}

Articles Analyzed: ${topic.articles.length}

Overall Trend: ${topic.summary.trend}

Key Themes: ${topic.summary.themes.join(', ')}

${topic.articles.map(article => `

${article.headline}

Source: ${article.source}

Key Points:
    ${article.keyPoints.map(point => `
  • ${point}
    • `).join('')}

Impact: ${article.impact}

`).join('')} `).join('')} `; } ```python Python News Extraction import requests def extract_news_article(article_url): api_key = "your_api_key_here" news_schema = { "type": "object", "properties": { "headline": {"type": "string"}, "author": {"type": "string"}, "publishDate": {"type": "string"}, "category": {"type": "string"}, "summary": {"type": "string", "maxLength": 300}, "keyPoints": {"type": "array", "items": {"type": "string"}}, "quotes": { "type": "array", "items": { "type": "object", "properties": { "text": {"type": "string"}, "speaker": {"type": "string"} } } }, "readingTime": {"type": "number"}, "credibilityScore": {"type": "number", "minimum": 1, "maximum": 10} }, "required": ["headline", "publishDate", "summary"] } response = requests.post( 'https://api.weblinq.dev/v1/web/ai-extract', headers={ 'Authorization': f'Bearer {api_key}', 'Content-Type': 'application/json' }, json={ 'url': article_url, 'response_format': { 'type': 'json_schema', 'json_schema': news_schema }, 'prompt': 'Extract comprehensive news article information' } ) data = response.json() if not data['success']: raise Exception(data.get('error', {}).get('message', 'Failed to extract')) return data['data']['extracted'] ```` ## Real-time Source Monitoring Monitor specific news sources for new content: ```javascript Source Monitor async function monitorNewsSources(sources, categories = []) { const apiKey = process.env.WEBLINQ_API_KEY; const baseUrl = 'https://api.weblinq.dev/v1'; const results = []; for (const source of sources) { try { // Extract all news links from the homepage const linksResponse = await fetch(`${baseUrl}/web/links`, { method: 'POST', headers: { 'Authorization': `Bearer ${apiKey}`, 'Content-Type': 'application/json' }, body: JSON.stringify({ url: source.url, includeExternal: false, }) }); const linksData = await linksResponse.json(); if (!linksData.success) continue; // Filter for article links const articleLinks = linksData.data.links.filter(link => link.url.match(/\/(article|story|news|post)\//) || link.url.match(/\/\d{4}\/\d{2}\/\d{2}\//) // Date-based URLs ); // Extract headlines and metadata const sourceSchema = { type: "object", properties: { topStories: { type: "array", items: { type: "object", properties: { headline: { type: "string" }, category: { type: "string" }, publishTime: { type: "string" }, urgency: { type: "string", enum: ["low", "medium", "high"] } } } }, breakingNews: { type: "array", items: { type: "string" } }, trendingTopics: { type: "array", items: { type: "string" } } } }; const extractResponse = await fetch(`${baseUrl}/web/ai-extract`, { method: 'POST', headers: { 'Authorization': `Bearer ${apiKey}`, 'Content-Type': 'application/json' }, body: JSON.stringify({ url: source.url, response_format: { type: "json_schema", json_schema: sourceSchema }, prompt: "Extract current top stories, breaking news, and trending topics from this news homepage" }) }); const extractData = await extractResponse.json(); results.push({ source: source.name, url: source.url, articles: articleLinks.slice(0, 15), metadata: extractData.success ? extractData.data.extracted : null, scannedAt: new Date().toISOString() }); } catch (error) { console.error(`Error monitoring ${source.name}:`, error); } await new Promise(resolve => setTimeout(resolve, 2000)); } return { sources: sources.map(s => s.name), results, summary: { totalArticles: results.reduce((sum, r) => sum + r.articles.length, 0), breakingNewsCount: results.reduce((sum, r) => sum + (r.metadata?.breakingNews?.length || 0), 0 ), activeSources: results.filter(r => r.articles.length > 0).length } }; } // Usage with major news sources const newsSources = [ { name: 'CNN', url: 'https://cnn.com' }, { name: 'BBC', url: 'https://bbc.com/news' }, { name: 'Reuters', url: 'https://reuters.com' }, { name: 'AP News', url: 'https://apnews.com' } ]; const sourceUpdate = await monitorNewsSources(newsSources); ``` ## API Coverage for News Find the latest news articles across multiple sources and topics. Extract clean, readable content from news articles without ads or clutter. Use structured schemas to extract headlines, quotes, entities, and sentiment from news. Monitor news homepages and discover new articles as they're published. Capture visual records of breaking news for archival and verification. Generate comprehensive news reports and summaries for distribution. **News Monitoring Pro Tip**: Combine search for discovery, ai-extract for structured analysis, markdown for clean content, and PDF for professional reports. This creates a complete news intelligence pipeline. * **Database integration** for news aggregation systems * **Consistent metadata** across multiple sources * **Automated categorization** and tagging * **Structured fact-checking** workflows * **Search and filtering** by specific fields * **Editorial analysis** and bias detection * **Context and background** explanation * **Audience-specific summaries** (executive, general, technical) * **Creative rewrites** for different platforms * **Critical thinking** and media literacy analysis ``` ``` # Security Best Practices Source: https://docs.weblinq.dev/guides/security Essential security practices for WebLinq API usage # Security Best Practices Protect your API keys and data when using WebLinq. ## API Key Security Never expose API keys in client-side code, logs, or public repositories ### Environment Variables Store API keys securely: ```bash # .env file WEBLINQ_API_KEY=your_api_key_here ``` ```javascript // Always use environment variables const response = await fetch('https://api.weblinq.dev/v1/web/markdown', { method: 'POST', headers: { Authorization: `Bearer ${process.env.WEBLINQ_API_KEY}`, 'Content-Type': 'application/json', }, body: JSON.stringify({ url }), }); ``` Add `.env` to your `.gitignore` file ### Key Rotation Regularly rotate your API keys: 1. Generate new key in [dashboard](https://weblinq.dev/dashboard/api-keys) 2. Update your applications 3. Test thoroughly 4. Delete old key ## Request Validation ### URL Validation Always validate URLs before making requests: ```javascript function validateUrl(url) { try { const parsed = new URL(url); // Only allow HTTP/HTTPS if (!['http:', 'https:'].includes(parsed.protocol)) { throw new Error('Invalid protocol'); } // Block private networks const hostname = parsed.hostname; if ( hostname === 'localhost' || hostname.startsWith('127.') || hostname.startsWith('192.168.') || hostname.startsWith('10.') || hostname.startsWith('172.') ) { throw new Error('Private networks not allowed'); } return true; } catch (error) { throw new Error(`Invalid URL: ${error.message}`); } } ``` ### Input Sanitization Sanitize user inputs before processing: ```javascript function sanitizeInput(input) { // Remove potentially harmful characters return input .replace(/[<>]/g, '') // Remove HTML brackets .replace(/javascript:/gi, '') // Remove javascript: protocol .replace(/data:/gi, '') // Remove data: protocol .trim() .substring(0, 2000); // Limit length } ``` ## Response Handling ### Secure Content Processing Validate API responses before using them: ```javascript function validateResponse(response) { // Check response structure if (!response.success) { throw new Error(response.error?.message || 'API request failed'); } // Validate data exists if (!response.data) { throw new Error('No data in response'); } return response.data; } // Usage const response = await makeAPIRequest(url); const safeData = validateResponse(response); ``` ### Content Filtering Filter potentially malicious content: ```javascript function filterContent(content) { const suspiciousPatterns = [ /]*>/gi, /javascript:/gi, /on\w+\s*=/gi, // Event handlers like onclick= /]*>/gi, ]; for (const pattern of suspiciousPatterns) { if (pattern.test(content)) { console.warn('Suspicious content detected'); content = content.replace(pattern, ''); } } return content; } ``` ## Error Handling Never expose sensitive information in error messages: ```javascript async function safeAPICall(url) { try { validateUrl(url); const response = await fetch('https://api.weblinq.dev/v1/web/markdown', { method: 'POST', headers: { Authorization: `Bearer ${process.env.WEBLINQ_API_KEY}`, 'Content-Type': 'application/json', }, body: JSON.stringify({ url }), }); const data = await response.json(); return validateResponse(data); } catch (error) { // Log detailed error for debugging console.error('API call failed:', error); // Return safe error message to user throw new Error('Failed to process request'); } } ``` ## Access Control ### Server-Side Proxy Never call WebLinq API directly from client-side code: ```javascript // ❌ NEVER do this in client-side code // fetch('https://api.weblinq.dev/v1/web/markdown', { // headers: { 'Authorization': 'Bearer ' + API_KEY } // }); // ✅ Create a server-side proxy instead app.post('/api/extract-content', async (req, res) => { const { url } = req.body; try { validateUrl(url); const result = await callWebLinqAPI(url); res.json(result); } catch (error) { res.status(500).json({ error: 'Processing failed' }); } }); ``` ### Rate Limiting Implement request rate limiting in your application: ```javascript const rateLimit = require('express-rate-limit'); const limiter = rateLimit({ windowMs: 15 * 60 * 1000, // 15 minutes max: 10, // limit each IP to 10 requests per windowMs message: 'Too many requests from this IP', }); app.use('/api/extract-content', limiter); ``` ## Security Checklist Store in environment variables, never in code Check URLs and sanitize all user inputs Never expose API keys to client-side code Protect your endpoints from abuse Track API usage and watch for anomalies Rotate API keys periodically # Social Media Monitoring Source: https://docs.weblinq.dev/guides/social-media Monitor social media content using WebLinq API Monitor and analyze social media content using WebLinq API workflows and AI extract schemas. Always respect platform terms of service and rate limits ## Advanced Profile Analysis Extract comprehensive profile data with structured schemas and flexible text analysis: `````javascript Structured Profile Data async function analyzeProfile(profileUrl) { const apiKey = process.env.WEBLINQ_API_KEY; // Define comprehensive profile schema const profileSchema = { type: "object", properties: { profile: { type: "object", properties: { username: { type: "string" }, displayName: { type: "string" }, bio: { type: "string" }, followers: { type: "number" }, following: { type: "number" }, postsCount: { type: "number" }, verified: { type: "boolean" }, profileImage: { type: "string", format: "uri" }, joinDate: { type: "string" }, location: { type: "string" }, website: { type: "string", format: "uri" } } }, recentPosts: { type: "array", items: { type: "object", properties: { content: { type: "string" }, timestamp: { type: "string" }, likes: { type: "number" }, shares: { type: "number" }, comments: { type: "number" }, engagement: { type: "number" }, hashtags: { type: "array", items: { type: "string" } }, mentions: { type: "array", items: { type: "string" } }, mediaType: { type: "string", enum: ["text", "image", "video", "link"] } } } }, engagement: { type: "object", properties: { averageLikes: { type: "number" }, averageComments: { type: "number" }, engagementRate: { type: "number" }, postFrequency: { type: "string" } } }, topHashtags: { type: "array", items: { type: "string" } }, topMentions: { type: "array", items: { type: "string" } } }, required: ["profile", "recentPosts"] }; const response = await fetch('https://api.weblinq.dev/v1/web/ai-extract', { method: 'POST', headers: { 'Authorization': `Bearer ${apiKey}`, 'Content-Type': 'application/json' }, body: JSON.stringify({ url: profileUrl, response_format: { type: "json_schema", json_schema: profileSchema }, prompt: "Extract comprehensive social media profile information including recent posts and engagement metrics" }) }); const data = await response.json(); if (!data.success) { throw new Error(data.error?.message || 'Failed to extract profile data'); } return data.data.extracted; } ````javascript Content Strategy Analysis (Text) async function analyzeContentStrategy(profileUrl) { const apiKey = process.env.WEBLINQ_API_KEY; // Use text response for strategic insights const response = await fetch('https://api.weblinq.dev/v1/web/ai-extract', { method: 'POST', headers: { 'Authorization': `Bearer ${apiKey}`, 'Content-Type': 'application/json' }, body: JSON.stringify({ url: profileUrl, prompt: `Analyze this social media profile's content strategy and provide insights on: **Content Strategy Analysis:** 1. **Voice & Tone**: How does this account communicate? (professional, casual, humorous, etc.) 2. **Content Themes**: What topics and themes does this account focus on? 3. **Audience Engagement**: How does this account interact with and build community? 4. **Posting Strategy**: What patterns do you notice in timing, frequency, and content types? 5. **Brand Consistency**: How consistent is the messaging and visual identity? 6. **Growth Tactics**: What specific strategies are they using to grow their audience? 7. **Competitive Advantages**: What makes this account stand out from others in the space? 8. **Areas for Improvement**: What opportunities do you see for optimization? Provide actionable insights that could help improve social media strategy.`, response_format: { type: "text" } }) }); const data = await response.json(); if (!data.success) { throw new Error(data.error?.message || 'Failed to analyze strategy'); } return data.data.extracted; } ```javascript Audience Persona Analysis (Text) async function generateAudiencePersona(profileUrl) { const apiKey = process.env.WEBLINQ_API_KEY; const response = await fetch('https://api.weblinq.dev/v1/web/ai-extract', { method: 'POST', headers: { 'Authorization': `Bearer ${apiKey}`, 'Content-Type': 'application/json' }, body: JSON.stringify({ url: profileUrl, prompt: `Based on this social media profile's content, followers, and engagement patterns, create a detailed audience persona: **Audience Persona:** - **Demographics**: Age range, gender, location, income level - **Interests**: What topics, hobbies, and industries interest them? - **Pain Points**: What challenges or problems do they face? - **Content Preferences**: What types of content do they engage with most? - **Online Behavior**: How and when do they use social media? - **Goals & Motivations**: What are they trying to achieve? - **Influence Factors**: What drives their purchasing or engagement decisions? Write this as a comprehensive persona that could guide content creation and marketing strategies.`, response_format: { type: "text" } }) }); const data = await response.json(); if (!data.success) { throw new Error(data.error?.message || 'Failed to generate persona'); } return data.data.extracted; } ```python Python Comprehensive Analysis import requests def comprehensive_social_analysis(profile_url): api_key = "your_api_key_here" # Structured data extraction profile_schema = { "type": "object", "properties": { "profile": { "type": "object", "properties": { "username": {"type": "string"}, "displayName": {"type": "string"}, "bio": {"type": "string"}, "followers": {"type": "number"}, "following": {"type": "number"}, "verified": {"type": "boolean"} } }, "recentPosts": { "type": "array", "items": { "type": "object", "properties": { "content": {"type": "string"}, "likes": {"type": "number"}, "shares": {"type": "number"}, "hashtags": {"type": "array", "items": {"type": "string"}} } } }, "engagement": { "type": "object", "properties": { "averageLikes": {"type": "number"}, "engagementRate": {"type": "number"} } } }, "required": ["profile", "recentPosts"] } # Get structured profile data structured_response = requests.post( 'https://api.weblinq.dev/v1/web/ai-extract', headers={'Authorization': f'Bearer {api_key}', 'Content-Type': 'application/json'}, json={ 'url': profile_url, 'response_format': {'type': 'json_schema', 'json_schema': profile_schema}, 'prompt': 'Extract social media profile and engagement data' } ) # Get strategic content analysis strategy_response = requests.post( 'https://api.weblinq.dev/v1/web/ai-extract', headers={'Authorization': f'Bearer {api_key}', 'Content-Type': 'application/json'}, json={ 'url': profile_url, 'prompt': '''Analyze this social media account for: - Content strategy and messaging approach - Target audience and demographics - Competitive positioning and unique value - Growth tactics and engagement strategies - Brand voice and personality - Opportunities for improvement Provide strategic insights for social media optimization.''', 'response_format': {'type': 'text'} } ) return { 'profile_data': structured_response.json()['data']['extracted'], 'strategic_analysis': strategy_response.json()['data']['extracted'] } } ````` ## Brand Monitoring Workflow Comprehensive brand monitoring across social platforms: ```javascript Brand Monitoring System async function monitorBrandMentions(brandKeywords, platforms = ['twitter', 'linkedin', 'reddit']) { const apiKey = process.env.WEBLINQ_API_KEY; const baseUrl = 'https://api.weblinq.dev/v1'; const allMentions = []; for (const platform of platforms) { for (const keyword of brandKeywords) { try { // Step 1: Search for brand mentions const searchQuery = `"${keyword}" site:${platform}.com OR site:${platform}`; const searchResponse = await fetch(`${baseUrl}/web/search`, { method: 'POST', headers: { 'Authorization': `Bearer ${apiKey}`, 'Content-Type': 'application/json' }, body: JSON.stringify({ query: searchQuery, limit: 8 }) }); const searchData = await searchResponse.json(); if (!searchData.success) continue; // Step 2: Analyze each mention for sentiment and context for (const result of searchData.data.results.slice(0, 5)) { const mentionSchema = { type: "object", properties: { author: { type: "object", properties: { username: { type: "string" }, followers: { type: "number" }, verified: { type: "boolean" }, influence: { type: "string", enum: ["low", "medium", "high"] } } }, content: { type: "object", properties: { text: { type: "string" }, sentiment: { type: "string", enum: ["positive", "negative", "neutral"] }, topics: { type: "array", items: { type: "string" } }, urgency: { type: "string", enum: ["low", "medium", "high", "critical"] }, actionRequired: { type: "boolean" } } }, engagement: { type: "object", properties: { likes: { type: "number" }, shares: { type: "number" }, comments: { type: "number" }, reach: { type: "number" } } }, context: { type: "object", properties: { isComplaint: { type: "boolean" }, isPraise: { type: "boolean" }, isQuestion: { type: "boolean" }, competitorMention": { type: "boolean" } } } } }; const extractResponse = await fetch(`${baseUrl}/web/ai-extract`, { method: 'POST', headers: { 'Authorization': `Bearer ${apiKey}`, 'Content-Type': 'application/json' }, body: JSON.stringify({ url: result.url, response_format: { type: "json_schema", json_schema: mentionSchema }, prompt: `Analyze this social media post mentioning "${keyword}" for sentiment, urgency, and context` }) }); // Step 3: Take screenshot for visual record const screenshotResponse = await fetch(`${baseUrl}/web/screenshot`, { method: 'POST', headers: { 'Authorization': `Bearer ${apiKey}`, 'Content-Type': 'application/json' }, body: JSON.stringify({ url: result.url, screenshotOptions: { fullPage: false, type: 'png' } }) }); const [mentionData, screenshot] = await Promise.all([ extractResponse.json(), screenshotResponse.json() ]); if (mentionData.success) { allMentions.push({ keyword, platform, url: result.url, title: result.title, analyzed: mentionData.data.extracted, screenshot: screenshot.success ? screenshot.data.permanentUrl : null, foundAt: new Date().toISOString() }); } await new Promise(resolve => setTimeout(resolve, 1500)); } } catch (error) { console.error(`Error monitoring ${keyword} on ${platform}:`, error); } } } return { keywords: brandKeywords, platforms, mentions: allMentions, analysis: { totalMentions: allMentions.length, bySentiment: { positive: allMentions.filter(m => m.analyzed?.content?.sentiment === 'positive'), negative: allMentions.filter(m => m.analyzed?.content?.sentiment === 'negative'), neutral: allMentions.filter(m => m.analyzed?.content?.sentiment === 'neutral') }, urgentMentions: allMentions.filter(m => ['high', 'critical'].includes(m.analyzed?.content?.urgency) ), actionRequired: allMentions.filter(m => m.analyzed?.content?.actionRequired), byPlatform: platforms.reduce((acc, platform) => { acc[platform] = allMentions.filter(m => m.platform === platform); return acc; }, {}) } }; } ``` ## Competitor Analysis Dashboard Analyze competitor social media strategies: ```javascript Competitor Analysis async function analyzeCompetitorSocial(competitorProfiles) { const apiKey = process.env.WEBLINQ_API_KEY; const baseUrl = 'https://api.weblinq.dev/v1'; const competitorData = []; for (const competitor of competitorProfiles) { try { // Extract comprehensive competitor data const competitorSchema = { type: "object", properties: { profile: { type: "object", properties: { name: { type: "string" }, followers: { type: "number" }, following: { type: "number" }, postsPerWeek: { type: "number" }, verified: { type: "boolean" } } }, contentStrategy: { type: "object", properties: { primaryTopics: { type: "array", items: { type: "string" } }, contentTypes: { type: "array", items: { type: "string" } }, postingFrequency: { type: "string" }, bestPerformingContent: { type: "array", items: { type: "string" } } } }, engagement: { type: "object", properties: { averageLikes: { type: "number" }, averageComments: { type: "number" }, engagementRate: { type: "number" }, topHashtags: { type: "array", items: { type: "string" } } } }, trends: { type: "object", properties: { growthRate: { type: "string" }, engagementTrend: { type: "string", enum: ["rising", "stable", "declining"] }, contentGaps: { type: "array", items: { type: "string" } } } } } }; const extractResponse = await fetch(`${baseUrl}/web/ai-extract`, { method: 'POST', headers: { 'Authorization': `Bearer ${apiKey}`, 'Content-Type': 'application/json' }, body: JSON.stringify({ url: competitor.url, response_format: { type: "json_schema", json_schema: competitorSchema }, prompt: `Analyze this competitor's social media strategy, content performance, and growth trends` }) }); // Get links to recent posts for deeper analysis const linksResponse = await fetch(`${baseUrl}/web/links`, { method: 'POST', headers: { 'Authorization': `Bearer ${apiKey}`, 'Content-Type': 'application/json' }, body: JSON.stringify({ url: competitor.url, includeExternal: false, }) }); const [competitorAnalysis, recentLinks] = await Promise.all([ extractResponse.json(), linksResponse.json() ]); if (competitorAnalysis.success) { // Analyze recent posts for content strategy const postLinks = recentLinks.success ? recentLinks.data.links.filter(link => link.url.includes('/post/') || link.url.includes('/status/') ).slice(0, 5) : []; competitorData.push({ competitor: competitor.name, url: competitor.url, platform: competitor.platform, analysis: competitorAnalysis.data.extracted, recentPosts: postLinks, analyzedAt: new Date().toISOString() }); } } catch (error) { console.error(`Error analyzing ${competitor.name}:`, error); } await new Promise(resolve => setTimeout(resolve, 2000)); } // Generate competitive insights const insights = generateCompetitiveInsights(competitorData); return { competitors: competitorData, insights, summary: { totalCompetitors: competitorData.length, avgFollowers: competitorData.reduce((sum, c) => sum + (c.analysis?.profile?.followers || 0), 0) / competitorData.length, topPerformers: competitorData .sort((a, b) => (b.analysis?.engagement?.engagementRate || 0) - (a.analysis?.engagement?.engagementRate || 0)) .slice(0, 3), opportunities: insights.contentGaps, threats: insights.risingCompetitors } }; } function generateCompetitiveInsights(competitorData) { // Analyze patterns across competitors const allTopics = competitorData.flatMap(c => c.analysis?.contentStrategy?.primaryTopics || [] ); const topicCounts = allTopics.reduce((acc, topic) => { acc[topic] = (acc[topic] || 0) + 1; return acc; }, {}); return { commonTopics: Object.entries(topicCounts) .sort(([,a], [,b]) => b - a) .slice(0, 5) .map(([topic]) => topic), contentGaps: findContentGaps(competitorData), risingCompetitors: competitorData.filter(c => c.analysis?.trends?.engagementTrend === 'rising' ), bestPractices: extractBestPractices(competitorData) }; } ``` ## Influencer Discovery Find and analyze potential influencers in your niche: ```javascript Influencer Discovery async function discoverInfluencers(niche, platform = 'instagram', minFollowers = 10000) { const apiKey = process.env.WEBLINQ_API_KEY; const baseUrl = 'https://api.weblinq.dev/v1'; // Step 1: Search for potential influencers const searchQuery = `${niche} influencer site:${platform}.com`; const searchResponse = await fetch(`${baseUrl}/web/search`, { method: 'POST', headers: { 'Authorization': `Bearer ${apiKey}`, 'Content-Type': 'application/json' }, body: JSON.stringify({ query: searchQuery, limit: 15 }) }); const searchData = await searchResponse.json(); if (!searchData.success) { throw new Error('Search failed'); } const influencers = []; // Step 2: Analyze each potential influencer for (const result of searchData.data.results) { try { const influencerSchema = { type: "object", properties: { profile: { type: "object", properties: { username: { type: "string" }, displayName: { type: "string" }, bio: { type: "string" }, followers: { type: "number" }, following: { type: "number" }, postsCount: { type: "number" }, verified: { type: "boolean" }, niche: { type: "array", items: { type: "string" } } } }, engagement: { type: "object", properties: { averageLikes: { type: "number" }, averageComments: { type: "number" }, engagementRate": { type: "number" }, recentPostsPerformance: { type: "string" } } }, content: { type: "object", properties: { contentTypes: { type: "array", items: { type: "string" } }, postingFrequency: { type: "string" }, brandCollaborations: { type: "number" }, authenticity: { type: "string", enum: ["high", "medium", "low"] } } }, audience: { type: "object", properties: { demographics: { type: "string" }, interests: { type: "array", items: { type: "string" } }, engagement: { type: "string", enum: ["active", "moderate", "passive"] } } }, collaboration: { type: "object", properties: { contactInfo": { type: "string" }, rateRange: { type: "string" }, brandFit: { type: "number", minimum: 1, maximum: 10 }, availability: { type: "string", enum: ["available", "limited", "unavailable"] } } } } }; const extractResponse = await fetch(`${baseUrl}/web/ai-extract`, { method: 'POST', headers: { 'Authorization': `Bearer ${apiKey}`, 'Content-Type': 'application/json' }, body: JSON.stringify({ url: result.url, response_format: { type: "json_schema", json_schema: influencerSchema }, prompt: `Analyze this ${niche} influencer profile for collaboration potential, engagement quality, and audience fit` }) }); const influencerData = await extractResponse.json(); if (influencerData.success) { const profile = influencerData.data.extracted; // Filter by minimum followers if (profile.profile?.followers >= minFollowers) { influencers.push({ url: result.url, title: result.title, platform, ...profile, discoveredAt: new Date().toISOString(), searchRank: influencers.length + 1 }); } } } catch (error) { console.error(`Error analyzing influencer ${result.url}:`, error); } await new Promise(resolve => setTimeout(resolve, 1500)); } // Rank influencers by collaboration potential const rankedInfluencers = influencers.sort((a, b) => { const scoreA = calculateInfluencerScore(a); const scoreB = calculateInfluencerScore(b); return scoreB - scoreA; }); return { niche, platform, totalFound: rankedInfluencers.length, influencers: rankedInfluencers, topPicks: rankedInfluencers.slice(0, 5), insights: { avgFollowers: rankedInfluencers.reduce((sum, inf) => sum + (inf.profile?.followers || 0), 0) / rankedInfluencers.length, avgEngagement: rankedInfluencers.reduce((sum, inf) => sum + (inf.engagement?.engagementRate || 0), 0) / rankedInfluencers.length, commonInterests: findCommonInterests(rankedInfluencers), priceRanges: analyzePriceRanges(rankedInfluencers) } }; } function calculateInfluencerScore(influencer) { let score = 0; // Engagement rate (40% weight) const engagementRate = influencer.engagement?.engagementRate || 0; score += (engagementRate \* 40); // Brand fit (30% weight) const brandFit = influencer.collaboration?.brandFit || 0; score += (brandFit \* 3); // Authenticity (20% weight) const authenticity = influencer.content?.authenticity; if (authenticity === 'high') score += 20; else if (authenticity === 'medium') score += 10; // Availability (10% weight) if (influencer.collaboration?.availability === 'available') score += 10; else if (influencer.collaboration?.availability === 'limited') score += 5; return score; } ``` ## Social Media Reporting Generate comprehensive social media reports: ```javascript Social Media Reports async function generateSocialMediaReport(profiles, timeframe = 'monthly') { const apiKey = process.env.WEBLINQ_API_KEY; const baseUrl = 'https://api.weblinq.dev/v1'; const reportData = []; for (const profile of profiles) { // Extract detailed analytics const analyticsSchema = { type: "object", properties: { overview: { type: "object", properties: { totalPosts: { type: "number" }, totalLikes: { type: "number" }, totalComments: { type: "number" }, totalShares: { type: "number" }, followerGrowth: { type: "number" }, engagementRate: { type: "number" } } }, topPerformingPosts: { type: "array", items: { type: "object", properties: { content: { type: "string" }, likes: { type: "number" }, engagement: { type: "number" }, contentType: { type: "string" } } } }, contentAnalysis: { type: "object", properties: { mostUsedHashtags: { type: "array", items: { type: "string" } }, contentTypes: { type: "array", items: { type: "string" } }, postingTimes: { type: "array", items: { type: "string" } }, engagementPatterns: { type: "string" } } }, audienceInsights: { type: "object", properties: { demographics: { type: "string" }, interests: { type: "array", items: { type: "string" } }, activeHours: { type: "array", items: { type: "string" } }, growthTrends: { type: "string" } } } } }; const analyticsResponse = await fetch(`${baseUrl}/web/ai-extract`, { method: 'POST', headers: { 'Authorization': `Bearer ${apiKey}`, 'Content-Type': 'application/json' }, body: JSON.stringify({ url: profile.url, response_format: { type: "json_schema", json_schema: analyticsSchema }, prompt: `Extract ${timeframe} social media analytics and performance metrics for this profile` }) }); const analytics = await analyticsResponse.json(); if (analytics.success) { reportData.push({ profile: profile.name, platform: profile.platform, url: profile.url, analytics: analytics.data.extracted, reportPeriod: timeframe }); } await new Promise(resolve => setTimeout(resolve, 1000)); } // Create comprehensive report const reportHtml = generateSocialReportHTML(reportData, timeframe); // Convert to PDF const dataUri = 'data:text/html;base64,' + Buffer.from(reportHtml).toString('base64'); const pdfResponse = await fetch(`${baseUrl}/web/pdf`, { method: 'POST', headers: { 'Authorization': `Bearer ${apiKey}`, 'Content-Type': 'application/json' }, body: JSON.stringify({ url: dataUri }) }); const pdf = await pdfResponse.json(); return { profiles: profiles.map(p => p.name), timeframe, reportData, pdfUrl: pdf.success ? pdf.data.permanentUrl : null, summary: { totalProfiles: reportData.length, avgEngagement: calculateAverageEngagement(reportData), topPerformer: findTopPerformer(reportData), insights: generateSocialInsights(reportData) }, generatedAt: new Date().toISOString() }; } function generateSocialReportHTML(reportData, timeframe) { return ` Social Media Report - ${timeframe}

Social Media Performance Report

Report Period: ${timeframe}

Generated: ${new Date().toLocaleString()}

${reportData.map(profile => `

${profile.profile} (${profile.platform})

Total Posts
${profile.analytics?.overview?.totalPosts || 'N/A'}
Total Likes
${profile.analytics?.overview?.totalLikes || 'N/A'}
Engagement Rate
${profile.analytics?.overview?.engagementRate || 'N/A'}%
Follower Growth
${profile.analytics?.overview?.followerGrowth || 'N/A'}

Top Performing Content

${profile.analytics?.topPerformingPosts?.map(post => `

Content: ${post.content}

Likes: ${post.likes} | Type: ${post.contentType}

`).join('') || '

No data available

'}

Content Insights

Top Hashtags: ${profile.analytics?.contentAnalysis?.mostUsedHashtags?.join(', ') || 'N/A'}

Content Types: ${profile.analytics?.contentAnalysis?.contentTypes?.join(', ') || 'N/A'}

Best Posting Times: ${profile.analytics?.contentAnalysis?.postingTimes?.join(', ') || 'N/A'}

`).join('')} `; } ``` ## Real-time Trend Monitoring Monitor social media trends and viral content: ```javascript Trend Monitoring async function monitorTrends(platforms = ['twitter', 'tiktok', 'instagram']) { const apiKey = process.env.WEBLINQ_API_KEY; const baseUrl = 'https://api.weblinq.dev/v1'; const trends = []; for (const platform of platforms) { try { // Search for trending content const searchResponse = await fetch(`${baseUrl}/web/search`, { method: 'POST', headers: { 'Authorization': `Bearer ${apiKey}`, 'Content-Type': 'application/json' }, body: JSON.stringify({ query: `trending viral ${platform} today`, limit: 10 }) }); const searchData = await searchResponse.json(); if (!searchData.success) continue; for (const result of searchData.data.results.slice(0, 5)) { const trendSchema = { type: "object", properties: { trend: { type: "object", properties: { topic: { type: "string" }, hashtags: { type: "array", items: { type: "string" } }, description: { type: "string" }, viralityScore: { type: "number", minimum: 1, maximum: 10 }, category: { type: "string" }, demographics: { type: "string" } } }, metrics: { type: "object", properties: { estimatedReach: { type: "number" }, engagementLevel: { type: "string", enum: ["low", "medium", "high", "viral"] }, growthRate: { type: "string" }, peakTime: { type: "string" } } }, relevance: { type: "object", properties: { brandOpportunity: { type: "number", minimum: 1, maximum: 10 }, riskLevel: { type: "string", enum: ["low", "medium", "high"] }, actionable: { type: "boolean" }, timeframe: { type: "string" } } } } }; const trendResponse = await fetch(`${baseUrl}/web/ai-extract`, { method: 'POST', headers: { 'Authorization': `Bearer ${apiKey}`, 'Content-Type': 'application/json' }, body: JSON.stringify({ url: result.url, response_format: { type: "json_schema", json_schema: trendSchema }, prompt: `Analyze this trending social media content for virality, engagement, and brand opportunity` }) }); const trendData = await trendResponse.json(); if (trendData.success) { trends.push({ platform, url: result.url, title: result.title, ...trendData.data.extracted, discoveredAt: new Date().toISOString() }); } await new Promise(resolve => setTimeout(resolve, 1000)); } } catch (error) { console.error(`Error monitoring trends on ${platform}:`, error); } } return { platforms, trends: trends.sort((a, b) => (b.trend?.viralityScore || 0) - (a.trend?.viralityScore || 0) ), opportunities: trends.filter(t => t.relevance?.brandOpportunity >= 7 && t.relevance?.riskLevel !== 'high' ), alerts: trends.filter(t => t.metrics?.engagementLevel === 'viral' || t.trend?.viralityScore >= 8 ), summary: { totalTrends: trends.length, viralContent: trends.filter(t => t.metrics?.engagementLevel === 'viral').length, highOpportunity: trends.filter(t => t.relevance?.brandOpportunity >= 8).length, platforms: platforms.reduce((acc, platform) => { acc[platform] = trends.filter(t => t.platform === platform).length; return acc; }, {}) } }; } ``` ## API Coverage for Social Media Discover social media mentions, influencers, and trending content across platforms. Extract structured data from social profiles, posts, and engagement metrics using JSON schemas. Monitor social media pages for new posts and discover viral content as it emerges. Capture visual proof of social media mentions, posts, and trending content for reports. Generate professional social media reports, influencer profiles, and trend analyses. Extract clean text content from social posts for sentiment analysis and content research. Always respect platform terms of service and rate limits when monitoring social media sites. **Social Media Pro Tip**: Combine search for discovery, ai-extract with schemas for structured analysis, screenshot for visual documentation, and PDF for professional reporting. This creates a complete social media intelligence system. * **CRM integration** for lead tracking and customer data * **Analytics dashboards** with consistent metrics * **Automated monitoring** and alert systems * **Competitive benchmarking** with standardized data * **Performance tracking** over time * **Strategic insights** and content recommendations * **Audience persona development** and targeting * **Brand voice analysis** and consistency checks * **Competitive intelligence** and positioning * **Creative content ideas** and campaign concepts ``` ``` # WebLinq API Documentation Source: https://docs.weblinq.dev/index Extract data, capture screenshots, and search the internet with our web scraping and browser automation platform **New to WebLinq?** Get your first API response in under 5 minutes with our [quickstart guide](/getting-started/quickstart). ## What you can build Convert any webpage to clean Markdown format Capture high-quality screenshots with custom viewport settings Generate PDFs from any webpage AI extract structured data Extract specific elements using CSS selectors Extract all internal and external links Get raw HTML content from any page Search the web programmatically ## Common use cases Build content feeds, news aggregators, or monitoring systems. Create visual documentation, page previews, or archival systems. Chain search → markdown → JSON extraction for instant research briefs Track prices, inventory and promotions across entire stores Generate winning content strategies from competitive analysis Auto-create PDF price reports and highlight the best deals ## Need help? Email us for technical support and questions See real-world integration examples # Privacy Policy Source: https://docs.weblinq.dev/legal/privacy-policy Comprehensive privacy policy for Weblinq API - detailing data collection, processing, and protection practices for web scraping and browser automation services **Last Updated:** June 2025 This Privacy Policy explains how Weblinq collects, uses, and protects your personal information when you use our web scraping and browser automation services. ## 1. Introduction and Scope ### 1.1 Our Commitment to Privacy At Weblinq, we are committed to protecting your privacy and handling your personal data with transparency and respect. This Privacy Policy describes how we collect, use, store, and protect your information when you use our Services. ### 1.2 Who We Are **Weblinq** is a cloud-based API platform providing web scraping, browser automation, and data extraction services. We act as both a data controller and data processor depending on the context of data handling. **Data Controller:** When we collect and process your account information, usage data, and billing details. **Data Processor:** When we process data on your behalf through our scraping and automation services. ### 1.3 Scope of This Policy This Privacy Policy applies to: * Our website (weblinq.dev) * The Weblinq API platform and services * Customer support interactions * Marketing communications * Any other services we provide ## 2. Information We Collect ### 2.1 Information You Provide Directly We collect information that you voluntarily provide when using our Services: * **Personal Details:** Name, email address, company name * **Authentication:** Username, password (encrypted) * **Profile Data:** Job title, industry, use case descriptions * **Communication:** Support tickets, feedback, and correspondence * **Payment Details:** Billing address, VAT number, tax information - **Transaction Data:** Payment history, subscription details, usage billing - **Financial Records:** Invoices, receipts, payment method information * **API Settings:** Endpoint configurations, request parameters * **Automation Scripts:** Custom scraping rules and workflow definitions * **Integration Data:** Third-party service connections and webhooks ### 2.2 Information We Collect Automatically When you use our Services, we automatically collect certain information: * **API Requests:** Request frequency, endpoints accessed, response times * **Service Performance:** Success rates, error logs, system performance metrics * **Feature Usage:** Which features you use and how frequently * **Session Data:** Login times, session duration, user interactions * **Device Data:** IP address, browser type, operating system, device identifiers - **Network Information:** ISP, geographic location (country/region level) - **Log Data:** Server logs, error reports, debugging information - **Cookies:** Session cookies, preference cookies, analytics cookies * **Target URLs:** Websites and endpoints you scrape through our service * **Extracted Content:** Data retrieved on your behalf (temporary storage only) * **Processing Logs:** Records of scraping activities for debugging and optimization ### 2.3 Information from Third Parties We may receive information from third-party sources: Stripe provides payment confirmation and billing information OAuth providers (Google, GitHub) for account creation and login Google Analytics provides website usage insights (anonymized) Referral partners may provide contact information for leads ## 3. How We Use Your Information ### 3.1 Primary Service Purposes We use your information to provide and improve our Services: * **Account Management:** Create and maintain your account * **API Operations:** Process your scraping and automation requests * **Data Processing:** Extract, transform, and deliver data as requested * **Infrastructure Management:** Maintain servers and ensure service availability * **Technical Support:** Troubleshoot issues and provide assistance - **Documentation:** Provide relevant guides and best practices - **Feature Requests:** Understand and implement user needs - **Service Updates:** Communicate changes and improvements * **Payment Processing:** Handle subscriptions and transaction processing * **Usage Tracking:** Monitor API usage for billing and limits * **Compliance:** Meet legal and regulatory requirements * **Record Keeping:** Maintain necessary business and tax records ### 3.2 Service Improvement and Analytics All analytics and improvement activities use aggregated, anonymized data that cannot identify individual users or specific scraping targets. We analyze usage patterns to: * **Performance Optimization:** Improve response times and reliability * **Feature Development:** Build new capabilities based on user needs * **Security Enhancement:** Detect and prevent abuse or security threats * **Market Research:** Understand industry trends and user behavior ### 3.3 Legal Basis for Processing (GDPR) Our legal basis for processing personal data varies by purpose: Processing required to provide our Services, including account management, service delivery, and billing. Improving our Services, security monitoring, and fraud prevention, balanced against your privacy rights. Meeting regulatory requirements, tax obligations, and responding to lawful requests. Marketing communications, optional analytics, and non-essential cookies (where required by law). ## 4. Data Sharing and Disclosure ### 4.1 Service Providers and Partners We share information with trusted third parties who help us provide our Services: * **Cloud Hosting:** AWS, Google Cloud Platform for service hosting * **CDN Services:** CloudFlare for content delivery and security * **Monitoring:** DataDog, New Relic for system monitoring and performance * **Security:** Third-party security services for threat detection * **Payment Processing:** Stripe for billing and payment handling - **Email Services:** SendGrid, Mailgun for transactional emails - **Customer Support:** Intercom, Zendesk for support ticket management - **Analytics:** Google Analytics (anonymized), Mixpanel for product analytics * **Legal Advisors:** Lawyers and legal counsel for compliance matters * **Accountants:** Financial advisors for tax and accounting services * **Auditors:** Security and compliance auditors for certifications * **Consultants:** Technical and business consultants for specific projects ### 4.2 Legal Requirements and Protection We may disclose information when required by law or to protect our rights: We will notify you of legal requests for your data when legally permitted and feasible to do so. **Legal Disclosure Scenarios:** * **Government Requests:** Subpoenas, court orders, and regulatory investigations * **Law Enforcement:** Criminal investigations and national security matters * **Compliance:** Tax audits, regulatory compliance, and legal obligations * **Rights Protection:** Protecting our intellectual property and enforcing our Terms ### 4.3 Business Transfers In the event of a merger, acquisition, or sale of assets, your information may be transferred to the new entity, subject to the same privacy protections outlined in this policy. ## 5. Data Security and Protection ### 5.1 Technical Safeguards We implement comprehensive security measures to protect your data: * **Data in Transit:** TLS 1.3 encryption for all API communications * **Data at Rest:** AES-256 encryption for stored data * **Database Security:** Encrypted database storage with access controls * **Key Management:** Secure key rotation and management practices * **Authentication:** Multi-factor authentication for admin access - **Authorization:** Role-based access control with principle of least privilege - **Network Security:** VPN access, firewalls, and network segmentation - **Audit Logging:** Comprehensive logging of all data access and modifications * **Server Hardening:** Regular security updates and configuration management * **Vulnerability Management:** Regular security scans and penetration testing * **Incident Response:** 24/7 monitoring and incident response procedures * **Backup Security:** Encrypted backups with tested recovery procedures ### 5.2 Organizational Safeguards Regular privacy and security training for all employees handling personal data We collect and retain only the minimum data necessary for our Services Privacy considerations are built into all new features and systems All third-party vendors undergo security assessments and sign data processing agreements ## 6. Your Privacy Rights ### 6.1 Universal Rights Regardless of your location, you have the following rights: * **Right to Know:** What personal data we collect and how we use it * **Data Access:** Request a copy of your personal data in a readable format * **Processing Activities:** Information about who we share your data with and why * **Data Correction:** Update or correct inaccurate personal information - **Account Management:** Modify your account settings and preferences - **Communication Preferences:** Opt out of marketing communications * **Data Deletion:** Request deletion of your personal data (subject to legal requirements) * **Data Portability:** Receive your data in a machine-readable format * **Service Termination:** Close your account and delete associated data ### 6.2 GDPR Rights (EU/EEA Residents) If you are located in the European Union or European Economic Area, you have additional rights under the General Data Protection Regulation (GDPR). **Enhanced GDPR Rights:** * **Right to Rectification:** Correct inaccurate personal data * **Right to Erasure:** Request deletion of personal data ("right to be forgotten") * **Right to Restrict Processing:** Limit how we process your data * **Right to Object:** Object to processing based on legitimate interests * **Right to Data Portability:** Receive your data and transfer it to another service * **Right to Withdraw Consent:** Withdraw consent for processing at any time * **Right to Lodge Complaints:** File complaints with your local data protection authority ### 6.3 CCPA Rights (California Residents) **California Consumer Privacy Act (CCPA) Rights:** * **Right to Know:** Categories and specific personal information we collect * **Right to Delete:** Request deletion of personal information * **Right to Opt-Out:** Opt out of the "sale" of personal information (we don't sell data) * **Right to Non-Discrimination:** Equal service regardless of privacy choices ### 6.4 Exercising Your Rights To exercise any of these rights: Contact us at [privacy@weblinq.dev](mailto:privacy@weblinq.dev) with your specific request We may ask for information to verify your identity and account ownership We will process your request within 30 days (or as required by local law) We will provide confirmation of actions taken or explanations if requests cannot be fulfilled ## 7. Data Retention and International Transfers ### 7.1 Data Retention Periods We retain different types of data for varying periods: **Account Information:** Retained while your account is active, plus 7 years after deletion for legal compliance **Billing Records:** Retained for 7 years after final transaction for tax and audit requirements **API Usage Logs:** Retained for 90 days for debugging and optimization **Scraped Content:** Temporary cache only, deleted within 24-48 hours ### 7.2 International Data Transfers When we transfer data internationally, we use appropriate safeguards: Transfers to countries with EU adequacy decisions EU-approved contract terms for secure transfers ## 8. Cookies and Tracking ### 8.1 Cookie Types We use different categories of cookies: Required for basic website functionality and service operation Help us understand how our Services are used (anonymized data) Remember your settings and preferences ### 8.2 Managing Cookies You can control cookies through your browser settings, though disabling essential cookies may affect functionality. ## 9. Updates and Contact Information ### 9.1 Policy Updates We may update this Privacy Policy from time to time. Material changes will be communicated via email and our website with at least 30 days notice. ### 9.2 Contact Us For privacy-related questions or requests: [privacy@weblinq.dev](mailto:privacy@weblinq.dev) *** This Privacy Policy is effective as of the date listed above. By using our Services, you acknowledge that you have read and understand this Privacy Policy. # Terms of Service Source: https://docs.weblinq.dev/legal/terms Complete terms of service for Weblinq API - covering web scraping, browser automation, and data extraction services **Last Updated:** June 2025 These Terms of Service ("Terms") govern your access to and use of Weblinq API services. By using our services, you agree to be bound by these Terms. ## 1. Overview and Acceptance ### 1.1 Agreement Scope These Terms of Service ("Terms") constitute a legal agreement between you ("Customer", "User", or "you") and Weblinq ("Company", "we", "us", or "our") governing your access to and use of the Weblinq API services, including web scraping, browser automation, data extraction, and related functionalities (the "Services"). ### 1.2 Acceptance By accessing or using our Services, you acknowledge that you: * Have read and understood these Terms * Agree to be bound by these Terms and our Privacy Policy * Have the legal authority to enter into this agreement * Are using the Services for professional or business purposes only If you do not agree to these Terms, you must not access or use our Services. ### 1.3 Updates to Terms We may modify these Terms at any time. Material changes will be communicated via email or through our platform. Continued use of our Services after such modifications constitutes acceptance of the updated Terms. ## 2. Description of Services ### 2.1 Core Services Weblinq provides cloud-based API services for: Extract structured and unstructured data from publicly accessible websites and web applications. Automate browser interactions, including form submissions, navigation, and dynamic content handling. Transform, clean, and format extracted data according to your specifications. Route requests through our managed proxy infrastructure to ensure reliability and compliance. ### 2.2 Service Availability Services are provided on a "Software as a Service" basis and are generally available 24/7, subject to scheduled maintenance and unforeseen technical issues. We strive to maintain high availability but do not guarantee uninterrupted service. Planned maintenance will be communicated in advance when possible. ## 3. Account Registration and Access ### 3.1 Account Creation To use our Services, you must: * Create an account with valid and accurate information * Provide a working email address for communications * Choose a secure password and maintain its confidentiality * Accept responsibility for all activities under your account ### 3.2 Account Types We offer different account tiers with varying features and usage limits: Limited requests for evaluation and small-scale projects Scaled usage limits for production applications Custom limits and dedicated support for large-scale operations Tailored solutions for specific business requirements ### 3.3 Account Security You are responsible for: * Keeping your credentials secure and confidential * Immediately notifying us of any unauthorized access * Ensuring compliance with these Terms by all users under your account ## 4. Acceptable Use Policy ### 4.1 Permitted Uses You may use our Services for legitimate business purposes, including: * Price monitoring and comparison * Product catalog analysis * Market trend research * Competitive landscape mapping * Lead generation from public directories - Social media sentiment analysis - News and content aggregation - Real estate listing data * Website performance monitoring * Content verification and compliance checking * SEO analysis and optimization * Brand protection and monitoring ### 4.2 Prohibited Activities The following activities are strictly prohibited and may result in immediate account termination: You may **NOT** use our Services to: #### Data and Content Restrictions * Access or scrape websites that explicitly prohibit automated access in their terms of service or robots.txt files * Extract copyrighted content without proper authorization * Collect personal data in violation of privacy laws (GDPR, CCPA, etc.) * Bypass authentication mechanisms or access password-protected content without authorization #### Technical Restrictions * Overload target websites with excessive requests * Circumvent or disable security measures, rate limiting, or anti-bot protections * Use the Services to attack, harm, or disrupt any network, service, or infrastructure * Reverse engineer, decompile, or attempt to extract our proprietary technology #### Legal and Ethical Restrictions * Violate any applicable laws, regulations, or third-party rights * Engage in fraudulent, deceptive, or malicious activities * Create or distribute malware, viruses, or harmful code * Infringe on intellectual property rights of any third party #### Commercial Restrictions * Resell or redistribute our Services without explicit written permission * Create competing services using our infrastructure * Share API credentials with unauthorized third parties ### 4.3 Website Compliance Always review the terms of service and robots.txt files of target websites before scraping. When in doubt, seek explicit permission from website owners. You are solely responsible for ensuring that your use of our Services complies with: * Target website terms of service and acceptable use policies * Applicable laws in your jurisdiction and the target website's jurisdiction * Industry regulations and ethical standards * Privacy and data protection requirements ## 5. Pricing and Payment ### 5.1 Subscription Plans Our Services are provided on a subscription basis with the following billing models: Select from Free, Professional, or Enterprise tiers based on your usage requirements. Most plans include a base allocation with additional usage charged per request or data volume. Choose between monthly flexibility or annual savings with our billing cycles. ### 5.2 Payment Terms * **Payment Processing**: All payments are processed through Stripe, our secure payment processor * **Billing Cycle**: Charges are billed in advance on a monthly or annual basis * **Automatic Renewal**: Subscriptions automatically renew unless cancelled before the next billing cycle * **Failed Payments**: Services may be suspended for non-payment after reasonable notice ### 5.3 Refunds and Cancellations You may cancel your subscription at any time through your account dashboard. Cancellation takes effect at the end of the current billing period. Refunds are generally not provided for unused services, except in cases of service non-availability exceeding our SLA commitments. Upgrades are prorated for the remainder of the billing cycle. Downgrades take effect at the next billing cycle. ## 6. Data Ownership and Privacy ### 6.1 Your Data You retain ownership of all data you provide to our Services and data extracted on your behalf. We act as a data processor under applicable privacy laws. ### 6.2 Our Data Rights We may collect and analyze aggregated, anonymized usage data to: * Improve service performance and reliability * Develop new features and services * Monitor for abuse and ensure compliance * Generate insights for product development All analysis is performed on aggregated, non-personally identifiable data that cannot be traced back to individual users or specific data sources. ### 6.3 Data Security We implement industry-standard security measures including: * Encryption in transit and at rest * Regular security audits and assessments * Access controls and authentication * Incident response procedures ## 7. Service Level Agreement ### 7.1 Uptime Commitment We strive to maintain 99.9% uptime for our core API services, excluding: * Scheduled maintenance windows (announced in advance) * Third-party service dependencies beyond our control * Force majeure events and circumstances beyond our reasonable control ### 7.2 Performance Standards Our Services are designed to provide: * Low-latency response times for standard requests * Reliable data extraction with built-in retry mechanisms * Scalable infrastructure to handle varying load demands ### 7.3 Support Response Times Different response times based on your subscription level: Community support via documentation and forums Email support with 24-48 hour response time during business hours Priority support with dedicated account management and faster response times ## 8. Intellectual Property ### 8.1 Our Intellectual Property Weblinq retains all rights to our Services, including: * The Weblinq API and underlying technology * Our proprietary algorithms and methodologies * Documentation, guides, and educational materials * Trademarks, logos, and brand elements ### 8.2 License Grant We grant you a limited, non-exclusive, non-transferable license to use our Services in accordance with these Terms. This license does not include rights to: * Reverse engineer or replicate our technology * Create derivative works based on our Services * Remove or modify any proprietary notices ### 8.3 Respect for Third-Party Rights You must ensure that your use of our Services respects the intellectual property rights of website owners and content creators. ## 9. Liability and Disclaimers ### 9.1 Service Disclaimers Our Services are provided "AS IS" and "AS AVAILABLE" without warranties of any kind, either express or implied. We specifically disclaim warranties regarding: * Uninterrupted or error-free operation * Accuracy, completeness, or timeliness of extracted data * Compatibility with all target websites or applications * Fitness for any particular purpose ### 9.2 Limitation of Liability Our liability for direct damages is limited to the amount paid by you for the Services in the 12 months preceding the event giving rise to the claim. We shall not be liable for any indirect, incidental, special, consequential, or punitive damages, including lost profits, data loss, or business interruption. You agree to indemnify and hold us harmless from any third-party claims arising from your use of our Services or violation of these Terms. ### 9.3 Indemnification You agree to indemnify, defend, and hold harmless Weblinq and its officers, directors, employees, and agents from and against any claims, damages, obligations, losses, liabilities, costs, or debt arising from: * Your use of the Services in violation of these Terms * Your violation of any third-party rights * Your violation of applicable laws or regulations * Any content or data you submit through our Services ## 10. Termination ### 10.1 Termination by You You may terminate your account at any time by: * Cancelling your subscription through the dashboard * Contacting our support team * Providing written notice of termination ### 10.2 Termination by Us We may terminate or suspend your account immediately if: * You violate these Terms or our Acceptable Use Policy * Your account poses a security risk to our Services * You fail to pay applicable fees after reasonable notice * We are required to do so by law or regulation ### 10.3 Effect of Termination Upon termination: * Your access to the Services will be immediately suspended * We may delete your account data after a reasonable retention period * Outstanding fees remain due and payable * Provisions regarding liability, indemnification, and intellectual property survive termination ## 11. General Legal Provisions ### 11.1 Governing Law These Terms are governed by and construed in accordance with the laws of \[Your Jurisdiction], without regard to conflict of law principles. ### 11.2 Dispute Resolution We encourage resolving disputes through direct communication with our support team. If direct resolution fails, disputes may be resolved through binding mediation. For disputes that cannot be resolved through mediation, binding arbitration may be required as specified in your subscription agreement. ### 11.3 Severability If any provision of these Terms is found to be unenforceable, the remaining provisions will continue in full force and effect. ### 11.4 Force Majeure Neither party shall be liable for any failure to perform due to causes beyond their reasonable control, including natural disasters, government actions, or network failures. ### 11.5 Entire Agreement These Terms, together with our Privacy Policy and any applicable subscription agreements, constitute the entire agreement between you and Weblinq regarding the Services. ## 12. Contact Information For questions about these Terms or our Services, please contact us: [support@weblinq.dev](mailto:support@weblinq.dev) *** By using Weblinq API Services, you acknowledge that you have read, understood, and agree to be bound by these Terms of Service. {' '} # Contact Support Source: https://docs.weblinq.dev/support/contact Get help from the WebLinq team and community We're here to help you succeed with the WebLinq API. Choose the best support option for your needs. ## Support Options Contact our support team directly: - Technical issues and bugs - Account and billing questions - Feature requests - Partnership inquiries ## Before Contacting Support To help us resolve your issue quickly, please include: * Your account email * API key (last 4 characters) * Subscription plan * Request ID(s) * API endpoint used * Complete error message * Steps to reproduce Provide a minimal code sample that demonstrates the issue: ```javascript const response = await fetch('https://api.weblinq.dev/v1/web/content', { method: 'POST', headers: { 'Authorization': 'Bearer xxxxx', // Replace with your key 'Content-Type': 'application/json' }, body: JSON.stringify({ url: 'https://example.com', selectors: { title: 'h1', content: '.main-content' } }) }); ``` * What you've tried so far * Expected vs actual behavior * Screenshots (if relevant) * Browser/environment details Enterprise customers receive 24/7 support with guaranteed response times based on issue severity. ## Feature Requests Have an idea for improving WebLinq? We'd love to hear it! When submitting a feature request, please include: - Clear description of the feature - Use case and benefits - Potential implementation ideas - Impact on existing functionality ## Community Guidelines When interacting with our support channels: * Treat others with respect - Use inclusive language - Be patient with responses - Appreciate volunteer help {' '} * Write clear descriptions - Include relevant details - Use proper formatting - Stay on topic * Share solutions when you find them - Help others if you can - Document workarounds - Provide feedback ## Status Updates Check our status page for: - Current system status - Incident reports - Scheduled maintenance - Historical uptime {' '} # Frequently Asked Questions Source: https://docs.weblinq.dev/support/faq Common questions about using the WebLinq API Find answers to common questions about using the WebLinq API. ## General Questions WebLinq is a powerful web scraping and content extraction API that helps you: * Extract clean, structured content from any webpage * Take high-quality screenshots * Generate PDFs * Extract structured data using AI * Monitor websites for changes Our API is designed to be reliable, scalable, and easy to use. WebLinq offers flexible pricing plans: - **Free Tier**: 1,000 credits/month, perfect for testing - **Developer**: $49/month for 10,000 credits - **Business**: $199/month for 50,000 credits - **Enterprise**: Custom pricing for high-volume needs One credit typically equals one API request. Some advanced features like AI extraction may use more credits. Yes! You can sign up for a free account and get 1,000 credits to test the API. No credit card required. The free tier includes: - Full API access - Basic rate limits - Community support - Core features and endpoints WebLinq is a REST API that works with any programming language. We provide official SDKs for: * [Node.js](/sdks/node) * [Python](/sdks/python) * [PHP](/sdks/php) The API can be used with any language that can make HTTP requests. ## Technical Questions We provide several options for JavaScript-heavy sites: 1. **Wait Until Options**: ```json { "wait_until": "networkidle0" // Wait for network to be idle } ``` 2. **JavaScript Execution**: ```json { "javascript": true, // Enable JavaScript execution "wait_time": 5000 // Wait 5 seconds after load } ``` 3. **Custom Triggers**: ```json { "wait_for": ".dynamic-content", // Wait for element "timeout": 10000 // Max wait time } ``` Rate limits vary by plan: - **Free**: 60 requests/minute - **Developer**: 300 requests/minute - **Business**: 1,000 requests/minute - **Enterprise**: Custom limits We provide rate limit info in response headers: `http X-RateLimit-Limit: 60 X-RateLimit-Remaining: 45 X-RateLimit-Reset: 1625097600 ` Implement exponential backoff for rate-limited requests. We support multiple authentication methods for target websites: 1. **Cookie-based**: ```json { "headers": { "Cookie": "session=abc123" } } ``` 2. **Header-based**: ```json { "headers": { "Authorization": "Bearer token123" } } ``` 3. **Form Login**: ```json { "authentication": { "type": "form", "loginUrl": "https://example.com/login", "username": "user", "password": "pass" } } ``` We employ several strategies to handle anti-bot measures: 1. Rotating IP addresses 2. Browser fingerprint randomization 3. Human-like behavior patterns 4. Automatic CAPTCHA solving (Enterprise plans) Some websites may explicitly prohibit scraping. Always check terms of service. ## Features & Capabilities We support multiple output formats: * **Text**: Clean, formatted text * **HTML**: Raw or cleaned HTML * **JSON**: Structured data * **Markdown**: Formatted markdown * **PDF**: High-quality PDFs * **Images**: PNG, JPEG, WebP Specify the format in your request: ```json { "format": "markdown", "clean": true // Remove ads, nav, etc. } ``` Yes! Use CSS selectors to target specific elements: ```json { "selectors": { "title": "h1.article-title", "author": ".author-bio h3", "content": "article.main-content", "date": "time.published-date" } } ``` We also support: * XPath selectors * Regular expressions * AI-powered content extraction We offer several pagination strategies: 1. **Auto-pagination**: ```json { "pagination": { "enabled": true, "maxPages": 5 } } ``` 2. **Custom Selectors**: ```json { "pagination": { "nextButton": ".pagination .next", "maxPages": 5 } } ``` 3. **URL Patterns**: ```json { "pagination": { "pattern": "page={i}", "start": 1, "end": 5 } } ``` We support geolocation-specific requests: ```json { "geolocation": { "country": "US", "city": "New York" } } ``` Enterprise customers can also: * Use dedicated IP ranges * Specify residential proxies * Set custom User-Agent strings ## Troubleshooting Common error codes and solutions: * **401**: Invalid API key * **403**: Permission denied * **429**: Rate limit exceeded * **500**: Server error * **504**: Timeout See our [Error Handling](/api-reference/errors) guide for details. Timeouts can occur for several reasons: 1. **Slow Website**: - Increase timeout: `"timeout": 30000` - Use `wait_until: "networkidle0"` 2. **Complex JavaScript**: - Enable JS: `"javascript": true` - Add wait time: `"wait_time": 5000` 3. **Large Pages**: - Use selective extraction - Implement pagination Try these troubleshooting steps: 1. **Check Selectors**: - Use browser dev tools to verify - Try multiple selector options 2. **JavaScript Content**: - Enable JS execution - Add appropriate wait conditions 3. **Dynamic Content**: - Use `wait_for` with specific elements - Implement retry logic If you're getting blocked: 1. **Implement Delays**: ```json { "delay": { "min": 2000, "max": 5000 } } ``` 2. **Rotate User Agents**: ```json { "headers": { "User-Agent": "custom-agent" } } ``` 3. **Use Enterprise Features**: * Residential proxies * IP rotation * Custom headers ## Business & Support Yes! Enterprise plans include: * Dedicated support team * Custom feature development * SLA guarantees * Technical consulting * Custom integrations [Contact us](mailto:enterprise@weblinq.dev) for details. We take data privacy seriously: - Data encrypted in transit and at rest - No persistent storage of scraped content - GDPR and CCPA compliant - Regular security audits - SOC 2 Type II certified See our [Privacy Policy](/legal/privacy) for details. Yes! Our API is production-ready with: - 99.9% uptime guarantee - Global CDN distribution - Automatic scaling - Real-time monitoring - Production support Many companies use WebLinq in production for: - Content aggregation - Price monitoring - Market research - Data analytics We offer multiple support channels: * [Documentation](/api-reference/overview) * [Discord Community](https://discord.gg/weblinq) * [Email Support](mailto:support@weblinq.dev) * [GitHub Issues](https://github.com/devhims/weblinq/issues) Enterprise customers get priority support via dedicated channels. ## Still have questions? Our team is ready to help with any questions {' '}