MCP-RSS-Crawler

By mshk GitHub

RSS Crawler MCP Server

Overview

what is MCP-RSS-Crawler?

MCP-RSS-Crawler is an RSS feed management tool that automatically fetches and organizes RSS feeds, allowing users to retrieve and manage articles efficiently.

how to use MCP-RSS-Crawler?

To use MCP-RSS-Crawler, clone the repository, set up the environment, install dependencies, and start the server. Configure your LLM client to use the MCP endpoint for accessing the latest RSS feeds.

key features of MCP-RSS-Crawler?

Automatic fetching and caching of RSS feeds
Customizable refresh intervals for feed updates
Keyword management for filtering articles
Comprehensive API endpoints for feed management
Command-line interface for server administration

use cases of MCP-RSS-Crawler?

Managing multiple RSS feeds for news aggregation.
Filtering articles based on specific keywords for targeted content.
Integrating with LLMs to provide real-time updates from RSS feeds.

FAQ from MCP-RSS-Crawler?

Can I add multiple RSS feeds?

Yes! You can add and manage multiple RSS feed URLs.

How do I filter articles?

You can define keywords to filter and prioritize content based on your preferences.

Is there a command-line interface?

Yes! MCP-RSS-Crawler includes a command-line interface for server administration.

Content

MCP-RSS-Crawler

An MCP (Message Chain Protocol) server that fetches RSS feeds and shares them with LLMs.

Features

Fetching and caching of RSS feeds (SQLite database)
MCP protocol implementation for seamless LLM integration
Support for filtering feeds by category, source, or keywords
Comprehensive API endpoints for feed management
- Add, update, and delete feeds
Support for fetching articles from Firecrawl

Requirements

Bun
Firecrawl API key
Claude Desktop or other MCP client

Setup as MCP Server

Clone this repository
Create a claude_desktop_config.json file based on claude_desktop_config.json.example with your configuration

{
  "mcpServers": {
    "rss-crawler": {
      "command": "/path/to/bun",
      "args": ["run", "/path/to/mcp-rss-crawler/apps/mcp-server/src/mcp-cli.ts"],
      "cwd": "/path/to/mcp-rss-crawler",
      "env": {
        "PORT": "5556",
        "DB_DIR": "/path/to/mcp-rss-crawler",
        "FIRECRAWL_API_KEY": "fc-<YOUR_FIRECRAWL_API_KEY>"
      }
    }
  }
}

Install dependencies:
```
bun install
```
Start Claude Desktop:

MCP Protocol

The server implements the Message Chain Protocol (MCP) which allows LLMs to access your latest RSS feeds. The MCP endpoint accepts POST requests with a JSON body containing a messages array and returns a response with the latest feed items.

Example request:

{
  "messages": [
    {
      "role": "user",
      "content": "What are the latest news from my RSS feeds?"
    }
  ]
}

Example response:

{
  "messages": [
    {
      "role": "assistant",
      "content": "Here are the latest articles from your RSS feeds:",
      "name": "rss-mcp"
    },
    {
      "role": "tool",
      "content": "[{\"title\":\"Article Title\",\"summary\":\"Article summary...\",\"published\":\"2025-03-16T04:30:00.000Z\",\"origin\":\"Feed Name\",\"link\":\"https://example.com/article\"}]",
      "name": "rss-feeds"
    }
  ]
}

Configuration Options

The server can be configured through environment variables or a .env file:

PORT - Server port (default: 5556)
FIRECRAWL_API_KEY - Firecrawl API key
DB_DIR - Database directory (default: ~/.mcp-rss-crawler)

Troubleshooting

For connection issues, check your network settings and firewall configuration
Logs are available in the console and can be used to diagnose problems
For more detailed logging, set the DEBUG=mcp-rss:* environment variable

No tools information available.

No content found.