Databricks MCP Server
A FastAPI-based MCP (Model Context Protocol) server that provides tools for local file management and Databricks operations.
Features
Local File Management
- Create folders and directories
- Create Python files with content
- Edit existing files
Databricks Operations
- Submit code to Databricks clusters
- Create and run Databricks jobs
- Create Delta Live Tables (DLT) pipelines
- Get job errors and status information
Setup
1. Install Dependencies
pip install -r requirements.txt
2. Environment Configuration
Create a .env file in the project root:
DATABRICKS_HOST=https://e2-demo-west.cloud.databricks.com/
DATABRICKS_TOKEN=YOUR_DAPI_TOKEN
3. Run the Server
uvicorn main:app --reload --host 0.0.0.0 --port 8000
API Endpoints
File Management
Create Folder
POST /file/create_folder
{
"path": "/path/to/folder"
}
Create Python File
POST /file/create_py_file
{
"path": "/path/to/file.py",
"content": "print('Hello, World!')"
}
Edit File
POST /file/edit_file
{
"path": "/path/to/file.py",
"content": "print('Updated content')"
}
Databricks Operations
Submit Code
POST /databricks/submit_code
{
"code": "print('Hello from Databricks!')",
"cluster_id": "your-cluster-id"
}
Create Job
POST /databricks/create_job
{
"job_config": {
"name": "My Job",
"new_cluster": {
"spark_version": "11.3.x-scala2.12",
"node_type_id": "i3.xlarge",
"num_workers": 1
},
"notebook_task": {
"notebook_path": "/Users/your.email@databricks.com/your_notebook"
}
}
}
Run Job
POST /databricks/run_job
{
"job_id": "your-job-id"
}
Create DLT Pipeline
POST /databricks/create_dlt_pipeline
{
"pipeline_config": {
"name": "My DLT Pipeline",
"storage": "dbfs:/pipelines/storage",
"clusters": [{"label": "default", "num_workers": 1}],
"libraries": [{"notebook": {"path": "/Users/your.email@databricks.com/your_dlt_notebook"}}]
}
}
Get Job Error
POST /databricks/get_job_error
{
"run_id": "your-run-id"
}
Check Job Status
POST /databricks/check_job_status
{
"job_id": "your-job-id",
"run_id": "your-run-id"
}
Claude Desktop Integration
1. Copy Configuration Files
Copy the MCP configuration files to your Claude Desktop configuration directory:
macOS:
cp mcp_server_config.json ~/Library/Application\ Support/Claude/claude_desktop_config.json
cp mcp_tools.json ~/Library/Application\ Support/Claude/mcp_tools.json
Windows:
copy mcp_server_config.json %APPDATA%\Claude\claude_desktop_config.json
copy mcp_tools.json %APPDATA%\Claude\mcp_tools.json
Linux:
cp mcp_server_config.json ~/.config/Claude/claude_desktop_config.json
cp mcp_tools.json ~/.config/Claude/mcp_tools.json
2. Start Your MCP Server
Make sure your Databricks MCP server is running:
uvicorn main:app --host 0.0.0.0 --port 8000
3. Restart Claude Desktop
Restart Claude Desktop to load the new MCP configuration.
Claude Desktop will now have access to all the Databricks and file management tools. You can ask Claude to:
- "Create a folder called 'my_project'"
- "Create a Python file with some Databricks code"
- "Submit this code to my Databricks cluster"
- "Create a DLT pipeline for data processing"
- "Check the status of my job"
Testing
Run the test suite:
Error Handling
The server provides detailed error messages and logging. All operations return a consistent response format:
{
"status": "success|error",
"message": "Description of what happened",
"detail": "Error details (if status is error)"
}
Security Notes
- Store your Databricks token securely
- Use environment variables for sensitive configuration
- Consider using Databricks workspace-specific tokens with limited permissions
- The server runs on
0.0.0.0:8000 by default - adjust for your security requirements
Troubleshooting
Common Issues
- Environment Variables Not Loaded: Make sure you have
python-dotenv installed and a .env file in the project root. - Databricks Connection Issues: Verify your host URL and token are correct. Test with a simple API call first.
- Permission Errors: Ensure the server has write permissions for file operations.
- Port Already in Use: Change the port in the uvicorn command or kill the existing process.
Logs
The server provides detailed logging. Check the console output for debugging information.
1. Create the Subfolder and Minimal Server
Directory structure:
mcp_test_server/
├── main.py
└── requirements.txt
main.py:
from fastapi import FastAPI
app = FastAPI()
@app.get("/")
def root():
return {"status": "ok", "message": "MCP test server is running"}
requirements.txt:
2. Install Dependencies
cd mcp_test_server
python3 -m venv venv
source venv/bin/activate
pip install -r requirements.txt
3. Test the Server Manually
venv/bin/python -m uvicorn main:app --host 0.0.0.0 --port 9000
- Visit http://localhost:9000/ in your browser. You should see:
{"status": "ok", "message": "MCP test server is running"}
4. Update Claude Config
Edit your claude_desktop_config.json to point to this test server:
{
"mcpServers": {
"test-mcp": {
"command": "/full/path/to/mcp_test_server/venv/bin/python",
"args": ["-m", "uvicorn", "main:app", "--host", "0.0.0.0", "--port", "9000"]
}
}
}
- Replace
/full/path/to/ with the actual path on your system.
5. Restart Claude Desktop
- Fully quit and restart Claude Desktop.
- See if you get any connection or server errors.
6. What to Look For
- If Claude Desktop can connect, you should see requests in your test server’s terminal.
- If not, and you still see no requests, the issue is with Claude’s config, file location, or local network.
7. If You Want, I Can Generate the Files
Let me know if you want me to generate the exact code for main.py and requirements.txt for you!
This minimal test will help you isolate whether the problem is with your main server, your config, or Claude Desktop’s ability to reach any local MCP server.
Let me know when you’ve tried this and what you see!
Why This Happens
- Claude Desktop starts the server in the directory:
/Users/stephen.hsu/Desktop/cursor_test/mcp_dbx_pure - But your test server code is in:
/Users/stephen.hsu/Desktop/cursor_test/mcp_dbx_pure/mcp_test_server/main.py
So, when it runs:
venv/bin/python -m uvicorn main:app --host 0.0.0.0 --port 9000
from the parent directory, there is no main.py in the current directory, so import fails.
How to Fix
1. Update the "args" in your config to include the subfolder:
Change from:
"args": ["-m", "uvicorn", "main:app", "--host", "0.0.0.0", "--port", "9000"]
to:
{
"mcpServers": {
"test-mcp": {
"command": "/Users/stephen.hsu/Desktop/cursor_test/mcp_dbx_pure/mcp_test_server/venv/bin/python",
"args": ["-m", "uvicorn", "mcp_test_server.main:app", "--host", "0.0.0.0", "--port", "9000"]
}
}
}
This tells Uvicorn to look for the main module inside the mcp_test_server package.
2. Make mcp_test_server a Python package
Add an empty __init__.py file to mcp_test_server:
touch mcp_test_server/__init__.py
3. Update Your Config
Your claude_desktop_config.json should look like:
{
"mcpServers": {
"test-mcp": {
"command": "/Users/stephen.hsu/Desktop/cursor_test/mcp_dbx_pure/mcp_test_server/venv/bin/python",
"args": ["-m", "uvicorn", "mcp_test_server.main:app", "--host", "0.0.0.0", "--port", "9000"]
}
}
}
4. Restart Claude Desktop
- Save the config.
- Restart Claude Desktop.
Why This Works
uvicorn mcp_test_server.main:app tells Python to look for main.py inside the mcp_test_server package, no matter what the current working directory is.- The
__init__.py file makes mcp_test_server a valid Python package.
Summary
- Update the
"args" to use mcp_test_server.main:app - Add
__init__.py to mcp_test_server - Restart Claude Desktop
This should allow Claude Desktop to start and connect to your minimal MCP test server!
Let me know if you see requests in your test server terminal after this change, or if you get a new error.