How to Evaluate an MCP Server Before You Connect It (Security Checklist)

Connecting one is a real grant of access

The MCP ecosystem went from novelty to normal in about a year, and the connect flow is now three clicks. That is the problem: three clicks hands an AI assistant live access to a live account, and the assistant then decides for itself which tools to call.

Most servers are fine. The cost of the ones that are not is high enough to spend five minutes checking. Here is what to check.

1. What scope does it ask for, and is read-only offered?

The single best signal of a server built by people who thought about this. If the only option is full access, the provider has not considered that you might want to explore before you trust.

A read-only connection should be genuinely read-only — not "writes are discouraged" but the write tools are not listed to it at all, so the model cannot call them even if it decides to.

What good looks like: you pick read-only or read and write at approval time, and the tool list you get differs accordingly.

2. Which account or workspace is it bound to?

An assistant that can reach all your workspaces is an assistant that can mix them up. Binding should be explicit and visible, chosen when you approve rather than inferred.

This matters more than it sounds. Many products auto-create a personal workspace for every user; authorise against that by accident and every call fails with a permission error and no explanation — or worse, succeeds against the wrong data.

What good looks like: the approval screen asks which workspace, and afterwards something in the product tells you which one each connection acts on.

3. Can it spend money, and does it say so?

Assistants are enthusiastic. Ask about five videos and one may analyse eight. If each operation costs real money, that decision is being made by a model rather than by you.

What good looks like: each tool states what it costs in its own description, a hard quota caps the total, and the expensive operations are separable from the free ones via scope.

4. Can you revoke it yourself, immediately?

Ask this before connecting, not after. There are two sides — the connector in your AI client, and the grant on the provider's side — and removing only the first leaves tokens that still work.

A provider with no self-service revocation is telling you that cutting access means emailing support and waiting.

What good looks like: a connected-apps list showing every connection, what it can do, when it was last used, and a revoke button that works on both sides.

5. OAuth or a pasted key?

Prefer OAuth for a hosted assistant. Nothing is copied into a config file, tokens are short-lived and rotate automatically, and revocation is centralised.

A pasted API key in a client config is a long-lived credential in plain text on disk, frequently with no scope limits, and often shared across machines. It is not wrong — local MCP clients often need it — but it is a worse posture, and it should be your choice rather than the only path.

6. What do the tool descriptions actually say?

This is the one people miss, and it is the most interesting.

Tool descriptions are text the model reads and trusts. They arrive from the server, not from you, and they sit in a privileged position in the context. A malicious or careless server can write descriptions that steer the model — instructing it to prefer certain tools, to ignore other instructions, or to include context it should not.

Most clients let you inspect the advertised tools. Read them once. You are looking for plain descriptions of what each tool does, and the absence of anything that reads like an instruction to the model rather than a description of a capability.

7. Does it tell the truth about latency and failure?

A well-written server says when something is slow and returns a job handle rather than hanging. A badly written one lets the assistant sit in a timeout loop and burn your context.

What good looks like: async operations return a poll URL and a realistic retry interval; failures return a machine-readable code the assistant can reason about, not a prose string it has to guess at.

8. Is coverage stated honestly?

The failure mode specific to data products: an empty result that means "we do not have this yet" looks identical to one meaning "there is nothing". An assistant cannot tell the difference, so it will report the second.

What good looks like: the response distinguishes them, and the documentation states coverage rather than implying completeness.

The checklist, short version

• Is read-only offered, and genuinely enforced?
• Is it bound to one explicit account, chosen by you?
• Can it spend money, and does each tool say what it costs?
• Can you revoke it yourself, on both sides, immediately?
• OAuth rather than a long-lived pasted key?
• Do the tool descriptions describe capabilities, not instruct the model?
• Are async operations and failures reported honestly?
• Is data coverage stated rather than implied?

How FYPNow answers these

Since we publish an MCP server, it is only fair to answer our own checklist.

Read-only is offered and enforced by not listing the five write tools at all. Connections are bound to one workspace, chosen on our own consent screen, which also shows the client's name and redirect URI. Each write tool states the quota it draws down. Connections appear in Settings → API with a revoke button that cuts both our grant and the OAuth token. Claude connects over OAuth with no key; bearer keys exist for clients that cannot do OAuth. Async operations return a poll URL and a realistic interval. And where coverage is partial — about 63% of analysed videos carry the embedding semantic search needs — the response says so rather than returning an empty list.

Not every answer is "yes" yet. That is the point of publishing the checklist.

Where to go next

• MCP vs API — when a plain API is the better choice
• The MCP server — tools, scopes, costs
• Connect TikTok data to Claude — the OAuth flow end to end