feat: Add structured audit logging for MCP feature server by SIDDHESH1564 · Pull Request #6456 · feast-dev/feast

SIDDHESH1564 · 2026-05-30T17:59:21Z

What this PR does / why we need it:

This PR adds structured audit logging in JSONL format for the Python MCP feature server.

Previously, the MCP integration only logged startup events and errors. Operators did not have a unified audit trail for MCP tool calls, internal REST requests, authentication results, authorization decisions, or request correlation.

This change introduces an audit_logging configuration section under feature_server and adds structured audit events for:

MCP tool invocations
MCP JSON-RPC requests
REST endpoint requests
Authentication outcomes
Authorization decisions
Request correlation through X-Request-Id
Client IP address and transport metadata

Each audit event follows a stable schema and includes:

event_type
UTC timestamp
request_id
principal (username, roles, and auth_type)
source (ip and transport)
action (MCP tool name and/or HTTP path)
resource (type, name, and permitted actions)
outcome
duration_ms

Sensitive data is intentionally excluded from audit logs. Tokens, entity rows, feature values, and request payloads are not logged.

Audit logging layers

Layer	Scope	Event types
`McpAuditMiddleware`	`/mcp` JSON-RPC requests	`mcp.tools.call`, `mcp.request`
`AuditLoggingMiddleware`	REST endpoints such as `/get-online-features` and `/push`	`http.request`
`AuditLogger` helpers	Authentication and authorization hooks	`authn.success`, `authn.failure`, `authz.decision`

New files

sdk/python/feast/audit/audit_logger.py
- Adds the AuditEvent model.
- Adds sink abstractions: StdoutAuditSink, FileAuditSink, and LoggerAuditSink.
- Adds AuditLogger helper methods for MCP, HTTP, authentication, and authorization events.
sdk/python/feast/audit/audit_middleware.py
- Adds AuditLoggingMiddleware for REST endpoint auditing.
- Adds McpAuditMiddleware for MCP JSON-RPC request auditing.

Modified files

sdk/python/feast/infra/feature_servers/base_config.py
- Adds AuditLoggingConfig to BaseFeatureServerConfig.
sdk/python/feast/feature_server.py
- Initializes the audit logger lifecycle.
- Wires the audit middleware into get_app().

Example configuration

feature_server:
  type: mcp
  mcp_enabled: true
  audit_logging:
    enabled: true
    sink: stdout          # stdout | file | logger
    # file_path: /var/log/feast_audit.jsonl
    log_successful_reads: true

Example JSONL event

{
  "event_type": "mcp.tools.call",
  "timestamp": "2026-05-28T12:00:00.000Z",
  "request_id": "...",
  "principal": {
    "username": "jane@co.com",
    "roles": ["reader"],
    "auth_type": "oidc"
  },
  "source": {
    "ip": "10.0.0.1",
    "transport": "mcp-http"
  },
  "action": {
    "mcp_tool": "get_online_features",
    "path": "/mcp"
  },
  "outcome": "success",
  "duration_ms": 42.0
}

Which issue(s) this PR fixes:

Fixes #6452

Checks

I've made sure the tests are passing.
My commits are signed off (git commit -s)
My PR title follows conventional commits format

Testing Strategy

Unit tests
Integration tests
Manual tests
Testing is not required for this change

Added 37 unit tests covering:

AuditEvent serialization in JSONL format, including exclusion of None values
AuditLogger helpers:
- log_mcp_call
- log_http_request
- log_authn
- log_authz
Suppression of successful read events when log_successful_reads: false
All supported sinks:
- StdoutAuditSink
- FileAuditSink
- LoggerAuditSink
create_audit_logger_from_config factory behavior and fallback handling
AuditLoggingConfig validation and inheritance through McpFeatureServerConfig
AuditLoggingMiddleware behavior:
- HTTP request logging
- Health and MCP route skipping
- Error outcomes
- X-Request-Id propagation
- Resource mapping
McpAuditMiddleware behavior:
- MCP tool-call extraction from JSON-RPC request bodies
- Generic MCP request logging
- Non-JSON request-body handling

Misc

audit_logging is defined on BaseFeatureServerConfig, making it available to both local and MCP feature server types.
Principal metadata is extracted from the existing SecurityManager context and supports no_auth, oidc, and kubernetes authentication modes.
Registry MCP parity and an OpenTelemetry sink are intentionally left as follow-up improvements, as noted in the linked issue.

Signed-off-by: Siddhesh Khairnar <khairnarsiddhesh4057@gmail.com>

ntkathole · 2026-05-31T15:34:15Z

@SIDDHESH1564

Registry MCP parity and an OpenTelemetry sink are intentionally left as follow-up improvements, as noted in the linked issue.

OTel is a follow-up is fine, but the core schema and sink abstraction feel like reinventing what opentelemetry-sdk already provides. Also, AuditEvent should carry a trace_id/span_id if OpenTelemetry is active.

Also, the fundamental problem with this is that it intercepts at the HTTP transport layer rather than at the MCP protocol layer. It should implement bidirectional correlation - linking a request to its response.

…orrelation Signed-off-by: Siddhesh Khairnar <khairnarsiddhesh4057@gmail.com>

SIDDHESH1564 · 2026-05-31T17:19:43Z

Thanks for the review, @ntkathole. Both points have been addressed.

Added trace_id and span_id to AuditEvent, populated from the active OTel span when available.
Added MCP request-response correlation using the JSON-RPC id.
Added response-body inspection for JSON-RPC errors and a separate mcp_error outcome.

ntkathole · 2026-06-01T12:16:15Z

+        jsonrpc_id: Optional[str] = None
+        if request.method == "POST":
+            try:
+                body = await request.body()


This audit hook is at the wrong layer. Instead of parsing raw HTTP bodies, wrap mcp.server.request_handlers[CallToolRequest] inside add_mcp_support_to_app() after FastApiMCP.setup_server(). That gives you typed tool_name, arguments, CallToolResult.isError

Thanks for the detailed review @ntkathole. I’ll study the suggested MCP protocol-layer approach and make the required changes. Will also address the thread-safety comment for the logger. Thanks!

@ntkathole, I’ll remove the raw HTTP body parsing from McpAuditMiddleware and wrap the MCP server’s CallToolRequest handler after FastApiMCP.setup_server(). This will provide the typed tool name, arguments, and CallToolResult.isError directly, without json.loads() or response-body replay.

ntkathole · 2026-06-01T12:18:42Z

+    try:
+        from feast.permissions.security_manager import get_security_manager
+
+        sm = get_security_manager()


MPC server don't use get_security_manager - again this is HTTP layer, will always return an empty principal for MCP

@ntkathole, Since get_security_manager() does not apply to MCP requests, I’ll derive the authentication metadata from mcp.server.request_context.request.headers.

ntkathole · 2026-06-01T12:20:17Z

+class LoggerAuditSink(AuditSink):
+    """Emit audit events through Python's ``logging`` module at INFO level."""
+
+    def __init__(self, logger_name: str = "feast.audit") -> None:
+        self._logger = logging.getLogger(logger_name)
+
+    def emit(self, event: AuditEvent) -> None:
+        self._logger.info(event.to_jsonl())


Add a threading.Lock around emit(), when multiple tasks or workers emit the logs, it should handle

@ntkathole, I’ll add a threading.Lock around sink.emit() in the audit logging path.

SIDDHESH1564 · 2026-06-01T15:47:45Z

Will update the tests accordingly and push the revised commit once the changes are complete. Thanks!

…lock Signed-off-by: Siddhesh Khairnar <khairnarsiddhesh4057@gmail.com>

patelchaitany · 2026-06-04T11:04:36Z

@SIDDHESH1564

Suggestion:

It might help to generate request_id once in the tools/call wrapper and propagate the same value to the internal REST call (e.g. via X-Request-Id and FastApiMCP(headers=[..., "x-request-id"])). Right now the MCP path always mints a new ID while REST reads the header, and most MCP clients don’t send X-Request-Id, so mcp.tools.call and http.request may not correlate in SIEM. Worth keeping jsonrpc_id separate from audit request_id. A test that both events share one request_id per tool call would be great.

SIDDHESH1564 · 2026-06-06T18:55:21Z

@patelchaitany The mcp.tools.call and http.request events currently get independent request_id values, so they won't correlate in SIEM. Will add "x-request-id" to FastApiMCP's forwarded headers and propagate the audit request_id from the wrapper into the internal REST call. jsonrpc_id is already kept separate from request_id.

SIDDHESH1564 · 2026-06-06T19:02:41Z

Now the flow is:

audited_call_tool generates request_id = "aaa-111-..." and stores it in mcp_audit_request_id ContextVar.
Calls original(ctx, params) → fastapi_mcp makes internal httpx request to /get-online-features.
AuditLoggingMiddleware runs, checks ContextVar first:
propagated_id = mcp_audit_request_id.get() → "aaa-111-..."
request_id = propagated_id or ...
Both events now have request_id = "aaa-111-..."
ContextVar is reset in finally so it doesn't leak to subsequent requests.

Signed-off-by: Siddhesh Khairnar <khairnarsiddhesh4057@gmail.com>

patelchaitany · 2026-06-08T10:01:30Z

@SIDDHESH1564 - have you tested this locally by running the Feast feature server and checking audit logs end-to-end?

My setup: fastapi-mcp==0.4.0, mcp==1.27.0, MCP enabled with mcp_transport: http and audit_logging enabled (file sink).

What I see: REST audit (http.request) works, but MCP tool-call audit does not. On startup I get: Cannot wrap MCP call_tool handler: _request_handlers not found The wrapper never attaches — the handler stays handler, not audited_call_tool — so no mcp.tools.call events are emitted. Unit tests pass because they mock _request_handlers["tools/call"], which doesn’t exist in 0.4.0.

Root cause: The PR wraps mcp.server._request_handlers["tools/call"], but in fastapi-mcp 0.4.0 + mcp 1.x the hook is mcp.server.request_handlers[CallToolRequest] with signature async def handler(req: CallToolRequest).

Suggested fix: Update _wrap_call_tool_handler() in mcp_server.py to wrap request_handlers[CallToolRequest], read principal from server.request_context, and forward x-request-id / x-feast-auth-type in FastApiMCP(headers=...). Add a unit test using a real FastApiMCP instance (not a mocked _request_handlers).

Signed-off-by: Siddhesh Khairnar <khairnarsiddhesh4057@gmail.com>

SIDDHESH1564 · 2026-06-08T11:18:17Z

Thanks for the detailed report, @patelchaitany. you're right about the root cause. I've confirmed and fixed the issue.

The wrapper was targeting mcp.server._request_handlers["tools/call"], which was the internal API of an older mcp SDK.

feat: Add structured audit logging for MCP feature server

262f4e9

Signed-off-by: Siddhesh Khairnar <khairnarsiddhesh4057@gmail.com>

SIDDHESH1564 requested a review from a team as a code owner May 30, 2026 17:59

feat: Enhance audit logging with OpenTelemetry support and JSON-RPC c…

6e38935

…orrelation Signed-off-by: Siddhesh Khairnar <khairnarsiddhesh4057@gmail.com>

Merge branch 'master' into feat/mcp-audit-logging

0da402d

ntkathole reviewed Jun 1, 2026

View reviewed changes

refactor: Audit MCP at protocol layer, fix empty principal, add emit …

61c857e

…lock Signed-off-by: Siddhesh Khairnar <khairnarsiddhesh4057@gmail.com>

SIDDHESH1564 requested a review from ntkathole June 3, 2026 06:32

Merge branch 'feast-dev:master' into feat/mcp-audit-logging

b25c1d6

fix(mcp): Propagate request_id for audit logging

b95cd7b

Signed-off-by: Siddhesh Khairnar <khairnarsiddhesh4057@gmail.com>

refactor(mcp): Update MCP server handler for audit logging

5d32d9f

Signed-off-by: Siddhesh Khairnar <khairnarsiddhesh4057@gmail.com>

Conversation

SIDDHESH1564 commented May 30, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What this PR does / why we need it:

Audit logging layers

New files

Modified files

Example configuration

Example JSONL event

Which issue(s) this PR fixes:

Checks

Testing Strategy

Misc

Uh oh!

ntkathole commented May 31, 2026

Uh oh!

SIDDHESH1564 commented May 31, 2026

Uh oh!

ntkathole Jun 1, 2026

Choose a reason for hiding this comment

Uh oh!

SIDDHESH1564 Jun 1, 2026

Choose a reason for hiding this comment

Uh oh!

SIDDHESH1564 Jun 1, 2026

Choose a reason for hiding this comment

Uh oh!

ntkathole Jun 1, 2026

Choose a reason for hiding this comment

Uh oh!

SIDDHESH1564 Jun 1, 2026

Choose a reason for hiding this comment

Uh oh!

ntkathole Jun 1, 2026

Choose a reason for hiding this comment

Uh oh!

SIDDHESH1564 Jun 1, 2026

Choose a reason for hiding this comment

Uh oh!

SIDDHESH1564 commented Jun 1, 2026

Uh oh!

patelchaitany commented Jun 4, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

SIDDHESH1564 commented Jun 6, 2026

Uh oh!

SIDDHESH1564 commented Jun 6, 2026

Uh oh!

patelchaitany commented Jun 8, 2026

Uh oh!

SIDDHESH1564 commented Jun 8, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

SIDDHESH1564 commented May 30, 2026 •

edited

Loading

patelchaitany commented Jun 4, 2026 •

edited

Loading