Relay Node Connections to MPC Nodes Are Being Denied

[vinay10949]

Description:
In our setup, RPC calls to MPC nodes should only be allowed from localhost or the relay node, which runs on localhost:8081. However, we are experiencing issues where connections from the relay node to MPC nodes are being denied, even though all services are running on localhost.

Code Implementation:
To enforce this restriction, we applied a HostFilterLayer to allow only specific hosts:

async fn start_rpc_server(port: String, is_relay_node: bool) -> Result<ServerHandle, Box<dyn Error>> {
    let rpc = RpcServerImpl::new();

    // Allow connections only from localhost and relay node
    let host_filter = HostFilterLayer::new(["localhost", "127.0.0.1", "localhost:8081", "127.0.0.1:8081"]).unwrap();
    let middleware = ServiceBuilder::new().layer(host_filter);

    if is_relay_node {
        let server = ServerBuilder::default().build(format!("0.0.0.0:{}", port)).await.map_err(Box::new)?;
        let addr = server.local_addr().unwrap();
        info!("Starting Relay node with RPC at {:?}", addr);
        return Ok(server.start(rpc.into_rpc()));
    } else {
        let server = ServerBuilder::default().set_http_middleware(middleware).build(format!("0.0.0.0:{}", port)).await.map_err(Box::new)?;
        let addr = server.local_addr().unwrap();
        info!("Starting local RPC server at {:?}", addr);
        return Ok(server.start(rpc.into_rpc()));
    }
}

Expected Behavior:

• RPC calls to MPC nodes should be accepted if they originate from localhost or the relay node running on localhost:8081.

Actual Behavior:

• Connections from the relay node (running at localhost:8081) to MPC nodes are being denied.

Additional Information:

• All services are bound to 0.0.0.0, which should accept connections on all network interfaces.
• It is unclear if HostFilterLayer is matching localhost:8081 as intended or if there's a configuration detail we might have missed.

Request:
Could this be an issue with HostFilterLayer? Is there a recommended approach for ensuring that connections from localhost:8081 are recognized and allowed as expected?

[niklasad1]

Hmm, what's the http status/error you got when the connection is denied? I guess it's is Provided Host header is not whitelisted?

Are you sure that "client that tries to connect" sets the port number in the host header properly? I guess it could some mismatch between default ports or something....

I would just try to do (to workaround that):

// accept any localhost connection
let host_filter = HostFilterLayer::new(["localhost:*", "127.0.0.1:*"]).unwrap();

I also tried to re-produce by the following (but it works intended):

#[tokio::test]
async fn ws_host_filtering_wildcard_works() {
	use jsonrpsee::server::*;

	init_logger();

	let middleware =
		tower::ServiceBuilder::new().layer(HostFilterLayer::new(["localhost:9999", "127.0.0.1:9999"]).unwrap());

	let server = ServerBuilder::default().set_http_middleware(middleware).build("127.0.0.1:9999").await.unwrap();
	let mut module = RpcModule::new(());
	let addr = server.local_addr().unwrap();
	module.register_method("say_hello", |_, _, _| "hello").unwrap();

	let _handle = server.start(module);

	let server_url = format!("ws://{}", addr);
	let client = WsClientBuilder::default().build(&server_url).await.unwrap();

	assert!(client.request::<String, ArrayParams>("say_hello", rpc_params![]).await.is_ok());
}

I don't know whether if I changed that in polkadot-sdk but perhaps we should just do localhost:* there as well to be on the safe-side.

[vinay10949]

I used to get TransportError Rejected 403 something. Let me try the change you mentioned

[niklasad1]

Cool, if that doesn't fix it please try to run the server for a short while with RUST_LOG="jsonrpsee=trace".

Then you should be able to see logs such as:

Request { method: GET, uri: /, version: HTTP/1.1, headers: {"host": "127.0.0.1:9999", "upgrade": "websocket", "connection": "Upgrade", "sec-websocket-key": "ND7zx3QHC0qmvkqnN9PZ3A==", "sec-websocket-version": "13"}, body: Body(UnsyncBoxBody) }
2024-11-05T17:10:55.756242Z DEBUG jsonrpsee-server: Denied request: Request { method: GET, uri: /, version: HTTP/1.1, headers: {"host": "127.0.0.1:9999", "upgrade": "websocket", "connection": "Upgrade", "sec-websocket-key": "ND7zx3QHC0qmvkqnN9PZ3A==", "sec-websocket-version": "13"}, body: Body(UnsyncBoxBody) }

If something fails please paste such logs which would help me to understand what's going on...

[niklasad1]

Ok, I think I found a similar issue in polkadot-sdk where we now open two sockets (one ipv4 and one ipv6) then for instance tools like cURL try to connect using ipv6 before ipv4.

Because we had different host filters on those interfaces the ipv6 connection will use localhost as host header and be rejected.

➜ wasm-tests (update-artifacts-1731284930) ✗ curl localhost:9944 -v
* Host localhost:9944 was resolved.
* IPv6: ::1
* IPv4: 127.0.0.1
*   Trying [::1]:9944...
* Connected to localhost (::1) port 9944
> GET / HTTP/1.1
> Host: localhost:9944
> User-Agent: curl/8.5.0
> Accept: */*
>
< HTTP/1.1 403 Forbidden
< content-type: text/plain
< content-length: 41
< date: Tue, 12 Nov 2024 12:42:15 GMT
<
Provided Host header is not whitelisted.
* Connection #0 to host localhost left intact

[niklasad1]

@vinay10949 any update on this?