Azure SignalR Service - Things about upstream

Upstream is a new feature that allows Azure SignalR Service to send messages and connection events to a set of endpoints in serverless mode. This post is explain more details about this feature.

What is upstream for in Azure Azure SignalR Service

The Azure SignalR Service introduced a new feature Upstream on June 2020. Upstream allows Azure SignalR Service to send messages and connection events to a set of endpoints in serverless mode. You can use upstream to invoke a hub method from clients in serverless mode and let endpoints get notified when client connections are connected or disconnected.

Why introduced the Upstream

The Azure SignalR Service has 2 service modes, default mode and serverless mode.

The goal of Serverless Websocket is to allow the Azure SignalR Service to handle pure WebSocket requests without SignalR:

  • Talk to an upstream application using the HTTP protocol.
  • Handle connection connect/disconnect events.
  • Handle the WebSocket handshake, with the ability to configure the SubProtocol and reject connections.
  • Handle WebSocket messages, supporting both text and binary messages within one connection.

The Upstream is a feature that introduces an upstream server that processes WebSocket connections and messages. The upstream server can be any kind of server that can handle HTTP requests, like the Azure Function service.

Default Mode

Default mode is the default value for service mode when you create a new SignalR resource. In this mode, your application works as a typical ASP.NET Core (or ASP.NET) SignalR application, where you have a web server that hosts a hub (called hub server hereinafter) and clients can have duplex real-time communication with the hub server. The only difference is instead of connecting client and server directly, client and server both connect to SignalR service and use the service as a proxy. Below is a diagram that illustrates the typical application structure in default mode:

Default Mode

Pros:

  • With Default mode, the app server hosts a hub. The clients can have duplex real-time communication. It allows the client to receive and sends messages.
  • The server is stateful, the client is bond with a server connection, the client’s data stores in the app server.

Cons:

  • The app server only supports Asp.Net Core and/or Asp.Net SignalR application. And not supports other languages like Java, NodeJS, Python, etc.
  • Client connections will be bond with a server connection. In case a server connection dropped for a reason, multiple client connections are affected. Here is an example, there are 5 server connections and 1000 client connections connected to the Azure SignalR Service. The Azure SignalR Server routes the 1000 clients to 5 server connections. For each server connection, it maps to 200 client connections. If a server connection dropped, then 200 client connections will be dropped by the SignalR Service at once.

Serverless Mode - Typical

With the Serverless mode, there is not any hub server. Comparing to default mode, in this mode client doesn’t require a hub server to get connected. All connections are connected to the SignalR service in a “serverless” mode and SignalR Service is responsible for maintaining client connections. If you try to use service SDK to establish a server connection, you will get an error. Therefore there is also no connection routing and server-client stickiness. The clients have persistent connections to Azure SignalR Service. Since there is no application server to handle traffic, clients are in LISTEN mode, which means they can only receive messages but can’t send messages. SignalR Service will disconnect any client who sends messages because it is an invalid operation.

Serverless mode

As there is no hub, you have another choice to send message to the clients through SignalR service. You can use REST APIs for one-time send. For the .Net application, you can leverage SignalR service management SDK, for other languages should invoke the REST APIs following this spec.

Pros:

  • The client connects to the SignalR Serivce directly.

Cons:

  • The client can only receives messages, and can’t send message to the SignalR Service.

Serverless Mode - Upstream

Upstream is a new feature of Azure SignalR Service. It allows Azure SignalR Service to send messages and connection events to a set of endpoints in serverless mode.
As it’s the serverless mode, there is no application hub, clients connect to the SignalR Service directly.
Furthermore, upstream allows the client to send messages to the SignalR Service, and the service will process to invoke a proper method in the upstream server. And notify the upstream server when client connections are connected or disconnected.

upstream arch

Pros:

  • As the serverless mode, the client connection connects to the SignalR Service directly.
  • With the Upstream, the client can also send messages to the SignalR Service.

Cons:

  • Compare with the default mode, the upstream server is used to receive the message, but not to store the client’s status. The upstream server is stateless.

How it works

How to Configure the Upstream

Configure the upstream setting is very simple. Firstly, you need an app server which can receive HTTP request. Then, Create upstream settings via the Azure portal to in the Azure SignalR Service.

Client Connection

The Client connection is similar to the default mode, the client is authorized with the app server, then negotiate a connection with Azure SignalR Service. With an upstream setting, it enables the client to send the message.

  1. App server authorized the client.
  2. A CORS check with Azure SignalR Service.
  3. Negotiate a connection with Azure SignalR Service, upgrade to WebSocket.
  4. Start to send messages.

App server authorized the client

The below HTTP content demonstrate the authorization.

Request:

The client sends a token to the app server. The app server checks the token and authorized the client。

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
POST https://sgfuncbidirectional.azurewebsites.net/api/negotiate HTTP/1.1
Host: sgfuncbidirectional.azurewebsites.net
Connection: keep-alive
Content-Length: 0
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/87.0.4280.88 Safari/537.36
X-Requested-With: XMLHttpRequest
Authorization: Bearer
Content-Type: text/plain;charset=UTF-8
Accept: */*
Origin: http://127.0.0.1:5500
Sec-Fetch-Site: cross-site
Sec-Fetch-Mode: cors
Sec-Fetch-Dest: empty
Referer: http://127.0.0.1:5500/
Accept-Encoding: gzip, deflate, br
Accept-Language: zh-CN,zh;q=0.9,en-US;q=0.8,en;q=0.7,de;q=0.6
Cookie: ARRAffinitySameSite=fe426c2b3f59aea53da10ec07007362de3a309eb4eea448e8a780f2c5e203c41

Response:

The app server authorized the client, and return the SiganlR Service URL and a token as a JSON string. The SignalR Service’s URL contains in the URL field, the token contains in the accessToken field.

1
2
3
4
5
6
7
8
9
10
11
HTTP/1.1 200 OK
Content-Length: 394
Content-Type: application/json; charset=utf-8
Vary: Accept-Encoding
Set-Cookie: ARRAffinity=b6ea3b3c934d4da5aa14d06ea7be650e3f33d2bb2477b5e43c7b747af936eb02;Path=/;HttpOnly;Domain=sgfuncbidirectional.azurewebsites.net
Request-Context: appId=cid-v1:ba0fcbf3-a61e-4cc1-9b01-86a0058aef23
Access-Control-Allow-Origin: http://127.0.0.1:5500
Access-Control-Allow-Credentials: true
Date: Mon, 28 Dec 2020 02:51:29 GMT

{"url":"https://sgtest.service.signalr.net/client/?hub=simplechat","accessToken":"eyJhbGciOiJIUzI1NiIsImtpZCI6IjIxMDExMDU4NDEiLCJ0eXAiOiJKV1QifQ.eyJuYW1laWQiOiJTb25pYyBHdW8iLCJleHAiOjE2MDkxMjc0ODksImFkbWluIjp0cnVlLCJuYmYiOjE2MDkxMjM4ODksImlhdCI6MTYwOTEyMzg4OSwiYXVkIjoiaHR0cHM6Ly9zZ3Rlc3Quc2VydmljZS5zaWduYWxyLm5ldC9jbGllbnQvP2h1Yj1zaW1wbGVjaGF0In0.1K3RttJeAg1FP3gi1GYJrd_2y-hWvGZwGQQJRq0d9LM"}

A CORS check with Azure SignalR Service

The client sends a request to the Azure SignalR Service for the CORS check.
The verbose is OPTIONS, URL was provided by the app server.

Request:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
OPTIONS https://sgtest.service.signalr.net/client/negotiate?hub=simplechat HTTP/1.1
Host: sgtest.service.signalr.net
Connection: keep-alive
Accept: */*
Access-Control-Request-Method: POST
Access-Control-Request-Headers: authorization,x-requested-with
Origin: http://127.0.0.1:5500
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/87.0.4280.88 Safari/537.36
Sec-Fetch-Mode: cors
Sec-Fetch-Site: cross-site
Sec-Fetch-Dest: empty
Referer: http://127.0.0.1:5500/
Accept-Encoding: gzip, deflate, br
Accept-Language: zh-CN,zh;q=0.9,en-US;q=0.8,en;q=0.7,de;q=0.6

Response:

If it succeeded, the response seems like the below.

1
2
3
4
5
6
7
8
HTTP/1.1 204 No Content
Server: nginx
Date: Mon, 28 Dec 2020 02:51:29 GMT
Connection: keep-alive
Access-Control-Allow-Credentials: true
Access-Control-Allow-Headers: authorization,x-requested-with
Access-Control-Allow-Methods: POST
Access-Control-Allow-Origin: http://127.0.0.1:5500

Negotiate a connection with Azure SignalR Service, upgrade to WebSocket

In the next action, the client posts the token to negotiate a connection.

Request:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
POST https://sgtest.service.signalr.net/client/negotiate?hub=simplechat HTTP/1.1
Host: sgtest.service.signalr.net
Connection: keep-alive
Content-Length: 0
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/87.0.4280.88 Safari/537.36
X-Requested-With: XMLHttpRequest
Authorization: Bearer eyJhbGciOiJIUzI1NiIsImtpZCI6IjIxMDExMDU4NDEiLCJ0eXAiOiJKV1QifQ.eyJuYW1laWQiOiJTb25pYyBHdW8iLCJleHAiOjE2MDkxMjc0ODksImFkbWluIjp0cnVlLCJuYmYiOjE2MDkxMjM4ODksImlhdCI6MTYwOTEyMzg4OSwiYXVkIjoiaHR0cHM6Ly9zZ3Rlc3Quc2VydmljZS5zaWduYWxyLm5ldC9jbGllbnQvP2h1Yj1zaW1wbGVjaGF0In0.1K3RttJeAg1FP3gi1GYJrd_2y-hWvGZwGQQJRq0d9LM
Content-Type: text/plain;charset=UTF-8
Accept: */*
Origin: http://127.0.0.1:5500
Sec-Fetch-Site: cross-site
Sec-Fetch-Mode: cors
Sec-Fetch-Dest: empty
Referer: http://127.0.0.1:5500/
Accept-Encoding: gzip, deflate, br
Accept-Language: zh-CN,zh;q=0.9,en-US;q=0.8,en;q=0.7,de;q=0.6

Response:

The Azure SignalR Service authorized the client, generate a connectionId VBS_IXc_uyOqFiFOSeGvmAdd6eeb0d1. Also provided a support transport list.

1
2
3
4
5
6
7
8
9
10
11
HTTP/1.1 200 OK
Server: nginx
Date: Mon, 28 Dec 2020 02:51:29 GMT
Content-Type: application/json
Connection: keep-alive
Vary: Accept-Encoding
Access-Control-Allow-Credentials: true
Access-Control-Allow-Origin: http://127.0.0.1:5500
Content-Length: 282

{"negotiateVersion":0,"connectionId":"VBS_IXc_uyOqFiFOSeGvmAdd6eeb0d1","availableTransports":[{"transport":"WebSockets","transferFormats":["Text","Binary"]},{"transport":"ServerSentEvents","transferFormats":["Text"]},{"transport":"LongPolling","transferFormats":["Text","Binary"]}]}

Start to send message

When the client supports the Websocket, its connection will upgrade to WebSocket, it sends Connection: Upgrade. The connection will turn into a persistent connection.

1
2
3
4
5
6
7
8
9
10
11
12
13
GET https://sgtest.service.signalr.net/client/?hub=simplechat&id=VBS_IXc_uyOqFiFOSeGvmAdd6eeb0d1&access_token=eyJhbGciOiJIUzI1NiIsImtpZCI6IjIxMDExMDU4NDEiLCJ0eXAiOiJKV1QifQ.eyJuYW1laWQiOiJTb25pYyBHdW8iLCJleHAiOjE2MDkxMjc0ODksImFkbWluIjp0cnVlLCJuYmYiOjE2MDkxMjM4ODksImlhdCI6MTYwOTEyMzg4OSwiYXVkIjoiaHR0cHM6Ly9zZ3Rlc3Quc2VydmljZS5zaWduYWxyLm5ldC9jbGllbnQvP2h1Yj1zaW1wbGVjaGF0In0.1K3RttJeAg1FP3gi1GYJrd_2y-hWvGZwGQQJRq0d9LM HTTP/1.1
Host: sgtest.service.signalr.net
Connection: Upgrade
Pragma: no-cache
Cache-Control: no-cache
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/87.0.4280.88 Safari/537.36
Upgrade: websocket
Origin: http://127.0.0.1:5500
Sec-WebSocket-Version: 13
Accept-Encoding: gzip, deflate, br
Accept-Language: zh-CN,zh;q=0.9,en-US;q=0.8,en;q=0.7,de;q=0.6
Sec-WebSocket-Key: zT38k+AZ6O5lwJbPp+lhGg==
Sec-WebSocket-Extensions: permessage-deflate; client_max_window_bits

The message sends to the WebSocket looks like below.

websocket

Server Connection

The Server connection is not required. In most of the scenarios, there is no server connection between the Upstream server and the SingnalR Service. Because both REST API and WebSocket are supported in SignalR service management SDK. If using a language other than .NET, you can also manually invoke the REST APIs following this spec.

When the Upstream server uses the SignalR Service Management SDK, and the app server supports Web Socket. The Upstream server can negotiate server connections with the SignalR Service. The server connection is a weak connection. The client connection will not route to the server connection. The reason is, in the serverless mode, the app server doesn’t store any client status. This is different from the Server Mode (default). In the Server Mode, the server connection is a strong connection. The server connection associates with client connections. The connection is session sticky. So the app server ables to store clients’ status.

Communications

Calling the Upstream from the Client

Not like the typical Serverless mode, the clients are able to send messages to the upstream app server. The client sent messages to the SignalR Service. The service then leverages the HTTP protocol to deliver WebSocket messages to Upstream. The dataflow is as below:

Client send message -- Client Connection -> ASRS -- HTTP --> Upstream sever

SignalR Service sends messages to the Upstream follow the following protocols.

Method: POST

Request header

Name Description
X-ASRS-Connection-Id The connection ID for the client connection.
X-ASRS-Hub The hub that the client connection belongs to.
X-ASRS-Category The category that the message belongs to.
X-ASRS-Event The event that the message belongs to.
X-ASRS-Signature A hash-based message authentication code (HMAC) that’s used for validation. See Signature for details.
X-ASRS-User-Claims A group of claims of the client connection.
X-ASRS-User-Id The user identity of the client that sends the message.
X-ASRS-Client-Query The query of the request when clients connect to the service.
Authentication An optional token when you’re using ManagedIdentity.

Request Body

Connected
Content-Type: application/json

Disconnected
Content-Type: application/json

Name Type Description
Error string The error message of a closed connection. Empty when connections close with no error.

Invocation message
Content-Type: application/json or application/x-msgpack

Name Type Description
InvocationId string An optional string that represents an invocation message. Find details in Invocations.
Target string The same as the event and the same as the target in an invocation message.
Arguments Array of object An array that contains arguments to apply to the method referred to in Target.

If the Upstream server is a Web App server, you should extract the HTTP contents in your code. If the Upstream is Function App, you can use SignalR Service trigger binding, it handles these protocols.

Send message from Upstream to client

The Azure SignalR Service tracks clients and has a result that can be used to send messages to a specific client or a set of clients. From the upstream, you can use the REST API to send messages to clients. For the .Net Core Application, you can leverage SignalR service management SDK。

Sample

This is a chatroom sample that demonstrates bidirectional message pushing between Azure SignalR Service and Azure Function in the serverless scenario. It leverages the upstream provided by Azure SignalR Service that features proxying messages from client to upstream endpoints in the serverless scenario. Azure Functions with SignalR trigger binding allows you to write code to receive and push messages in several languages, including JavaScript, Python, C#, etc.

Reference