Streaming firewalled data over a websocket

“I have a brilliant idea” our lead developer said. “To get better audit scores on our hosts, we should avoid having ports open to public by default, so let’s close all ports and use websockets to stream data to the client.”

And so the journey began…

---

First thing we needed was a websocket server to handle websocket connections. To keep the server lightweight and scalable, we decided to write it in GO. The server should be able to handle websocket requests from webclients and from tcpclients, with tcpclients being the hosts dialling out with a request to send data.

Of course there should be some sort of authorization involved while connecting the webclients to our server. To achieve this, we send the initial request for a new stream to our main platform, which performs security checks to see if a given user has access to the requested stream/host/container.

If access is granted, the platform does two things:

• On one hand it creates a websocket url, based on a unique token generated locally, which it returns to the webclient. The client can use this url to connect to the websocket server.
• On the other hand, the platform sets a new message on the message bus ( at the moment NATS in our case), with a command to start a new tcpclient on the host, passing the same token present in the websocket url that was returned to the client. On execution of that command, the host starts the tcpclient, which in his turn uses the included token to dial out to the websocket server. At this moment, there is an active connection from the host to the server. The tcpclient will also make a new tcp connection and will link both connections. ( what comes out one connection goes in the other and vice versa, you get the idea, but check the bottom for the actual code if you’re interested.)

Now when the webclient connects to the websocket server, using the given url, the websocket server checks if there has been a tcp connection request with a token that matches the requested token. If so, the websocket connection from the webclient is again linked to the websocket connection from the tcp client. If not, the webclient connection gets closed.

Mission accomplished. Data from the tcp connection is now streaming to and from the webclient through the websocket server, without opening any ports.

This configuration opens up a number of possibilities, such as opening your VM terminal with spice, or tailing logs in your browser, to name just a couple.

Below the core part of our GO connection copy logic:

func copyConnections(conn1 net.Conn, conn2 net.Conn) {
   doneChan := make(chan bool)
   go keepAlive(conn1, doneChan)
   go keepAlive(conn2, doneChan)
   go copyConnection(conn1, conn2, doneChan)
   copyConnection(conn2, conn1, doneChan)

   doneChan <- true
   debug(info("Copy connections done"))
}

func copyConnection(conn1 net.Conn, conn2 net.Conn, doneChan chan bool) {
   if _, err := io.Copy(conn1, conn2); err != nil {
      debug(fail(err))
      doneChan <- true
   }
}

func debug(message string) {
   if *enableDebug {
      log.Println(message)
   }
}

func keepAlive(conn net.Conn, doneChan chan bool) {
	ticker := time.NewTicker( 90 * time.Second )
	for {
		select{
		case <- doneChan:
			ticker.Stop()
			break
		case <- ticker.C:
			if _, err := conn.Write([]byte{}); err != nil {
				ticker.Stop()
				debug(fail("Connection keepalive failed."))
				break
			}
		}
	}
	doneChan <- true
}