This article is over complicated. If you are not sending binary data (streams, files) – YES use BiSON – but send it as a normal UTF-8 without encoding it to uInt Array

tested only on chrome 16.0.912.75, will check more browsers soon

Subject:

Find more efficient way to exchange JSON data between server and client in multiplayer game based on node.js and websockets.

Background:

After jugling with LZW, huffman’s algorithm and some other string compression concepts I eventually found an idea about replacing text JSON with more binary approach. In my research I managed to decrease bandwidth usage from:

14 KB/s to 11KB/s using LZW
14 KB/s to 9KB/s using BiSON (which I will talk about)
14 KB/s to 6KB/s using BiSON + fixed size packets

Solution:

You will need:

• Ivo Wetzel’s BiSON library
• Node.js websockets implementation with binary transfer support. I recommend using ws. Afaik. overblown Socket.IO don’t even have binary transfer method.

Server code (node.js):

var WebSocketServer = require('ws').Server;

var wss = new WebSocketServer({
  port: 5555,
  host: '0.0.0.0'
});

// some array for connected sockets

CLIENTS_COUNT = 0;
CLIENTS = { };

wss.on("connection", function(socket) {

  console.log("user connected");

  // store new connection so we can use it later

  CLIENTS[++CLIENTS_COUNT] = socket;
  socket.id = CLIENTS_COUNT;

  socket.on("close", function() {
    console.log("user disconnects");
    delete CLIENTS[this.id];
  });

  socket.on("message", function(message) {
    console.log("message from user", message);
  });
 
  // inform all connected clients about newcomer
  for(var i in CLIENTS) {
    wsSendBinary(CLIENTS[i], {
      message: "new user joins",
      foo: "bar"
    });
  }
});

function wsSendBinary(socket, data) {
  // Encode packet with BiSON to safe up to 50% against JSON.stringify
  var bisonPacket = BISON.encode(data);

  // A tricky part is that you will get nothing sending data as-is.
  // Our bison string is build of chars with indexes from 0 to 255.
  // utf-8 uses extra byte to encode char beyond index 127 so u will end up
  // sending value which can be represented by one byte in two bytes.
  // To take advantage of binary transport we will have to convert our data
  // to javascript typed array. Unsigned Int 8 will do best.

  var uint8Packet = new Uint8Array(packet.length);

  for(var i = 0, len = bisonPacket.length; i < len; i++) {
    uint8Packet[i] = packet.charCodeAt(i);
  }

  // This is how sending binary data looks like with ws library
  socket.send(uint8Packet, {binary: true, mask: true});
}

Client code (browser):

var socket = new WebSocket("ws://yourhost:8080");

socket.onmessage = function(event) {
  // Normally you'd expect that event.data is a string, but in
  // binary transfer u get a write-protected Blob of data
  // which can be read as a stream.

  var reader = new FileReader();

  // There is also readAsBinaryString method if you are not using typed arrays
  reader.readAsArrayBuffer(event.data);

  // As the stream finish to load we can use the results
  reader.onloadend = function() {

    // Another tricky part. Before you can read the results you have to create
    // a view for our typed array
    var view = new Uint8Array(this.result);

    // Now let's decode array containing char indexes to normal ol' utf8 string
    var str = "";
    for(var i = 0; i < view.length; i++) {
      str += String.fromCharCode(view[i]);
    }

    // Our string is still BISON encoded, so last conversion needs to be done.
    var message = BISON.decode(str);                   

    // Voilà, here comes object that we sent
    console.log(message);

    return message;
  };
}

This article is over complicated. If you are not sending binary data (streams, files) – YES use BiSON – but send it as a normal UTF-8 without encoding it to...